There are many mature traditional navigation algorithms, but most of them are insufficient in the function of environment perception and understanding, and reinforcement learning can give robots the ability to learn and make decisions. This paper proposes a robot reinforcement learning navigation algorithm and optimal control strategy based on deep reinforcement learning. Firstly, Markov decision modeling for local planning of the robot navigation system is implemented, and then a POMDP belief space dimensionality reduction algorithm based on the NMF update rule is proposed to address the situation of excessive dimensionality and combined with PRM to achieve global reinforcement learning planning. Finally, considering the external information interference problem, a power controller based on the TD3 algorithm is designed to ensure that the robot navigation system can accurately track the signals even under the external interference environment.The position error of the robot under the TD3 controller tends to be close to 0, which is much lower than that of the robot under the PD controller. The experimental results of this paper show that the designed TD3 controller can effectively improve the trajectory tracking accuracy of the robot navigation system and better realize the optimization of the robot tracking control function.