Author(s)

PingAn Ren¹, EnHui Ren², YiFan Qu^3*

Affiliation(s)

¹School of Intelligent Engineering, Xiangtan Institute of Technology, Xiangtan 411100, Hunan, China.

²School of Computer Science and Technology, Zhoukou Normal University, Zhoukou 466001, Henan, China.

³School of Automation, Harbin University of Science and Technology, Harbin 150080, Heilongjiang, China.

Corresponding Author

YiFan Qu

ABSTRACT

This study presents a quadrotor UAV tracking control algorithm that addresses the issues of parameter uncertainty and external disturbances during trajectory tracking. The algorithm combines the Q-Learning reinforcement learning algorithm with a nonsingular terminal sliding mode controller. Firstly, a four-rotor UAV model based on tracking error is defined, and the coupling and external interference between channels are converted into lumped interference. Extended state observers are designed for estimation and compensation of lumped interference by the outer loop position subsystem and the inner loop attitude subsystem. At the same time, a fast non-singular terminal sliding mode UAV controller is constructed, which includes an outer loop position controller and an inner loop attitude controller. Then, a Q-learning algorithm based on fuzzy strategy is proposed to realize the adaptive adjustment of the key parameters of the controller and the observer. By designing the reward function, the Q values of the UAV under different flight states are iteratively optimized and the Q table is constantly updated. Finally, the trained Q table is used for drone control. The algorithm can not only save the complicated process of manual parameter adjustment, but also realize the adaptive adjustment of key parameters in the face of different flight environments and flight states. The simulation and comparison experiments show that the proposed algorithm has a higher degree of fit with the reference trajectory in the trajectory tracking control process and has good robustness.

KEYWORDS

Quadrotor UAV; Trajectory tracking control; Fast nonsingular terminal sliding mode; Reinforcement learning; Extended state observer

CITE THIS PAPER

PingAn Ren, EnHui Ren, YiFan Qu. Anti-interference trajectory tracking control of quadrotor UAV based on reinforcement learning. Journal of Computer Science and Electrical Engineering. 2026, 8(1): 32-40. DOI: https://doi.org/10.61784/jcsee3115.

REFERENCES

[1] Bennaceu S, Azouz N. Modelling and control of a quadrotor with flexible arms. Alexandria Engineering Journal, 2023, 65: 209-231.

[2] Xu Z, Fan L, Qiu W, et al. A robust disturbance rejection controller using model predictive control for quadrotor UAV in tracking aggressive trajectory. Drones, 2023, 7(6): 369.

[3] Yesmin A, Sinha A. Sliding mode controller for quadcopter UAVs: a comprehensive survey. Drones, 2025, 9(9): 625.

[4] Quan Q. Design and control of multi-rotor UAV. Beijing: Electronic Industry Press, 2017: 225-275.

[5] Zuo Z Y. Trajectory tracking control design with command-filtered compensation for quadrotor. IET Control Theory & Applications, 2010, 4(11): 2343-2355.

[6] Pounds P E I, Bersak D R, Dollar A M. Stability of small-scale UAV helicopters and quadrotors with added payload mass under PID control. Autonomous Robots, 2012, 33(1-2): 129-142.

[7] Wang C Y, Hu S B, Gao F H. Double fuzzy sliding mode control of four-rotor UAV with hanging load. Computer Era, 2021(12): 14-21.

[8] Zhao Z H, Li T, Jiang B. Compound continuous fast non-singular terminal sliding mode control for four-rotor UAV attitude system. Control Theory & Applications, 2023, 40(3): 459-467.

[9] Cheng X, Tang G, Wang P, et al. Predictive sliding mode control for attitude tracking of hypersonic vehicles using fuzzy disturbance observer. Mathematical Problems in Engineering, 2023, 2023(1): 1-8.

[10]  Abadi El, Amraoui A, Mekki H, et al. Robust tracking control of quadrotor based on flatness and active disturbance rejection control. IET Control Theory & Applications, 2020, 14(8): 1057-1068.

[11] Lhayani M, Abbou A, El Houm Y. Trajectory tracking control of quadrotors using nonsingular fast fixed time sliding mode and fixed time extended state observer. International Journal of Intelligent Engineering and Systems, 2025, 18(10): 123-134.

[12] Lin Z, Li P. Nonsingular fast terminal sliding mode attitude control for a quadrotor based on fuzzy extended state observer//Proceedings of the 33rd Chinese Control and Decision Conference. Nanchang: IEEE, 2019: 1717-1723.

[13] Nikitin D A. Large scale systems control. Automation and Remote Control, 2021, 80(9): 1717-1733.

[14] Yoo J, Jang D, Kim H J, et al. Hybrid reinforcement learning control for a micro quadrotor flight. IEEE Control Systems Letters, 2021, 5: 155-182.

[15] Li M, Cai Z, Zhao J, et al. Disturbance rejection and high dynamic quadrotor control based on reinforcement learning and supervised learning. Neural Computing and Applications, 2012, 34(13): 1141-1161.

[16] Zhang Z, Yang H, Fei Y, et al. Control of UAV quadrotor using reinforcement learning and robust controller. IET Control Theory & Applications, 2021, 15(10): 1599-1615.