Robust Waypoint Guidance of a Hexacopter on Mars using Meta-Reinforcement Learning

This paper presents a meta-reinforcement learning approach to the robust and autonomous waypoint guidance of a six-rotor unmanned aerial vehicle in Mars’ atmosphere. The metalearning is implemented by using a recurrent neural network as a control policy to map data about the hexacopter state provided by onboard sensors to the six rotor angular speeds. The network is trained by proximal policy optimization, a state-of-the-art policy gradient reinforcement learning algorithm. During the training, the network is also provided with information about the previous control output and reward, to improve the policy adaptability to different environment instances. Several mission scenarios, involving uncertainties on Mars’ atmosphere’s properties, the presence of random wind gusts, and Gaussian noise on the collected sensor data, are investigated to assess the robustness of the proposed approach in realistic operative conditions. The flexibility and performance of meta-reinforcement learning are also compared against standard reinforcement learning with a fully-connected neural network, to better highlight the potential of the proposed methodology in real-world autonomous guidance applications.