Meta-reinforcement learning for adaptive spacecraft guidance during finite-thrust rendezvous missions

Lorenzo Federici, Andrea Scorsoglio, Alessandro Zavoli, Roberto Furfaro

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

In this paper, a meta-reinforcement learning approach is investigated to design an adaptive guidance algorithm capable of carrying out multiple rendezvous space missions. Specifically, both a standard fully-connected network and a recurrent neural network are trained by proximal policy optimization on a wide distribution of finite-thrust rendezvous transfers between circular co-planar orbits. The recurrent network is also provided with the control and reward at the previous simulation step, thus allowing it to build, thanks to its history-dependent state, an internal representation of the considered task distribution. The ultimate goal is to generate a model which could adapt to unseen tasks and produce a nearly-optimal guidance law along any transfer leg of a multi-target mission. As a first step towards the solution of a complete multi-target problem, a sensitivity analysis on the single rendezvous leg is carried out in this paper, by varying the radius either of the initial or the final orbit, the transfer time, and the initial phasing between the chaser and the target. Numerical results show that the recurrent-network-based meta-reinforcement learning approach is able to better reconstruct the optimal control in almost all the analyzed scenarios, and, at the same time, to meet, with greater accuracy, the terminal rendezvous condition, even when considering problem instances that fall outside the original training domain.

Original languageEnglish (US)
Pages (from-to)129-141
Number of pages13
JournalActa Astronautica
Volume201
DOIs
StatePublished - Dec 2022

Keywords

  • Autonomous spacecraft guidance
  • Meta-reinforcement learning
  • Optimal control
  • Proximal policy optimization
  • Recurrent neural network
  • Rendezvous mission

ASJC Scopus subject areas

  • Aerospace Engineering

Fingerprint

Dive into the research topics of 'Meta-reinforcement learning for adaptive spacecraft guidance during finite-thrust rendezvous missions'. Together they form a unique fingerprint.

Cite this