Hard real-time core software of the AO RTC COSMIC platform: Architecture and performance

With the upcoming giant class of telescopes, Adaptive Optics (AO) has become more essential than ever before to get access to the full potential offered by those telescopes. The complexity of such AO systems is reaching extreme heights, and disruptive developments will have to be made in order to build them. One of the critical component of a AO system is the Real Time Controller (RTC) which will have to compute the slopes and the Deformable Mirror (DM) commands at high frequency, in a range of 0.5 to several kHz. Since the complexity of the computations involved in the RTC is increasing with the size of the telescope, fulfilling RTC requirements for Extremely Large Telescope (ELT) class is a challenge. As an example, the MICADO SCAO (Single Conjugate Adaptive Optics) system requires around 1 TMAC/s for the RTC to get sufficient performance. This complexity brings the need for High Performance Computing (HPC) techniques and standards, such as the use of hardware accelerator like GPU. On top of that, building a RTC is often project-dependent as the components and the interfaces change from one instrument to an other. The COSMIC platforms aims at developing a common AO RTC platform which is meant to be powerful, modular and available to the AO community. This development is a joint effort between Observatoire de Paris and the Australian National University (ANU) in collaboration with the Subaru Telescope. We focus here on the current status of the core hard real-time component of this platform. The H-RTC pipeline is composed of Business Units (BU): each BU is an independent process in charge of one particular operation, such as Matrix Vector Multiply (MVM) or centroid computation, that can be made on CPU or on GPU. BUs read and write data on Shared Memory (SHM) handled by the CACAO framework. Synchronization between each BU can then be made either by using semaphore or by busy waiting on the GPU to ensure very low jitter. The RTC pipeline can then be controlled through a Python interface. One of the key point of this architecture is that the interfaces of a BU with the various SHM is abstracted, so adding a new BU in the collection of available ones is straight forward. This approach allows a high performance, scalable, modular and configurable RTC pipeline that could fit the needs of any AO system configuration. Performance has been measured on a MICADO SCAO scale RTC pipeline with around 25,000 slopes by 5,000 actuators on a DGX-1 system equipped with 8 Tesla V100 GPUs. The considered pipeline is composed of two BUs: the first one takes an input the raw pyramid WFS image (produced by simulator), applies on it dark and flat references, and then extract the useful pixel from the image. The second BU performs the MVM and the integration of the commands following a classical integrator command law. Synchronization between the BU is made through GPU busy waiting on the BU inputs. Performance obtained shows a mean latency up to 235 µs using 4 GPUs, with a jitter of 4.4 µs rms and a maximum jitter of 30 µs