TY - GEN
T1 - Containerizing the telemetry data pipeline for MMTO subsystems
AU - Gibson, J. Duane
AU - Burguillo-Rodriguez, Carlos
AU - Pickering, Timothy
AU - Porter, Dallan
AU - Swindell, Scott
AU - Goble, Will
N1 - Publisher Copyright:
© COPYRIGHT SPIE. Downloading of the abstract is permitted for personal use only.
PY - 2022
Y1 - 2022
N2 - The telemetry data pipeline for the MMT Observatory (MMTO) describes the flow of data sampled from diverse hardware devices within MMTO subsystems, through logging into various databases, to user interfaces and monitoring services. Subsystems within the pipeline include the telescope mount, primary and secondary mirrors, instruments, and environmental sensors. Data acquisition services within the pipeline post new data with a uniform data structure to a master Redis server. These incoming data are transported in real-time to replicated Redis servers where they are logged into local MariaDB relational databases. Database tables for logged data from the subsystems are highly optimized for data storage, allowing the archival of billions of data points for thousands of parameters over the past 10-15 years. Because of ever increasing difficulty in supporting legacy servers and software, a large-scale containerization effort is underway of the various components of the telemetry pipeline and underlying cyberinfrastructure. These critical servers and services are single points of failure that could result in up to weeks of operational downtime. Containerization helps to reduce the risk of potential hardware failure, operating system upgrades, and software incompatibilities. Containerizing a service defines all the software requirements for that service, including the code, runtime, system tools, system libraries, and settings. It allows rapid and reliable redeployment of new and legacy services with minimal concern for the underlying hardware. Finally, a summary of the ongoing and planned future work is presented.
AB - The telemetry data pipeline for the MMT Observatory (MMTO) describes the flow of data sampled from diverse hardware devices within MMTO subsystems, through logging into various databases, to user interfaces and monitoring services. Subsystems within the pipeline include the telescope mount, primary and secondary mirrors, instruments, and environmental sensors. Data acquisition services within the pipeline post new data with a uniform data structure to a master Redis server. These incoming data are transported in real-time to replicated Redis servers where they are logged into local MariaDB relational databases. Database tables for logged data from the subsystems are highly optimized for data storage, allowing the archival of billions of data points for thousands of parameters over the past 10-15 years. Because of ever increasing difficulty in supporting legacy servers and software, a large-scale containerization effort is underway of the various components of the telemetry pipeline and underlying cyberinfrastructure. These critical servers and services are single points of failure that could result in up to weeks of operational downtime. Containerization helps to reduce the risk of potential hardware failure, operating system upgrades, and software incompatibilities. Containerizing a service defines all the software requirements for that service, including the code, runtime, system tools, system libraries, and settings. It allows rapid and reliable redeployment of new and legacy services with minimal concern for the underlying hardware. Finally, a summary of the ongoing and planned future work is presented.
KW - Containerization
KW - Cyberinfrastructure
KW - Data Logging
KW - Databases
KW - Docker
KW - MMT Observatory
KW - Telemetry
UR - http://www.scopus.com/inward/record.url?scp=85140086352&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85140086352&partnerID=8YFLogxK
U2 - 10.1117/12.2626505
DO - 10.1117/12.2626505
M3 - Conference contribution
AN - SCOPUS:85140086352
T3 - Proceedings of SPIE - The International Society for Optical Engineering
BT - Software and Cyberinfrastructure for Astronomy VII
A2 - Ibsen, Jorge
A2 - Chiozzi, Gianluca
PB - SPIE
T2 - Software and Cyberinfrastructure for Astronomy VII 2022
Y2 - 17 July 2022 through 21 July 2022
ER -