XPipeline: Starlight subtraction at scale for MagAO-X

Joseph D. Long; Jared R. Males; Sebastiaan Y. Haffert; Laird M. Close; Katie M. Morzinski; Kyle Van Gorkom; Jennifer Lumbres; Warren Foster; Alexander Hedglen; Maggie Kautz; Alex Rodack; Lauren Schatz; Kelsey Miller; David Doelman; Steven Bos; Matthew A. Kenworthy; Frans Snik; Gilles P.P.L. Otten

doi:10.1117/12.2628975

XPipeline: Starlight subtraction at scale for MagAO-X

Joseph D. Long, Jared R. Males, Sebastiaan Y. Haffert, Laird M. Close, Katie M. Morzinski, Kyle Van Gorkom, Jennifer Lumbres, Warren Foster, Alexander Hedglen, Maggie Kautz, Alex Rodack, Lauren Schatz, Kelsey Miller, David Doelman, Steven Bos, Matthew A. Kenworthy, Frans Snik, Gilles P.P.L. Otten

Astronomy

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

MagAO-X is an extreme adaptive optics (ExAO) instrument for the Magellan Clay 6.5-meter telescope at Las Campanas Observatory in Chile. Its high spatial and temporal resolution can produce data rates of 1 TB/hr or more, including all AO system telemetry and science images. We describe the tools and architecture we use for commanding, telemetry, and science data transmission and storage. The high data volumes require a distributed approach to data processing, and we have developed a pipeline that can scale from a single laptop to dozens of HPC nodes. The same codebase can then be used for both quick-look functionality at the telescope and for post-processing. We present the software and infrastructure we have developed for ExAO data post-processing, and illustrate their use with recently acquired direct-imaging data.

Original language	English (US)
Title of host publication	Adaptive Optics Systems VIII
Editors	Laura Schreiber, Dirk Schmidt, Elise Vernet
Publisher	SPIE
ISBN (Electronic)	9781510653511
DOIs	https://doi.org/10.1117/12.2628975
State	Published - 2022
Event	Adaptive Optics Systems VIII 2022 - Montreal, Canada Duration: Jul 17 2022 → Jul 22 2022

Publication series

Name	Proceedings of SPIE - The International Society for Optical Engineering
Volume	12185
ISSN (Print)	0277-786X
ISSN (Electronic)	1996-756X

Conference

Conference	Adaptive Optics Systems VIII 2022
Country/Territory	Canada
City	Montreal
Period	7/17/22 → 7/22/22

Keywords

coronagraphy
high-performance computing
post-processing

ASJC Scopus subject areas

Electronic, Optical and Magnetic Materials
Condensed Matter Physics
Computer Science Applications
Applied Mathematics
Electrical and Electronic Engineering

Access to Document

10.1117/12.2628975

Cite this

Long, J. D., Males, J. R., Haffert, S. Y., Close, L. M., Morzinski, K. M., Van Gorkom, K., Lumbres, J., Foster, W., Hedglen, A., Kautz, M., Rodack, A., Schatz, L., Miller, K., Doelman, D., Bos, S., Kenworthy, M. A., Snik, F., & Otten, G. P. P. L. (2022). XPipeline: Starlight subtraction at scale for MagAO-X. In L. Schreiber, D. Schmidt, & E. Vernet (Eds.), Adaptive Optics Systems VIII Article 121853P (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 12185). SPIE. https://doi.org/10.1117/12.2628975

XPipeline: Starlight subtraction at scale for MagAO-X. / Long, Joseph D.; Males, Jared R.; Haffert, Sebastiaan Y. et al.
Adaptive Optics Systems VIII. ed. / Laura Schreiber; Dirk Schmidt; Elise Vernet. SPIE, 2022. 121853P (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 12185).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Long, JD, Males, JR, Haffert, SY, Close, LM, Morzinski, KM, Van Gorkom, K, Lumbres, J, Foster, W, Hedglen, A, Kautz, M, Rodack, A, Schatz, L, Miller, K, Doelman, D, Bos, S, Kenworthy, MA, Snik, F & Otten, GPPL 2022, XPipeline: Starlight subtraction at scale for MagAO-X. in L Schreiber, D Schmidt & E Vernet (eds), Adaptive Optics Systems VIII., 121853P, Proceedings of SPIE - The International Society for Optical Engineering, vol. 12185, SPIE, Adaptive Optics Systems VIII 2022, Montreal, Canada, 7/17/22. https://doi.org/10.1117/12.2628975

@inproceedings{ef9849ef255749bb8013b34a8f761caf,

title = "XPipeline: Starlight subtraction at scale for MagAO-X",

abstract = "MagAO-X is an extreme adaptive optics (ExAO) instrument for the Magellan Clay 6.5-meter telescope at Las Campanas Observatory in Chile. Its high spatial and temporal resolution can produce data rates of 1 TB/hr or more, including all AO system telemetry and science images. We describe the tools and architecture we use for commanding, telemetry, and science data transmission and storage. The high data volumes require a distributed approach to data processing, and we have developed a pipeline that can scale from a single laptop to dozens of HPC nodes. The same codebase can then be used for both quick-look functionality at the telescope and for post-processing. We present the software and infrastructure we have developed for ExAO data post-processing, and illustrate their use with recently acquired direct-imaging data.",

keywords = "coronagraphy, high-performance computing, post-processing",

author = "Long, {Joseph D.} and Males, {Jared R.} and Haffert, {Sebastiaan Y.} and Close, {Laird M.} and Morzinski, {Katie M.} and {Van Gorkom}, Kyle and Jennifer Lumbres and Warren Foster and Alexander Hedglen and Maggie Kautz and Alex Rodack and Lauren Schatz and Kelsey Miller and David Doelman and Steven Bos and Kenworthy, {Matthew A.} and Frans Snik and Otten, {Gilles P.P.L.}",

note = "Publisher Copyright: {\textcopyright} 2022 SPIE.; Adaptive Optics Systems VIII 2022 ; Conference date: 17-07-2022 Through 22-07-2022",

year = "2022",

doi = "10.1117/12.2628975",

language = "English (US)",

series = "Proceedings of SPIE - The International Society for Optical Engineering",

publisher = "SPIE",

editor = "Laura Schreiber and Dirk Schmidt and Elise Vernet",

booktitle = "Adaptive Optics Systems VIII",

}

TY - GEN

T1 - XPipeline

T2 - Adaptive Optics Systems VIII 2022

AU - Long, Joseph D.

AU - Males, Jared R.

AU - Haffert, Sebastiaan Y.

AU - Close, Laird M.

AU - Morzinski, Katie M.

AU - Van Gorkom, Kyle

AU - Lumbres, Jennifer

AU - Foster, Warren

AU - Hedglen, Alexander

AU - Kautz, Maggie

AU - Rodack, Alex

AU - Schatz, Lauren

AU - Miller, Kelsey

AU - Doelman, David

AU - Bos, Steven

AU - Kenworthy, Matthew A.

AU - Snik, Frans

AU - Otten, Gilles P.P.L.

PY - 2022

Y1 - 2022

N2 - MagAO-X is an extreme adaptive optics (ExAO) instrument for the Magellan Clay 6.5-meter telescope at Las Campanas Observatory in Chile. Its high spatial and temporal resolution can produce data rates of 1 TB/hr or more, including all AO system telemetry and science images. We describe the tools and architecture we use for commanding, telemetry, and science data transmission and storage. The high data volumes require a distributed approach to data processing, and we have developed a pipeline that can scale from a single laptop to dozens of HPC nodes. The same codebase can then be used for both quick-look functionality at the telescope and for post-processing. We present the software and infrastructure we have developed for ExAO data post-processing, and illustrate their use with recently acquired direct-imaging data.

AB - MagAO-X is an extreme adaptive optics (ExAO) instrument for the Magellan Clay 6.5-meter telescope at Las Campanas Observatory in Chile. Its high spatial and temporal resolution can produce data rates of 1 TB/hr or more, including all AO system telemetry and science images. We describe the tools and architecture we use for commanding, telemetry, and science data transmission and storage. The high data volumes require a distributed approach to data processing, and we have developed a pipeline that can scale from a single laptop to dozens of HPC nodes. The same codebase can then be used for both quick-look functionality at the telescope and for post-processing. We present the software and infrastructure we have developed for ExAO data post-processing, and illustrate their use with recently acquired direct-imaging data.

KW - coronagraphy

KW - high-performance computing

KW - post-processing

UR - http://www.scopus.com/inward/record.url?scp=85178508773&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85178508773&partnerID=8YFLogxK

U2 - 10.1117/12.2628975

DO - 10.1117/12.2628975

M3 - Conference contribution

AN - SCOPUS:85178508773

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - Adaptive Optics Systems VIII

A2 - Schreiber, Laura

A2 - Schmidt, Dirk

A2 - Vernet, Elise

PB - SPIE

Y2 - 17 July 2022 through 22 July 2022

ER -

XPipeline: Starlight subtraction at scale for MagAO-X

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this