The automated data processing architecture for the GPI Exoplanet Survey

Jason J. Wang, Marshall D. Perrin, Dmitry Savransky, Pauline Arriaga, Jeffrey K. Chilcote, Robert J. De Rosa, Maxwell A. Millar-Blanchaer, Christian Marois, Julien Rameau, Schuyler G. Wolff, Jacob Shapiro, Jean Baptiste Ruffio, James R. Graham, Bruce Macintosh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

The Gemini Planet Imager Exoplanet Survey (GPIES) is a multi-year direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the GPIES Data Cruncher, combines multiple data reduction pipelines together to intelligently process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow-up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our data reduction pipelines. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real-time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.

Original languageEnglish (US)
Title of host publicationTechniques and Instrumentation for Detection of Exoplanets VIII
EditorsStuart Shaklan
PublisherSPIE
ISBN (Electronic)9781510612570
DOIs
StatePublished - 2017
Externally publishedYes
EventTechniques and Instrumentation for Detection of Exoplanets VIII 2017 - San Diego, United States
Duration: Aug 8 2017Aug 10 2017

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume10400
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Other

OtherTechniques and Instrumentation for Detection of Exoplanets VIII 2017
Country/TerritoryUnited States
CitySan Diego
Period8/8/178/10/17

Keywords

  • circumstellar disks
  • Data Cruncher
  • data processing
  • Dropbox
  • exoplanets
  • Flask
  • Gemini Planet Imager
  • GPI
  • high contrast imaging
  • Slack

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'The automated data processing architecture for the GPI Exoplanet Survey'. Together they form a unique fingerprint.

Cite this