TY - JOUR
T1 - Rigorous performance evaluation (previously, “validation”) for informed use of new technologies for sleep health measurement
AU - de Zambotti, Massimiliano
AU - Menghini, Luca
AU - Grandner, Michael A.
AU - Redline, Susan
AU - Zhang, Ying
AU - Wallace, Meredith L.
AU - Buxton, Orfeu M.
N1 - Funding Information:
MdZ has received funding from Noctrix Health, Inc., Lisa Health Inc., Verily Life Sciences LLC., the National Science Foundation (NSF) IIP-2111818; he has patents issued: US222921894, US20140316191, and pending: EP3591659, JP2020014841, US20200013511, US20200222699, AU2018302101, EP3654839, WO2019018400, WO2022006119 WO2022032121. MG received grants from Kemin Foods, Jazz Pharmaceuticals, and CeraZ; he has received consulting fees from Idorsia Pharmaceuticals, Fitbit, Smartypants Vitamins, Natrol, National Sleep Foundation, American Sleep Medicine Foundation; He has received publishing royalties from Elsevier; he has received speaking honoraria and/or travel support from New York University, University of Maryland, Stavis & Cohen, University of Miami, Clinical Education Alliance, and Jazz Pharmaceuticals. SR received consulting fees from Jazz Pharma and Eli Lilly, and has received consulting fees for serving as a member of the Scientific Advisory Board for Apnimed, Inc. Outside of the current work, OMB received subcontract grants to Penn State from Proactive Life LLC (formerly Mobile Sleep Technologies) doing business as SleepScape (NSF/STTR #1622766, NIH/NIA SBIR R43-AG056250, R44-AG056250), received honoraria/travel support for lectures from Boston University, Boston College, Tufts School of Dental Medicine, New York University, University of Miami, and Allstate, consulting fees from SleepNumber, and receives an honorarium for his role as the Editor-in-Chief of Sleep Health (sleephealthjournal.org). MLW received consulting fees from HealthRhythm, Sleep Number Corporation, and Noctem, unrelated to this work. Outside of the current work, MAG received grants from Kemin Foods, Jazz Pharmaceuticals, and CeraZ; he has received consulting fees from Idorsia Pharmaceuticals, Fitbit, Smartypants Vitamins, Natrol, National Sleep Foundation, American Sleep Medicine Foundation; he has received publishing fees from Elsevier; and he has received speaking honoraria and/or travel support from New York University, University of Maryland, Stavis & Cohen, and Jazz Pharmaceuticals. SR and YZ are supported by NIH contract, 75N92019C00011; National Sleep Research Resource. The remaining authors have no conflicts to disclose.
Publisher Copyright:
© 2022 The Authors
PY - 2022/6
Y1 - 2022/6
N2 - New sleep technologies have become pervasive in the consumer space, and are becoming highly common in research and clinical sleep settings. The rapid, widespread use of largely unregulated and unstandardized technology has enabled the quantification of many different facets of sleep health, driving scientific discovery. As sleep scientists, it is our responsibility to inform principles and practices for proper evaluation of any new technology used in the clinical and research settings, and by consumers. A current lack of standardized methods for evaluating technology performance challenges the rigor of our scientific methods for accurate representation of the sleep health facets of interest. This special article describes the rationale and priorities of an interdisciplinary effort for rigorous, standardized, and rapid performance evaluation (previously, “validation”) of new sleep and sleep disorders related technologies of all kinds (eg, devices or algorithms), including an associated article template for a new initiative for publication in Sleep Health of empirical studies systematically evaluating the performance of new sleep technologies. A structured article type should streamline manuscript development and enable more rapid writing, review, and publication. The goal is to promote rapid and rigorous evaluation and dissemination of new sleep technology, to enhance sleep research integrity, and to standardize terminology used in Rigorous Performance Evaluation papers to prevent misinterpretation while facilitating comparisons across technologies.
AB - New sleep technologies have become pervasive in the consumer space, and are becoming highly common in research and clinical sleep settings. The rapid, widespread use of largely unregulated and unstandardized technology has enabled the quantification of many different facets of sleep health, driving scientific discovery. As sleep scientists, it is our responsibility to inform principles and practices for proper evaluation of any new technology used in the clinical and research settings, and by consumers. A current lack of standardized methods for evaluating technology performance challenges the rigor of our scientific methods for accurate representation of the sleep health facets of interest. This special article describes the rationale and priorities of an interdisciplinary effort for rigorous, standardized, and rapid performance evaluation (previously, “validation”) of new sleep and sleep disorders related technologies of all kinds (eg, devices or algorithms), including an associated article template for a new initiative for publication in Sleep Health of empirical studies systematically evaluating the performance of new sleep technologies. A structured article type should streamline manuscript development and enable more rapid writing, review, and publication. The goal is to promote rapid and rigorous evaluation and dissemination of new sleep technology, to enhance sleep research integrity, and to standardize terminology used in Rigorous Performance Evaluation papers to prevent misinterpretation while facilitating comparisons across technologies.
KW - Actigraphy
KW - Algorithms
KW - Machine learning
KW - Performance evaluations
KW - Sensors
KW - Wearables
UR - http://www.scopus.com/inward/record.url?scp=85129409295&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85129409295&partnerID=8YFLogxK
U2 - 10.1016/j.sleh.2022.02.006
DO - 10.1016/j.sleh.2022.02.006
M3 - Article
C2 - 35513978
AN - SCOPUS:85129409295
SN - 2352-7218
VL - 8
SP - 263
EP - 269
JO - Sleep Health
JF - Sleep Health
IS - 3
ER -