Personalized alertness prediction using video-based ocular and facial features

Study Objectives: Alertness impairment is generally assessed by the psychomotor vigilance test (PVT). However, performing a PVT in the real world is not practical because it is time-consuming and interrupts everyday activities. Here, we aimed to replace the PVT with passively recorded facial videos and use these measurements to make personalized alertness-impairment predictions. Methods: We retrospectively analyzed data from a 62-hour total sleep deprivation (TSD) challenge involving 26 healthy young adults (14 men), where every 3 hours they performed a 5-minute PVT followed by a 3-minute video recording of the face. We then extracted ocular and facial features from the first 1 minute of the videos, used the features to train linear mixed-effects models that predicted PVT mean reaction times, and used the predicted PVT to customize the unified model of performance (UMP) and make personalized alertness-impairment predictions for each participant. Results: For the mixed-effects models, the average root mean square error (RMSE) between the measured and predicted PVT data was 39 ms (standard deviation, 9 ms). For the personalized UMP predictions based on PVT predicted from the videos, the average RMSE between the measured PVT data and the model-predicted alertness impairment was 36 ms (standard error, 5 ms), which is nearly indistinguishable from the within-participant variability of 30 ms for PVT mean reaction time under rested conditions. Conclusions: As a proof of principle, we developed a practical approach for predicting an individual’s alertness impairment using passively recorded facial videos.