Transparent deep learning to identify autism spectrum disorders (ASD) in EHR using clinical notes

Gondy Leroy, Jennifer G. Andrews, Madison Kealohi-Preece, Ajay Jaswani, Hyunju Song, Maureen Kelly Galindo, Sydney A. Rice

Research output: Contribution to journalArticlepeer-review


Objective: Machine learning (ML) is increasingly employed to diagnose medical conditions, with algorithms trained to assign a single label using a black-box approach. We created an ML approach using deep learning that generates outcomes that are transparent and in line with clinical, diagnostic rules. We demonstrate our approach for autism spectrum disorders (ASD), a neurodevelopmental condition with increasing prevalence. Methods: We use unstructured data from the Centers for Disease Control and Prevention (CDC) surveillance records labeled by a CDC-trained clinician with ASD A1-3 and B1-4 criterion labels per sentence and with ASD cases labels per record using Diagnostic and Statistical Manual of Mental Disorders (DSM5) rules. One rule-based and three deep ML algorithms and six ensembles were compared and evaluated using a test set with 6773 sentences (N = 35 cases) set aside in advance. Criterion and case labeling were evaluated for each ML algorithm and ensemble. Case labeling outcomes were compared also with seven traditional tests. Results: Performance for criterion labeling was highest for the hybrid BiLSTM ML model. The best case labeling was achieved by an ensemble of two BiLSTM ML models using a majority vote. It achieved 100% precision (or PPV), 83% recall (or sensitivity), 100% specificity, 91% accuracy, and 0.91 F-measure. A comparison with existing diagnostic tests shows that our best ensemble was more accurate overall. Conclusions: Transparent ML is achievable even with small datasets. By focusing on intermediate steps, deep ML can provide transparent decisions. By leveraging data redundancies, ML errors at the intermediate level have a low impact on final outcomes.

Original languageEnglish (US)
Pages (from-to)1313-1321
Number of pages9
JournalJournal of the American Medical Informatics Association
Issue number6
StatePublished - Jun 1 2024


  • autism spectrum disorders
  • deep learning
  • machine learning
  • natural language processing
  • transparency

ASJC Scopus subject areas

  • Health Informatics


Dive into the research topics of 'Transparent deep learning to identify autism spectrum disorders (ASD) in EHR using clinical notes'. Together they form a unique fingerprint.

Cite this