Investigating machine learning algorithms to classify label-free images of pancreatic neuroendocrine neoplasms

Significance: Pancreatic neuroendocrine neoplasms (PNENs) are an uncommon cancer whose incidence rate has increased dramatically in recent years. Surgery is the only potentially curative treatment, which relies on both preoperative tumor localization and postoperative margin definition using histopathological examination for decision making. If pathology could be automated, valuable time and resources could be saved. Aim: In this study, we investigate the ability of machine learning (ML) with handcrafted features, as well as deep learning, to classify label-free microscopy images of PNENs as a first step toward automated pathology of such tumors. Approach: Patient samples of two different preparation types were imaged, and ML and convolutional neural networks (CNNs) were developed to test the ability of such algorithms to classify PNENs. Results: Our classification algorithms were able to distinguish PNENs from normal tissue with high accuracy using multiphoton microscopy (MPM) images, regardless of sample preparation. Using a combined FFPE and fixed frozen dataset, we achieved an AUC value of 0.793 and an accuracy of 80.6% with ML, and an AUC value of 0.977 and an accuracy of 96.43% using CNNs. Conclusions: Label-free MPM combined with deep learning can provide fast, accurate classification of PNENs. With the ability to assess margins rapidly and potentially automatically, both disease recurrence and the need for resections after initial surgery could be reduced.