Deep learning of 3D computed tomography (CT) images for organ segmentation using 2D multi-channel SegNet model

Yingzhou Liu, Wanyi Fu, Vignesh Selvakumaran, Matthew Phelan, W. Paul Segars, Ehsan Samei, Maciej Mazurowski, Joseph Y. Lo, Geoffrey D. Rubin, Ricardo Henao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations


Purpose To accurately segment organs from 3D CT image volumes using a 2D, multi-channel SegNet model consisting of a deep Convolutional Neural Network (CNN) encoder-decoder architecture. Method We trained a SegNet model on the extended cardiac-Torso (XCAT) dataset, which was previously constructed based on patient Chest-Abdomen-Pelvis (CAP) Computed Tomography (CT) studies from 50 Duke patients. Each study consists of one low-resolution (5-mm section thickness) 3D CT image volume and its corresponding 3D, manually labeled volume. To improve modeling on such small sample size regime, we performed median frequency class balancing weighting in the loss function of the SegNet, data normalization adjusting for intensity coverage of CT volumes, data transformation to harmonize voxel resolution, CT section extrapolation to virtually increase the number of transverse sections available as inputs to the 2D multi-channel model, and data augmentation to simulate mildly rotated volumes. To assess model performance, we calculated Dice coefficients on a held-out test set, as well as qualitative evaluation of segmentation on high-resolution CTs. Further, we incorporated 50 patients high-resolution CTs with manually-labeled kidney segmentation masks for the purpose of quantitatively evaluating the performance of our XCAT trained segmentation model. The entire study was conducted from raw, identifiable data within the Duke Protected Analytics Computing Environment (PACE). Result We achieved median Dice coefficients over 0.8 for most organs and structures on XCAT test instances and observed good performance on additional images without manual segmentation labels, qualitatively evaluated by Duke Radiology experts. Moreover, we achieved 0.89 median Dice Coefficients for kidneys on high-resolution CTs. Conclusion 2D, multi-channel models like SegNet are effective for organ segmentations of 3D CT image volumes, achieving high segmentation accuracies.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2019
Subtitle of host publicationImaging Informatics for Healthcare, Research, and Applications
EditorsPo-Hao Chen, Peter R. Bak
ISBN (Electronic)9781510625556
StatePublished - 2019
Externally publishedYes
EventMedical Imaging 2019: Imaging Informatics for Healthcare, Research, and Applications - San Diego, United States
Duration: Feb 17 2019Feb 18 2019

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2019: Imaging Informatics for Healthcare, Research, and Applications
Country/TerritoryUnited States
CitySan Diego


  • Deep Learning
  • Dice Coefficient
  • Kidneys
  • PACE
  • SegNet
  • XCAT

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Deep learning of 3D computed tomography (CT) images for organ segmentation using 2D multi-channel SegNet model'. Together they form a unique fingerprint.

Cite this