Hierarchical, model-based merging of multiple fragments for improved three-dimensional segmentation of nuclei

Gang Lin, Monica K. Chawla, Kathy Olson, John F. Guzowski, Carol A. Barnes, Badrinath Roysam

Research output: Contribution to journalArticlepeer-review

96 Scopus citations


Background: Automated segmentation of fluorescently labeled cell nuclei in three-dimensional confocal images is essential for numerous studies, e.g., spatiotemporal fluorescence in situ hybridization quantification of immediate early gene transcription. High accuracy and automation levels are required in high-throughput and large-scale studies. Common sources of segmentation error include tight clustering and fragmentation of nuclei. Previous region-based methods are limited because they perform merging of two nuclear fragments at a time. To achieve higher accuracy without sacrificing scale, more sophisticated yet computationally efficient algorithms are needed. Methods: A recursive tree-based algorithm that can consider multiple object fragments simultaneously is described. Starting with oversegmented data, it searches efficiently for the optimal merging pattern guided by a quantitative scoring criterion based on object modeling. Computation is bounded by limiting the depth of the merging tree. Results: The proposed method was found to perform consistently better, achieving merging accuracy in the range of 92% to 100% compared with our previous algorithm, which varied in the range of 75% to 97%, even with a modest merging tree depth of 3. The overall average accuracy improved from 90% to 96%, with roughly the same computational cost for a set of representative images drawn from the CA1, CA3, and parietal cortex regions of the rat hippocampus. Conclusion: Hierarchical tree model-based algorithms significantly improve the accuracy of automated nuclear segmentation without sacrificing speed.

Original languageEnglish (US)
Pages (from-to)20-33
Number of pages14
JournalCytometry Part A
Issue number1
StatePublished - Jan 2005


  • Cell counting
  • Confocal microscopy
  • Fluorescence in situ hybridization quantification
  • Hierarchical
  • Image segmentation
  • Model based
  • Object features
  • Region merging
  • Three-dimensional image analysis
  • Watershed segmentation

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Histology
  • Cell Biology


Dive into the research topics of 'Hierarchical, model-based merging of multiple fragments for improved three-dimensional segmentation of nuclei'. Together they form a unique fingerprint.

Cite this