Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery*

Andre M. Schreiber; Minsik Hong; Jerzy W. Rozenblit

doi:10.1109/SMC52423.2021.9658708

Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery^*

Andre M. Schreiber, Minsik Hong, Jerzy W. Rozenblit

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

Depth estimation is an important challenge in the field of augmented reality. Supervised deep learning methods of depth estimation can be difficult to use in novel settings due to the need for labeled training data. The work presented in this paper overcomes the challenge in a laparoscopic surgical simulation environment by using synthetic data generation for RGB-D training data. We also provide a neural network architecture that can generate real-time 448x448 depth map outputs suitable for use in AR applications. Our approach shows satisfactory performance when tested on a non-synthetic test dataset with an RMSE of 2.50 cm, MAE of 1.04 cm, and δ < 1.25 of 0.987.

Original language	English (US)
Title of host publication	2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2121-2126
Number of pages	6
ISBN (Electronic)	9781665442077
DOIs	https://doi.org/10.1109/SMC52423.2021.9658708
State	Published - 2021
Event	2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021 - Melbourne, Australia Duration: Oct 17 2021 → Oct 20 2021

Publication series

Name	Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)	1062-922X

Conference

Conference	2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021
Country/Territory	Australia
City	Melbourne
Period	10/17/21 → 10/20/21

ASJC Scopus subject areas

Electrical and Electronic Engineering
Control and Systems Engineering
Human-Computer Interaction

Access to Document

10.1109/SMC52423.2021.9658708

Cite this

Schreiber, A. M., Hong, M., & Rozenblit, J. W. (2021). Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery^*. In 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021 (pp. 2121-2126). (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC52423.2021.9658708

Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery^*. / Schreiber, Andre M.; Hong, Minsik ; Rozenblit, Jerzy W.
2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. p. 2121-2126 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Schreiber, AM, Hong, M & Rozenblit, JW 2021, Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery^*. in 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021. Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, Institute of Electrical and Electronics Engineers Inc., pp. 2121-2126, 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021, Melbourne, Australia, 10/17/21. https://doi.org/10.1109/SMC52423.2021.9658708

Schreiber AM, Hong M , Rozenblit JW. Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery^*. In 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021. Institute of Electrical and Electronics Engineers Inc. 2021. p. 2121-2126. (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics). doi: 10.1109/SMC52423.2021.9658708

Schreiber, Andre M. ; Hong, Minsik ; Rozenblit, Jerzy W. / Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery^*. 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021. Institute of Electrical and Electronics Engineers Inc., 2021. pp. 2121-2126 (Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics).

@inproceedings{7185e56d688b4cb49cf5a1457723c1c8,

title = "Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery*",

abstract = "Depth estimation is an important challenge in the field of augmented reality. Supervised deep learning methods of depth estimation can be difficult to use in novel settings due to the need for labeled training data. The work presented in this paper overcomes the challenge in a laparoscopic surgical simulation environment by using synthetic data generation for RGB-D training data. We also provide a neural network architecture that can generate real-time 448x448 depth map outputs suitable for use in AR applications. Our approach shows satisfactory performance when tested on a non-synthetic test dataset with an RMSE of 2.50 cm, MAE of 1.04 cm, and δ < 1.25 of 0.987.",

author = "Schreiber, {Andre M.} and Minsik Hong and Rozenblit, {Jerzy W.}",

note = "Funding Information: *Research supported by National Science Foundation under Grant Number 1622589 “Computer Guided Laparoscopy Training”. Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021 ; Conference date: 17-10-2021 Through 20-10-2021",

year = "2021",

doi = "10.1109/SMC52423.2021.9658708",

language = "English (US)",

series = "Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2121--2126",

booktitle = "2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021",

}

TY - GEN

T1 - Monocular Depth Estimation using Synthetic Data for an Augmented Reality Training System in Laparoscopic Surgery*

AU - Schreiber, Andre M.

AU - Hong, Minsik

AU - Rozenblit, Jerzy W.

PY - 2021

Y1 - 2021

N2 - Depth estimation is an important challenge in the field of augmented reality. Supervised deep learning methods of depth estimation can be difficult to use in novel settings due to the need for labeled training data. The work presented in this paper overcomes the challenge in a laparoscopic surgical simulation environment by using synthetic data generation for RGB-D training data. We also provide a neural network architecture that can generate real-time 448x448 depth map outputs suitable for use in AR applications. Our approach shows satisfactory performance when tested on a non-synthetic test dataset with an RMSE of 2.50 cm, MAE of 1.04 cm, and δ < 1.25 of 0.987.

AB - Depth estimation is an important challenge in the field of augmented reality. Supervised deep learning methods of depth estimation can be difficult to use in novel settings due to the need for labeled training data. The work presented in this paper overcomes the challenge in a laparoscopic surgical simulation environment by using synthetic data generation for RGB-D training data. We also provide a neural network architecture that can generate real-time 448x448 depth map outputs suitable for use in AR applications. Our approach shows satisfactory performance when tested on a non-synthetic test dataset with an RMSE of 2.50 cm, MAE of 1.04 cm, and δ < 1.25 of 0.987.

UR - http://www.scopus.com/inward/record.url?scp=85124266918&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85124266918&partnerID=8YFLogxK

U2 - 10.1109/SMC52423.2021.9658708

DO - 10.1109/SMC52423.2021.9658708

M3 - Conference contribution

AN - SCOPUS:85124266918

T3 - Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics

SP - 2121

EP - 2126

BT - 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2021 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2021

Y2 - 17 October 2021 through 20 October 2021

ER -