TY - JOUR
T1 - Machine learning-based automatic classification of video recorded neonatal manipulations and associated physiological parameters
T2 - A feasibility study
AU - Singh, Harpreet
AU - Kusuda, Satoshi
AU - McAdams, Ryan M.
AU - Gupta, Shubham
AU - Kalra, Jayant
AU - Kaur, Ravneet
AU - Das, Ritu
AU - Anand, Saket
AU - Pandey, Ashish Kumar
AU - Cho, Su Jin
AU - Saluja, Satish
AU - Boutilier, Justin J.
AU - Saria, Suchi
AU - Palma, Jonathan
AU - Kaur, Avneet
AU - Yadav, Gautam
AU - Sun, Yao
N1 - Publisher Copyright:
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/ licenses/by/4.0/).
PY - 2021/1
Y1 - 2021/1
N2 - Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO2 )) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates <32 weeks’ gestation, diaper changes were associated with significant changes in HR and SpO2, and, for neonates ≥32 weeks’ gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.
AB - Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO2 )) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates <32 weeks’ gestation, diaper changes were associated with significant changes in HR and SpO2, and, for neonates ≥32 weeks’ gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.
KW - CNN
KW - Electronic medical records
KW - IoT
KW - LSTM
KW - Machine learning
KW - Neonatal intensive care units
KW - Physiological deviations
KW - Physiological parameters
KW - Streaming server
KW - Video monitoring
UR - http://www.scopus.com/inward/record.url?scp=85112280271&partnerID=8YFLogxK
U2 - 10.3390/children8010001
DO - 10.3390/children8010001
M3 - Article
AN - SCOPUS:85112280271
SN - 2227-9067
VL - 8
JO - Children
JF - Children
IS - 1
M1 - 1
ER -