E4768. Neural Network Segmentation of Soft Tissue Masses on Musculoskeletal Ultrasound
Authors
Farah Tamizuddin;
New York University Langone Health
Jason Wei;
New York University Langone Health
Haresh Rajamohan;
New York University
Cem Deniz;
New York University Langone Health
Ronald Adler;
New York University Langone Health
Objective:
Train a convolutional neural network (CNN) to segment soft tissue mass boundaries on musculoskeletal ultrasound. This algorithm is a first step toward quantitatively assessing sonographic masses in terms of lesion volume, vascularity, texture, and internal mechanical properties. Quantitative metrics can provide rapid objective analysis while maintaining workflow.
Materials and Methods:
Ultrasound examinations (51 men, 101 women, average age 49 y) of soft tissue masses were identified from 2017–2023. The examinations were completely anonymized. A radiology resident (RR1) used the open-source digital imaging and communications in medicine (DICOM) imaging viewer, Horos, to segment images of well circumscribed, varying echogenicity masses. The images were cropped to include only the mass and a small amount of homogeneous background tissue, to prevent vessels or other structures being segmented. Examinations were performed on Siemens, Sequoia, and Philips IU22 scanners. Probe type, frequency, gain, dynamic range, and color map were recorded. Segmentation was repeated for 50 masses by another radiology resident (RR2) to compare internal validity. The first dataset segmented by RR1 contained 152 scans; 141 were used for training and 82 for testing all the segmentation models. Five-fold cross validation was performed using the nnU-Net (Insenee, 2021) framework, and the five trained 2D U-Net models were ensembled to generate predictions on the test set. The models were trained using a combination of cross-entropy and dice loss. A second data set segmented by RR2 contained 99 images; 60 were used to train a second set of U-Net models using the same settings as before. Finally, a third dataset combined the first and second training sets to train a third set of U-Net models. All three models were tested on the first dataset's test set - the RR1 test set of 46 scans. A set of 48 images segmented by RR1 and RR2 was compared to determine interrater reliability. An additional 37 images with color Doppler were obtained and anonymized. The masses were segmented both by hand and using the model created from grayscale images, and analysis of the vascularity was performed.
Results:
Using the cropped images, the deep learning model trained on the RR1 training set had an accuracy of 0.92 and a Dice score of 0.93. The model trained on RR2 had an accuracy of 0.85 and a Dice score of 0.83. When interrater reliability was calculated, Cohen's kappa between RR1 and RR2 was 0.90. For color Doppler images, the segmentation model segmented the masses with an accuracy of 0.82 and a Dice score of 0.82.
Conclusion:
Using over 150 ultrasound examinations, we created a deep learning neural network that can accurately segment the borders of soft tissue masses. This algorithm can automatically detect lesion boundaries of both grayscale and color masses with the ultimate goal of characterizing internal features, thereby facilitating strategies for follow-up and diagnosis.
