1842. Development of a Volumetric Pancreas Segmentation CT Dataset Through Trained Technologists: A Study During the COVID-19 Containment Phase
Authors* Denotes Presenting Author
Garima Suman *;
Deep neural networks typically require large labeled training datasets to match human performance. Such datasets are generated through recruitment of domain experts, which makes this process expensive and not scalable, and is one of the key barriers for development of production-scale AI models. To circumvent this bottleneck, crowdsourcing medical image annotation tasks to untrained persons in the community-at-large has been attempted previously with variable success. However, a similar approach for pancreas segmentation has not been attempted, likely due to the complex morphology and geometry of the pancreas. We performed a pilot study to evaluate the feasibility of training technologists for volumetric pancreas segmentation on Computed Tomography (CT) to develop datasets for AI applications and to assess impact of focused supplementary training on their performance.
Materials and Methods:
In this IRB-approved study, 22 technologists were trained in pancreas segmentation on portal venous phase CT through radiologist-led interactive videoconferencing sessions based on an image-rich curriculum. Technologists segmented pancreas in 188 CT studies using freehand drawing tools on custom image-viewing software. Subsequent supplementary training included multimedia videos focused on common errors, which was followed by second batch of 159 segmentations. Two radiologists reviewed all the cases and corrected the inaccurate segmentations. Technologists’ segmentations were compared against radiologists’ segmentations using Dice-Sorenson coefficient (DSC), Jaccard coefficient (JC), and Bland Altman analysis.
Corrections were needed in 71 (38%) cases from first batch [26 (37%) oversegmentations and 45 (63%) undersegmentations] and in 77 (48%) cases from second batch [12 (16%) oversegmentations and 65 (84%) undersegmentations]. DSC, JC, false positive (FP) and false negative (FN) [mean (standard deviation)] in first versus second batches were 0.63 (0.15) versus 0.63 (0.16), 0.48 (0.15) versus 0.48 (0.15), 0.29 (0.21) versus 0.21 (0.10), and 0.36 (0.20) versus 0.43 (0.19), respectively. Differences were not significant (p > 0.05). However, range of mean pancreatic volume difference reduced in the second batch [-2.74 cc (min -92.96 cc, max 87.47 cc) versus -23.57 cc (min -77.32, max 30.19)].
Trained technologists can perform volumetric pancreas segmentation on CT with reasonable accuracy. Focused supplementary training reduced the range of volume difference in segmentations. Trained technologists could augment endeavors of development of labeled medical imaging datasets.Alternately, they could augment efforts of radiologists in such endeavors. Investment into training allied radiologic health staff could yield a trained workforce that could be gainfully redeployed during routine downtimes as well as during extraordinary circumstances such as COVID-19 containment phase.