E2644. Do We Really Need This? A Performance Evaluation for the Pulmonary Embolism AI Detection Algorithm at an Academic Institution
  1. Jessica Rubino; Dartmouth-Hitchcock Medical Center
  2. Matthew Maeder; Dartmouth-Hitchcock Medical Center
Artificial intelligence (AI) has been shown to be a useful triaging tool with high diagnostic performance, to aid radiologists in prioritizing high-risk scans and has already become ingrained in many academic and private practice radiology groups around the world. One type of application is an AI-powered triage tool designed to alert a radiologist of a positive result before anyone has had a chance to review the images. Although diagnostic AI pulmonary embolism (PE) algorithms have shown high sensitivity and specificity, limited studies have been published on AI triaging performance in the detection of PE. The purpose of this study was to introduce the concept of an AI triage tool implementation and workflow.

Materials and Methods:
This IRB approved retrospective study included 1899 contrast-enhanced chest CT angiography studies completed between January 2014 and December 2015. Our institution’s AI triage algorithm was applied to the images to detect PE and categorized as positive or negative for PE. Using a natural language processing (NLP) tool, the 1899 radiology reports were categorized as positive or negative results for PE. Discrepancies between AI and the radiologist’s interpretation were reviewed by an independent radiologist.

The prevalence of PE during the study period was 9.6% (182/1899). The AI algorithm correctly identified 30 PEs missed on initial radiology reports. The detection rate of the triage tool was 20.1% higher than the detection rate of our radiologists.

The AI triage tool used by our institution has shown a higher detection rate for pulmonary embolism (PE) than our radiologists. The tool and our radiologists have engaged in a feedback loop that is used to follow up on discrepant cases that need review and possible action by the interpreting radiologist. Currently, exams are prioritized by the clinician’s determination of routine versus stat status based on clinical exam and laboratory findings. Unfortunately, the clinical presentation of PE can be confounded by underlying cardiopulmonary conditions or nonspecific clinical symptoms. Based on these findings, a triage AI tool may give much-needed risk stratification. The increased detection rate of an AI triage tool is beneficial for prioritizing scans to allow earlier detection and potential for life-saving intervention for pulmonary embolism.