ERS3057. Real-World Validation of a Deep Learning AI-Based Detection Algorithm for Suspected Pulmonary Embolism
Authors* Denotes Presenting Author
Mustafa Khan *;
University of California - Irvine Medical Center
Saba Chowdhry;
Viz.AI
Aimee DeGaetano;
Viz.AI
Angela Ayobi;
Avicenna.AI
Sarah Quenet;
Avicenna.AI
Yasmina Chaibi;
Avicenna.AI
Peter Chang;
University of California - Irvine Medical Center
Objective:
Pulmonary embolism (PE) is a potentially life-threatening condition commonly missed in clinical practice as signs and symptoms are nonspecific. CTPA imaging is considered the gold standard to visualize pulmonary vasculature and diagnose a PE. However, large volumes of CT imaging in busy hospital settings place a high demand on Radiologists and, as such, a tool that can automatically analyze scans for suspected PE and prioritize cases for review may be highly valuable. The Viz Pulmonary Embolism Clot Detection algorithm (Viz PE), in collaboration with Avicenna, is an FDA 510k-approved software application designed to detect and alert suspected PE. The purpose of this study was to evaluate the performance of this deep learning AI-based algorithm.
Materials and Methods:
This was a large-scale blinded algorithm validation study. The algorithm was run on 1206 retrospectively collected chest CTPA images and the results were compared against ground-truth consensus diagnosis determined by three board-certified Radiologists.
Results:
The dataset consisted of 819 (67.9%) PE-negative exams and 387 (32.1%) PE-positive exams. Overall, the algorithm demonstrated a sensitivity of 91.4% [95% CI: 86.4% - 95.0%] and specificity of 94.7% [95% CI: 93.1% - 96%] with a PPV of 75.9% and a NPV of 98.4%.
Conclusion:
This study used real-world data to validate a deep learning AI-based detection algorithm for suspected PE. With its high detection accuracy, Viz PE may help Radiologists prioritize cases for review and serve as a secondary reading tool. False negatives were typically encountered when the PE was small and in the presence of artifacts, such as streak artifact, volume averaging, and poor bolus timing. Common causes of false positive results included motion and streak artifact as well as volume averaging related to adenopathy. Of note, because the algorithm is not designed for the detection of subsegmental PE, cases in which a subsegmental PE was correctly identified by the algorithm were considered false positive results. However, in clinical practice the algorithm would be able to alert the Radiologist to the presence of possible subtle subsegmental PE. Therefore, adoption of this tool into traditional hospital workflow may accelerate diagnostic workup and reduce report turn-around time (RTAT).
