ARRS 2022 Abstracts

RETURN TO ABSTRACT LISTING


ERS3345. Abdominal and Pelvic MRI Protocol Prediction using Natural Language Processing
Authors * Denotes Presenting Author
  1. Joshua Warner *; Mayo Clinic
  2. Daniel Blezek; Mayo Clinic
  3. Robert Hartman; Mayo Clinic
  4. John Thomas; Mayo Clinic
Objective:
We hypothesize it is possible to predict the correct MRI protocol for the majority of cases using repurposed state-of-the-art natural language processing (NLP) tools based on routinely provided information from referring providers. Such predictions could then be communicated back to schedulers and radiologists, ideally through integration with the Radiology Information System (RIS), to improve scheduling, protocoling efficiency, and accuracy.

Materials and Methods:
Google’s open-source NLP engine Bidirectional Encoder Representations from Transformers (BERT) has proven state-of-the-art performance across multiple NLP domains [1], including classification tasks in medicine [2]. Our subspecialty practice has a wide variety of named MRI protocols radiologists may employ, with or without additional customization. For the purposes of this work, we focused on abdominal and pelvic MRI protocols. The input universe was curated on basis of the originating orders, including some exams which ultimately were routed to other sections (e.g., MSK pelvis). Approximately 75% of the protocols were not further customized and these named protocols were used as classification labels; the remaining protocols were grouped into an “other” class. Key protocoling data was collected from all consecutive MRI exams from January 2019 through July 2021 including order, patient demographics, ordering department, free-text indication, notes, associated diagnosis, and linked encounter note. Data was split into training, validation, and test sets with the held-out test set consisting of the most recent 6 months of data to maximize generalizability. Training and hyperparameter optimization were conducted using PyTorch on a NVIDIA GeForce RTX 3090.

Results:
Optimization of the model’s hyperparameters and the best combination of classes and class groupings is ongoing. Our current best performing model consists of 24 named abdominal and pelvic MRI protocols plus “other”, with overall accuracy of 0.88 and Matthew’s Correlation Coefficient of 0.87 (higher is better, 1.0 would be perfect for both) representing state-of-the-art performance [3]. We continue to explore better methods to evaluate model performance, notably including partial credit for reasonable alternatives and weighting by prediction confidence. This excellent performance is predicated on a large amount of input data to train the model, in this work retrospective data from >40,000 protocoled exams.

Conclusion:
Accurate protocol prediction has numerous benefits including radiologist efficiency, value through practice standardization and efficiency, and education. 1) The algorithm-suggested protocol would significantly increase the radiologists’ or protocoling providers’ work efficiency. 2) Automated protocol prediction would allow schedulers to optimize scanner time, increasing efficiency and reducing delays, as the exam ordered may be significantly different in duration than the appropriate protocol. 3) Exam protocoling consistency would improve, benefitting practice standardization. 4) Suggested protocols would serve as a learning tool to guide trainees.