ARRS 2022 Abstracts


E2001. Interobserver Agreement Between Eight Observers Using IOTA Simple Rules and O-RADS Lexicon Descriptors for Adnexal Masses
  1. Neha Antil; Stanford University
  2. Amelie Lutz; Stanford University
  3. Thodsawit Tiyarattanachai; Stanford University
  4. Preethi Raghu; Stanford University
  5. Edwina Chang; Stanford University
  6. Aya Kamaya; Stanford University
The objective of this study was to evaluate interobserver agreement (IOA) in assigning imaging features and classifying adnexal masses (AMs) using the International Ovarian Tumor Analysis (IOTA) simple rules and the Ovarian-Adnexal Reporting and Data System (O-RADS) lexicon descriptors. We also aim to identify and highlight the causes of discrepancy between readers.

Materials and Methods:
In this retrospective HIPAA-compliant IRB-approved study, pelvic ultrasound (US) examinations in 110 women (mean age 45.0 ± 9.5 years; range 20–84 years) with 114 cystic AMs were evaluated by eight radiologists (four attendings and four fellows). All readers were blinded to the final diagnosis. AMs were evaluated according to five malignant (M) and five benign (B) feature categories of the IOTA simple rules and according to 10 feature categories that constitute O-RADS lexicon descriptors. Each feature category was analyzed for the IOA using the intraclass correlation coefficient (ICC) for ordinal variables and free marginal kappa for nominal variables. ICCs were interpreted as follows: < 0.40, poor; 0.40–0.59, fair; 0.60–0.74, good; 0.75–1.00, excellent. Free-marginal kappa values were interpreted as follows: < 0, poor; 0.01–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; 0.81–1.00, almost perfect agreement. The two-tailed significance level was set at 0.05.

IOA was almost perfect for three of five malignant lesion categories (M2-4) and substantial for the remaining two malignant feature categories (M1, M5) of the IOTA simple rules, with k values of 0.80–0.82 and 0.68–0.69, respectively. Interobserver agreement was almost perfect for two of five benign feature categories (B2, B3), substantial for two (B4, B5), and moderate for the remaining benign feature (B1), with k values of 0.81–0.90, 0.69–070, and 0.60, respectively. The final IOTA category conclusion was excellent for all eight readers combined and for attendings and fellows separately (ICC: 0.765, 0.769, and 0.757, respectively). For O-RADS, agreement in interpretation was almost perfect for two feature categories (presence of ascites and peritoneal nodules) with k values of 0.89 and 0.97. Agreement in interpretation for the remaining eight feature categories (lesion type, inner wall type, septation type, number of septa, number of solid components, contour of solid component, and color score) were variable, ranging from fair to substantial (ICC and k values range from 0.39–0.61). The O-RADS final conclusion was good for all eight readers together and for fellows separately, and fair for attendings, with ICC values of 0.621, 0.725, and 0.517, respectively.

The ACR O-RADS introduced in 2019 provides a structured risk stratification and management system intended to improve US reporting of adnexal cystic lesions and provide management recommendations. IOA of IOTA simple rules is greater than ACR O-RADS, which may be in part due to the fact that O-RADS provides a more nuanced prediction of benignity versus malignancy.