Effect of Deep Learning-Based Artificial Intelligence on Radiologists' Performance in Identifying Nigrosome 1 Abnormalities on Susceptibility Map-Weighted Imaging

Abstract

Objective: To evaluate the effect of deep learning (DL)-based artificial intelligence (AI) software on the diagnostic performance of radiologists with different experience levels in detecting nigrosome 1 (N1) abnormalities on susceptibility map-weighted imaging (SMwI). Materials and Methods: This retrospective diagnostic case-control study analyzed 139 SMwI scans of 59 patients with Parkinson's disease (PD) and 80 healthy participants. Participants were imaged using 3T MRI, and AI-generated assessments for N1 abnormalities were obtained using an AI model (version 1.0.1.0; Heuron Corporation, Seoul, Korea), which utilized YOLOX-based object detection and SparseInst segmentation models. Four radiologists (two experienced neuroradiologists and two less experienced residents) evaluated N1 abnormalities with and without AI in a crossover study design. Diagnostic performance metrics, inter-reader agreements, and reader responses to AI-generated assessments were evaluated. Results: Use of AI significantly improved diagnostic performance compared with interpretation without it across three readers, with significant increases in specificity (0.86 vs. 0.94, P = 0.004; 0.91 vs. 0.97, P = 0.024; and 0.90 vs. 0.97, P = 0.012). Inter-reader agreement also improved with AI, as Fleiss's kappa increased from 0.73 (95% confidence interval [CI]: 0.61-0.84) to 0.87 (95% CI: 0.76-0.99). The net reclassification index (NRI) demonstrated significant improvement in three of the four readers. When grouped by experience level, less experienced readers showed greater improvement (NRI = 12.8%, 95% CI: 0.067-0.190) than experienced readers (NRI = 0.8%, 95% CI:-0.037-0.051). In the less experienced group, reader-AI disagreement was significantly higher in the PD group than in the normal group (8.1% vs. 3.8%, P = 0.029). Conclusion: DL-based AI enhances the diagnostic performance in detecting N1 abnormalities on SMwI, particularly benefiting less experienced radiologists. These findings underscore the potential for improving diagnostic workflows for PD.