On July 3, 2023 Lunit (KRX:328130.KQ), a global provider of AI-powered cancer diagnostics solutions, reported the publication of a study exploring the impact of medical AI solutions’ accuracy on radiologists’ diagnostic determination (Press release, Lunit, JUL 3, 2023, View Source;published-in-radiology-301868701.html [SID1234633033]). The study, conducted by Seoul National University Hospital from December 2015 to February 2021, was recently published in ‘Radiology,’ a renowned peer-reviewed journal in medical imaging.
Schedule your 30 min Free 1stOncology Demo!
Discover why more than 1,500 members use 1stOncology™ to excel in:
Early/Late Stage Pipeline Development - Target Scouting - Clinical Biomarkers - Indication Selection & Expansion - BD&L Contacts - Conference Reports - Combinatorial Drug Settings - Companion Diagnostics - Drug Repositioning - First-in-class Analysis - Competitive Analysis - Deals & Licensing
Schedule Your 30 min Free Demo!
The study involved a cohort of 30 doctors, including 20 board-certified radiologists with 5 to 18 years of expertise and 10 radiology residents with 2 to 3 years of training. A total of 120 retrospectively collected chest radiographs were assessed, with 60 obtained from patients with lung cancer and the remaining 60 showing no abnormalities.
During the first session, the 30 readers were divided into two groups and analyzed 120 chest X-rays each without the assistance of AI. In the subsequent session, each group reinterpreted the images with the aid of either a high-accuracy or low-accuracy AI model.
The high-accuracy AI model utilized in the study was Lunit INSIGHT CXR, Lunit’s commercially available AI solution for chest X-ray analysis. In contrast, the low-accuracy model was trained using only 10% of the data available to Lunit INSIGHT CXR. The AUROC (area under the receiver operating characteristic curve), a commonly used metric for diagnostic accuracy, of Lunit INSIGHT CXR was 0.88, while the low-accuracy AI model only reached 0.77.
The study revealed that using the higher-accuracy AI model, Lunit INSIGHT CXR, significantly improved radiologists’ performance. The AUROC was remarkably advanced from 0.77 to 0.82 when assisted by the high-accuracy AI model.
Conversely, the radiologists from the other group did not experience any performance improvement when utilizing the low-accuracy AI model, as the AUROC remained at 0.75. Moreover, the group that employed the high-accuracy AI model demonstrated a higher susceptibility to AI suggestions. The radiologists accepted 67% of AI recommendations that contradicted the initial reading results, compared to 59% acceptance of the group that utilized the low-accuracy AI model.
Moreover, the study findings highlighted that factors such as radiologists’ individual expertise, experience with AI, or attitudes toward AI had negligible impact on their reading performance in the second session. Instead, the accuracy of the AI model and the radiologists’ initial diagnostic accuracy emerged as the primary determinant shaping the final diagnostic determination.
These findings underscore the significance of the AI model’s performance when radiologists use AI as a second reader. Furthermore, the study demonstrates how such AI assistance can increase radiologists’ susceptibility to AI suggestions, ultimately contributing to more accurate diagnoses.
"The study backs that irrespective of radiologists’ individual characteristics, the utilization of high-performance AI significantly enhances diagnostic accuracy and fosters a greater acceptance of AI within medical practices," said Brandon Suh, CEO of Lunit. "At Lunit, we are committed to developing AI-powered solutions that not only improve patient outcomes but also augment the expertise of healthcare professionals. This publication is a testament to our dedication to advancing the field of cancer diagnostics through cutting-edge technology."
Radiology, owned and published by the Radiological Society of North America (RSNA), is a prestigious publication with a distinguished Impact Factor of 29.146, making it the number one ranked journal in the field of medical imaging.