Lunit "Using High-Performance AI Improves Reader's Lung Cancer Detection Ability"

Research Results Published in International Journal Radiology

Medical AI company Lunit announced on the 3rd that a study published in the latest issue of the international journal Radiology showed that the detection ability of readers significantly improves only when using AI models with high accuracy.

Lunit's AI chest X-ray image analysis solution 'Lunit INSIGHT CXR'
[Photo by Lunit].

Radiology is an SCI-level international journal published by the Radiological Society of North America (RSNA) and is a globally authoritative journal in the field of radiology.

This study was conducted with the participation of 30 radiologists, retrospectively selecting 120 patients who underwent chest X-rays at Seoul National University Hospital from December 2015 to February 2021.

The researchers first performed a primary reading independently without AI assistance on a total of 120 images, including 60 chest X-ray images with detected lung cancer and 60 normal images without cancer. Then, the readers were divided into two groups of 15 each: Group A used a high-accuracy AI, and Group B used a low-accuracy AI for the secondary reading. For the study, Group A used the algorithm of Lunit Insight CXR, a chest X-ray AI image analysis solution, while Group B used a relatively low-performance algorithm trained on only 10% of the total training data.

As a result, in the AUROC (Area Under the Receiver Operating Characteristic) analysis, a performance evaluation metric for AI models, the standalone lung cancer detection ability of the Lunit Insight CXR algorithm showed a high accuracy of 0.88 compared to 0.77 for the low-performance algorithm. Furthermore, when Group A readers performed a secondary reading using Lunit Insight CXR after the primary reading, the AUROC score improved from 0.77 to 0.82. In contrast, Group B readers recorded 0.75 for both primary and secondary readings, showing no numerical change. The closer the AUROC value is to 1, the better the performance, and models with a score above 0.8 are considered high-performance.

Additionally, the study measured the rate at which readers revised their diagnoses based on AI suggestions when the AI results conflicted with the readers’ initial independent primary readings. The results showed that when AI results conflicting with the readers’ primary readings were presented in the secondary reading, the final diagnosis was reversed by accepting the AI suggestion in 67% of cases in Group A and 59% in Group B, indicating higher acceptance of AI in the group using the high-performance algorithm.

Professor Changmin Park of the Department of Radiology at Seoul National University Hospital explained, "In this study, to analyze factors affecting the accuracy of secondary readings using AI, we surveyed the readers’ years of experience, perceptions of AI, AI usage, and research experience in advance. It was found that only the individual reader’s primary diagnostic accuracy and the AI’s accuracy had a significant impact on the accuracy of medical staff performing secondary readings with AI, while the reader’s inherent experience and tendencies were unrelated."

Seobeom Seok, CEO of Lunit, said, "This study confirmed that regardless of individual medical staff characteristics, only the use of high-performance AI leads to improved reading accuracy and higher acceptance of AI by medical staff. Going forward, Lunit will continue to focus on enhancing AI performance to improve the accuracy of medical staff’s readings and provide more precise information to patients."