Skip to Content

Focus Area: Sybil

Significance of Image Reconstruction Parameters for Future Lung Cancer Risk Prediction Using Low-Dose Chest Computed Tomography and the Open-Access Sybil Algorithm

Purpose

Sybil is a validated publicly available deep learning–based algorithm that can accurately predict lung cancer risk from a single low-dose computed tomography (LDCT) scan. We aimed to study the effect of image reconstruction parameters and CT scanner manufacturer on Sybil's performance.

Materials and Methods

Using LDCTs of a subset of the National Lung Screening Trial participants, which we previously used for internal validation of the Sybil algorithm (test set), we ran the Sybil algorithm on LDCT series pairs matched on kilovoltage peak, milliampere-seconds, reconstruction interval, reconstruction diameter, and either reconstruction filter or axial slice thickness. We also evaluated the cumulative effect of these parameters by combining the best- and the worst-performing parameters. A subanalysis compared Sybil's performance by CT manufacturer. We considered any LDCT positive if future lung cancer was subsequently confirmed by biopsy or surgical resection. The areas under the curve (AUCs) for each series pair were compared using DeLong's test.

Results

There was no difference in Sybil's performance between 1049 pairs of standard versus bone reconstruction filter (AUC at 1 year 0.84 [95% confidence interval (CI): 0.70–0.99] vs 0.86 [95% CI: 0.75–0.98], P = 0.87) and 1961 pairs of standard versus lung reconstruction filter (AUC at 1 year 0.98 [95% CI: 0.97–0.99] vs 0.98 [95% CI: 0.96–0.99], P = 0.81). Similarly, there was no difference in 1288 pairs comparing 2-mm versus 5-mm axial slice thickness (AUC at 1 year 0.98 [95% CI: 0.94–1.00] vs 0.99 [95% CI: 0.97–0.99], P = 0.68). The best-case scenario combining a lung reconstruction filter with 2-mm slice thickness compared with the worst-case scenario combining a bone reconstruction filter with 2.5-mm slice thickness uncovered a significantly different performance at years 2–4 (P = 0.03). Subanalysis showed no significant difference in performance between Siemens and Toshiba scanners.

Conclusions

Sybil's predictive performance for future lung cancer risk is robust across different reconstruction filters and axial slice thicknesses, demonstrating its versatility in various imaging settings. Combining favorable reconstruction parameters can significantly enhance predictive ability at years 2–4. The absence of significant differences between Siemens and Toshiba scanners further supports Sybil's versatility.

Contributors: Judit Simon, Peter Mikhael, Alexander Graur, Allison E. B. Chang, Steven J Skates, Raymond U. Osarogiagbon, Lecia V. Sequist, Florian J. Fintelmann Learn more

Role of sex in lung cancer risk prediction based on single low-dose chest computed tomography

A validated open-source deep-learning algorithm called Sybil can accurately predict long-term lung cancer risk from a single low-dose chest computed tomography (LDCT). However, Sybil was trained on a majority-male cohort. Use of artificial intelligence algorithms trained on imbalanced cohorts may lead to inequitable outcomes in real-world settings. We aimed to study whether Sybil predicts lung cancer risk equally regardless of sex. We analyzed 10,573 LDCTs from 6127 consecutive lung cancer screening participants across a health system between 2015 and 2021. Sybil achieved AUCs of 0.89 (95% CI: 0.85–0.93) for females and 0.89 (95% CI: 0.85–0.94) for males at 1 year, p = 0.92. At 6 years, the AUC was 0.87 (95% CI: 0.83–0.93) for females and 0.79 (95% CI: 0.72–0.86) for males, p = 0.01. In conclusion, Sybil can accurately predict future lung cancer risk in females and males in a real-world setting and performs better in females than in males for predicting 6-year lung cancer risk.

Contributors: Judit Simon, Ismail Tahir, Alexander Graur, Stefan Ringer, Amanda Fata, Yang Chi-Fu Jeffrey, Jo-Anne Shepard, Francine Jacobson, Lecia V. Sequist, Lydia E. Pace Learn more

Sybil: a validated deep learning model to predict future lung cancer risk from a single low-dose chest computed tomography

Purpose: Low-dose computed tomography (LDCT) for lung cancer screening is effective, although most eligible people are not being screened. Tools that provide personalized future cancer risk assessment could focus approaches toward those most likely to benefit. We hypothesized that a deep learning model assessing the entire volumetric LDCT data could be built to predict individual risk without requiring additional demographic or clinical data.

Methods: We developed a model called Sybil using LDCTs from the National Lung Screening Trial (NLST). Sybil requires only one LDCT and does not require clinical data or radiologist annotations; it can run in real time in the background on a radiology reading station. Sybil was validated on three independent data sets: a heldout set of 6,282 LDCTs from NLST participants, 8,821 LDCTs from Massachusetts General Hospital (MGH), and 12,280 LDCTs from Chang Gung Memorial Hospital (CGMH, which included people with a range of smoking history including nonsmokers).

Results: Sybil achieved area under the receiver-operator curves for lung cancer prediction at 1 year of 0.92 (95% CI, 0.88 to 0.95) on NLST, 0.86 (95% CI, 0.82 to 0.90) on MGH, and 0.94 (95% CI, 0.91 to 1.00) on CGMH external validation sets. Concordance indices over 6 years were 0.75 (95% CI, 0.72 to 0.78), 0.81 (95% CI, 0.77 to 0.85), and 0.80 (95% CI, 0.75 to 0.86) for NLST, MGH, and CGMH, respectively.

Conclusion: Sybil can accurately predict an individual's future lung cancer risk from a single LDCT scan to further enable personalized screening. Future study is required to understand Sybil's clinical applications. Our model and annotations are publicly available.

Contributors: Adam Yala, Ludvig Karstens, Justin Xiang, Angelo K. Takigami, Patrick P. Bourgouin, PuiYee Chan, Sofiane Mrah, Wael Amayri, Yu-Hsiang Juan, Cheng-Ta Yang, Yung-Liang Wan, Gigin Lin, Lecia V. Sequist, Florian J. Fintelmann Learn more
image description