Focus Area: Epidemiology
Improving influenza A vaccine strain selection through deep evolutionary models
Even though vaccines have the potential to significantly alleviate the disease burden of epidemics such as the seasonal flu, current influenza vaccines offer limited protection. According to the Centers for Disease Control and Prevention (CDC), vaccine effectiveness has hovered below 50% for the past decade. Identifying the optimal strains to use in a vaccine is central to increasing its efficacy. However, this task is challenging due to the antigenic drift that occurs during the flu season. In this paper, we propose to select vaccines based on their escapability score, a metric that quantifies the antigenic similarity of vaccine strains with future dominant strains and demonstrates a strong correlation with clinical vaccine effectiveness. We introduce a deep learning-based approach that predicts both the antigenic properties of vaccine strains and the dominance of future circulating viruses, enabling efficient virtual screening of a large number of vaccine compositions. We utilized historical antigenic analysis data from the World Health Organization (WHO) to demonstrate that our model selects vaccine strains that reliably improve over the recommended ones.Contributors: Wenxian Shi, Rachel Menghua Wu Learn more
Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations
Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022.Contributors: Katharine Sherratt Learn more
Change is hard: a closer look at subpopulation shift
Machine learning models often perform poorly on subgroups that are under represented in the training data. Yet, little is understood on the variation in mechanisms that cause subpopulation shifts, and how algorithms generalize across such diverse shifts at scale. In this work, we provide a fine-grained analysis of subpopulation shift. We first propose a unified framework that dissects and explains common shifts in subgroups. We then establish a comprehensive benchmark of 20 state of-the-art algorithms evaluated on 12 real-world datasets invision, language, and healthcare domains. With results obtained from training over 10,000 models, we reveal intriguing observations for future progress in this space. First, existing algorithms only improve subgroup robustness over certain types of shifts but not others. Moreover, while current algorithms rely on group-annotated validation data for model selection, we find that a simple selection criterion based on worst-class accuracy is surprisingly effective even without any group information. Finally, unlike existing works that solely aim to improve worst-group accuracy (WGA), we demonstrate the fundamental tradeoff between WGA and other important metrics, highlighting the need to carefully choose testing metrics. Code and data are available at: https: //github.com/YyzHarry/SubpopBench.Contributors: Yuzhe Yang, Haoran Zhang, Dina Katabi Learn more
Sybil: a validated deep learning model to predict future lung cancer risk from a single low-dose chest computed tomography
Purpose: Low-dose computed tomography (LDCT) for lung cancer screening is effective, although most eligible people are not being screened. Tools that provide personalized future cancer risk assessment could focus approaches toward those most likely to benefit. We hypothesized that a deep learning model assessing the entire volumetric LDCT data could be built to predict individual risk without requiring additional demographic or clinical data.Methods: We developed a model called Sybil using LDCTs from the National Lung Screening Trial (NLST). Sybil requires only one LDCT and does not require clinical data or radiologist annotations; it can run in real time in the background on a radiology reading station. Sybil was validated on three independent data sets: a heldout set of 6,282 LDCTs from NLST participants, 8,821 LDCTs from Massachusetts General Hospital (MGH), and 12,280 LDCTs from Chang Gung Memorial Hospital (CGMH, which included people with a range of smoking history including nonsmokers).
Results: Sybil achieved area under the receiver-operator curves for lung cancer prediction at 1 year of 0.92 (95% CI, 0.88 to 0.95) on NLST, 0.86 (95% CI, 0.82 to 0.90) on MGH, and 0.94 (95% CI, 0.91 to 1.00) on CGMH external validation sets. Concordance indices over 6 years were 0.75 (95% CI, 0.72 to 0.78), 0.81 (95% CI, 0.77 to 0.85), and 0.80 (95% CI, 0.75 to 0.86) for NLST, MGH, and CGMH, respectively.
Conclusion: Sybil can accurately predict an individual's future lung cancer risk from a single LDCT scan to further enable personalized screening. Future study is required to understand Sybil's clinical applications. Our model and annotations are publicly available.
Contributors: Peter G. Mikhael, Jeremy Wohlwend, Ludvig Karstens, Justin Xiang, Angelo K. Takigami, Patrick P. Bourgouin, PuiYee Chan, Sofiane Mrah, Wael Amayri, Yu-Hsiang Juan, Cheng-Ta Yang, Yung-Liang Wan, Gigin Lin, Lecia V. Sequist, Florian J. Fintelmann Learn more
COVID-19: prediction, prevalence, and the operations of vaccine allocation
Problem definition: Mitigating the COVID-19 pandemic poses a series of unprecedented challenges, including predicting new cases and deaths, understanding true prevalence beyond what tests are able to detect, and allocating different vaccines across various regions. In this paper, we describe our efforts to tackle these issues and explore the impact on combating the pandemic in terms of case and death prediction, true prevalence, and fair vaccine distribution.Methodology/results: We present the methods we developed for predicting cases and deaths using a novel machine-learning-based aggregation method to create a single prediction that we call MIT-Cassandra. We further incorporate COVID-19 case prediction to determine true prevalence and incorporate this prevalence into an optimization model for efficiently and fairly managing the operations of vaccine allocation. We study the trade-offs of vaccine allocation between different regions and age groups, as well as first- and second-dose distribution of different vaccines. This also allows us to provide insights into how prevalence and exposure of the disease in different parts of the population can affect the distribution of different vaccine doses in a fair way.
Managerial implications: MIT-Cassandra is currently being used by the Centers for Disease Control and Prevention and is consistently among the best-performing methods in terms of accuracy, often ranking at the top. In addition, our work has been helping decision makers by predicting how cases and true prevalence of COVID-19 will progress over the next few months in different regions and utilizing the knowledge for vaccine distribution under various operational constraints. Finally, and very importantly, our work has specifically been used as part of a collaboration with the Massachusetts Institute of Technology's (MIT’s) Quest for Intelligence and as part of MIT’s process to reopen the institute.
Funding: Financial support from MIT Quest for Intelligence is gratefully acknowledged.
Contributors:Amine Bennouna, Joshua Joseph, David Nze-Ndong, Divya Singhvi , Omar Skali Lami , Yannis Spantidakis, Leann Thayaparan , Asterios Tsiourvas Learn more
COVID-19: A multiwave SIR-based model for learning waves
One of the greatest challenges of the COVID-19 pandemic has been the way evolving regulation, information, and sentiment have driven waves of the disease. Traditional epidemiology models, such as the SIR model, are not equipped to handle these behavioral-based changes. We propose a novel multiwave susceptible–infected–recovered (SIR) model, which can detect and model the waves of the disease. We bring together the SIR model’s compartmental structure with a change-point detection martingale process to identify new waves. We create a dynamic process where new waves can be flagged and learned in real time. We use this approach to extend the traditional susceptible–exposed–infected–recovered–dead (SEIRD) model into a multiwave SEIRD model and test it on forecasting COVID-19 cases from the John Hopkins University data set for states in the United States. We find that compared to the traditional SEIRD model, the multiwave SEIRD model improves mean absolute percentage error (MAPE) by 15%–25% for the United States. We benchmark the multiwave SEIRD model against top performing Center for Disease Control (CDC) models for COVID19 and find that the multiwave SERID model is able to outperform the majority of CDC models in long-term predictions.Contributors: Divya Singhvi, Omar Skali Lami, Leann Thayaparan Learn more
Toward robust mammography-based models for breast cancer risk
Improved breast cancer risk models enable targeted screening strategies that achieve earlier detection and less screening harm than existing guidelines. To bring deep learning risk models to clinical practice, we need to further refine their accuracy, validate them across diverse populations, and demonstrate their potential to improve clinical workflows. We developed Mirai, a mammography-based deep learning model designed to predict risk at multiple timepoints, leverage potentially missing risk factor information, and produce predictions that are consistent across mammography machines. Mirai was trained on a large dataset from Massachusetts General Hospital (MGH) in the United States and tested on held-out test sets from MGH, Karolinska University Hospital in Sweden, and Chang Gung Memorial Hospital (CGMH) in Taiwan, obtaining C-indices of 0.76 (95% confidence interval, 0.74 to 0.80), 0.81 (0.79 to 0.82), and 0.79 (0.79 to 0.83), respectively. Mirai obtained significantly higher 5-year ROC AUCs than the Tyrer-Cuzick model (P < 0.001) and prior deep learning models Hybrid DL (P < 0.001) and Image-Only DL (P < 0.001), trained on the same dataset. Mirai more accurately identified high-risk patients than prior methods across all datasets. On the MGH test set, 41.5% (34.4 to 48.5) of patients who would develop cancer within 5 years were identified as high risk, compared with 36.1% (29.1 to 42.9) by Hybrid DL (P = 0.02) and 22.9% (15.9 to 29.6) by the Tyrer-Cuzick model (P < 0.001). Learn more