-
Auditing the Fairness of COVID-19 Forecast Hub Case Prediction Models
Authors:
Saad Mohammad Abrar,
Naman Awasthi,
Daniel Smolyak,
Vanessa Frias-Martinez
Abstract:
The COVID-19 Forecast Hub, a repository of COVID-19 forecasts from over 50 independent research groups, is used by the Centers for Disease Control and Prevention (CDC) for their official COVID-19 communications. As such, the Forecast Hub is a critical centralized resource to promote transparent decision making. Nevertheless, by focusing exclusively on prediction accuracy, the Forecast Hub fails to…
▽ More
The COVID-19 Forecast Hub, a repository of COVID-19 forecasts from over 50 independent research groups, is used by the Centers for Disease Control and Prevention (CDC) for their official COVID-19 communications. As such, the Forecast Hub is a critical centralized resource to promote transparent decision making. Nevertheless, by focusing exclusively on prediction accuracy, the Forecast Hub fails to evaluate whether the proposed models have similar performance across social determinants that have been known to play a role in the COVID-19 pandemic including race, ethnicity and urbanization level. In this paper, we carry out a comprehensive fairness analysis of the Forecast Hub model predictions and we show statistically significant diverse predictive performance across social determinants, with minority racial and ethnic groups as well as less urbanized areas often associated with higher prediction errors. We hope this work will encourage COVID-19 modelers and the CDC to report fairness metrics together with accuracy, and to reflect on the potential harms of the models on specific social groups and contexts.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
COVID-19's Unequal Toll: An assessment of small business impact disparities with respect to ethnorace in metropolitan areas in the US using mobility data
Authors:
Saad Mohammad Abrar,
Kazi Tasnim Zinat,
Naman Awasthi,
Vanessa Frias-Martinez
Abstract:
Early in the pandemic, counties and states implemented a variety of non-pharmacological interventions (NPIs) focused on mobility, such as national lockdowns or work-from-home strategies, as it became clear that restricting movement was essential to containing the epidemic. Due to these restrictions, businesses were severely affected and in particular, small, urban restaurant businesses. In additio…
▽ More
Early in the pandemic, counties and states implemented a variety of non-pharmacological interventions (NPIs) focused on mobility, such as national lockdowns or work-from-home strategies, as it became clear that restricting movement was essential to containing the epidemic. Due to these restrictions, businesses were severely affected and in particular, small, urban restaurant businesses. In addition to that, COVID-19 has also amplified many of the socioeconomic disparities and systemic racial inequities that exist in our society. The overarching objective of this study was to examine the changes in small urban restaurant visitation patterns following the COVID-19 pandemic and associated mobility restrictions, as well as to uncover potential disparities across different racial/ethnic groups in order to understand inequities in the impact and recovery. Specifically, the two key objectives were: 1) to analyze the overall changes in restaurant visitation patterns in US metropolitan areas during the pandemic compared to a pre-pandemic baseline, and 2) to investigate differences in visitation pattern changes across Census Block Groups with majority Asian, Black, Hispanic, White, and American Indian populations, identifying any disproportionate effects. Using aggregated geolocated cell phone data from SafeGraph, we document the overall changes in small urban restaurant businesses' visitation patterns with respect to racial composition at a granularity of Census Block Groups. Our results show clear indications of reduced visitation patterns after the pandemic, with slow recoveries. Via visualizations and statistical analyses, we show that reductions in visitation patterns were the highest for small urban restaurant businesses in majority Asian neighborhoods.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Text Mining to Identify and Extract Novel Disease Treatments From Unstructured Datasets
Authors:
Rahul Yedida,
Saad Mohammad Abrar,
Cleber Melo-Filho,
Eugene Muratov,
Rada Chirkova,
Alexander Tropsha
Abstract:
Objective: We aim to learn potential novel cures for diseases from unstructured text sources. More specifically, we seek to extract drug-disease pairs of potential cures to diseases by a simple reasoning over the structure of spoken text.
Materials and Methods: We use Google Cloud to transcribe podcast episodes of an NPR radio show. We then build a pipeline for systematically pre-processing the…
▽ More
Objective: We aim to learn potential novel cures for diseases from unstructured text sources. More specifically, we seek to extract drug-disease pairs of potential cures to diseases by a simple reasoning over the structure of spoken text.
Materials and Methods: We use Google Cloud to transcribe podcast episodes of an NPR radio show. We then build a pipeline for systematically pre-processing the text to ensure quality input to the core classification model, which feeds to a series of post-processing steps for obtaining filtered results. Our classification model itself uses a language model pre-trained on PubMed text. The modular nature of our pipeline allows for ease of future developments in this area by substituting higher quality components at each stage of the pipeline. As a validation measure, we use ROBOKOP, an engine over a medical knowledge graph with only validated pathways, as a ground truth source for checking the existence of the proposed pairs. For the proposed pairs not found in ROBOKOP, we provide further verification using Chemotext.
Results: We found 30.4% of our proposed pairs in the ROBOKOP database. For example, our model successfully identified that Omeprazole can help treat heartburn.We discuss the significance of this result, showing some examples of the proposed pairs.
Discussion and Conclusion: The agreement of our results with the existing knowledge source indicates a step in the right direction. Given the plug-and-play nature of our framework, it is easy to add, remove, or modify parts to improve the model as necessary. We discuss the results showing some examples, and note that this is a potentially new line of research that has further scope to be explored. Although our approach was originally oriented on radio podcast transcripts, it is input-agnostic and could be applied to any source of textual data and to any problem of interest.
△ Less
Submitted 22 October, 2020;
originally announced November 2020.
