Showing 1–8 of 8 results for author: Spertus, J

Stylish Risk-Limiting Audits in Practice

Authors: Amanda K. Glazer, Jacob V. Spertus, Philip B. Stark

Abstract: Risk-limiting audits (RLAs) can use information about which ballot cards contain which contests (card-style data, CSD) to ensure that each contest receives adequate scrutiny, without examining more cards than necessary. RLAs using CSD in this way can be substantially more efficient than RLAs that sample indiscriminately from all cast cards. We describe an open-source Python implementation of RLAs… ▽ More Risk-limiting audits (RLAs) can use information about which ballot cards contain which contests (card-style data, CSD) to ensure that each contest receives adequate scrutiny, without examining more cards than necessary. RLAs using CSD in this way can be substantially more efficient than RLAs that sample indiscriminately from all cast cards. We describe an open-source Python implementation of RLAs using CSD for the Hart InterCivic Verity voting system and the Dominion Democracy Suite(R) voting system. The software is demonstrated using all 181 contests in the 2020 general election and all 214 contests in the 2022 general election in Orange County, CA, USA, the fifth-largest election jurisdiction in the U.S., with over 1.8 million active voters. (Orange County uses the Hart Verity system.) To audit the 181 contests in 2020 to a risk limit of 5% without using CSD would have required a complete hand tally of all 3,094,308 cast ballot cards. With CSD, the estimated sample size is about 20,100 cards, 0.65% of the cards cast--including one tied contest that required a complete hand count. To audit the 214 contests in 2022 to a risk limit of 5% without using CSD would have required a complete hand tally of all 1,989,416 cast cards. With CSD, the estimated sample size is about 62,250 ballots, 3.1% of cards cast--including three contests with margins below 0.1% and 9 with margins below 0.5%. △ Less

Submitted 16 September, 2023; originally announced September 2023.

COBRA: Comparison-Optimal Betting for Risk-limiting Audits

Authors: Jacob Spertus

Abstract: Risk-limiting audits (RLAs) can provide routine, affirmative evidence that reported election outcomes are correct by checking a random sample of cast ballots. An efficient RLA requires checking relatively few ballots. Here we construct highly efficient RLAs by optimizing supermartingale tuning parameters--$\textit{bets}$--for ballot-level comparison audits. The exactly optimal bets depend on the t… ▽ More Risk-limiting audits (RLAs) can provide routine, affirmative evidence that reported election outcomes are correct by checking a random sample of cast ballots. An efficient RLA requires checking relatively few ballots. Here we construct highly efficient RLAs by optimizing supermartingale tuning parameters--$\textit{bets}$--for ballot-level comparison audits. The exactly optimal bets depend on the true rate of errors in cast-vote records (CVRs)--digital receipts detailing how machines tabulated each ballot. We evaluate theoretical and simulated workloads for audits of contests with a range of diluted margins and CVR error rates. Compared to bets recommended in past work, using these optimal bets can dramatically reduce expected workloads--by 93% on average over our simulated audits. Because the exactly optimal bets are unknown in practice, we offer some strategies for approximating them. As with the ballot-polling RLAs described in ALPHA and RiLACs, adapting bets to previously sampled data or diversifying them over a range of suspected error rates can lead to substantially more efficient audits than fixing bets to $\textit{a priori}$ values, especially when those values are far from correct. We sketch extensions to other designs and social choice functions, and conclude with some recommendations for real-world comparison audits. △ Less

Submitted 16 March, 2023; originally announced April 2023.

Comments: 16 pages, 2 figures, 2 tables. Accepted to the Workshop on Advances in Secure Electronic Voting (Voting'23)

Sweeter than SUITE: Supermartingale Stratified Union-Intersection Tests of Elections

Authors: Jacob V. Spertus, Philip B. Stark

Abstract: Stratified sampling can be useful in risk-limiting audits (RLAs), for instance, to accommodate heterogeneous voting equipment or laws that mandate jurisdictions draw their audit samples independently. We combine the union-intersection tests in SUITE, the reduction of RLAs to testing whether the means of a collection of lists are all $\leq 1/2$ of SHANGRLA, and the nonnegative supermartingale (NNSM… ▽ More Stratified sampling can be useful in risk-limiting audits (RLAs), for instance, to accommodate heterogeneous voting equipment or laws that mandate jurisdictions draw their audit samples independently. We combine the union-intersection tests in SUITE, the reduction of RLAs to testing whether the means of a collection of lists are all $\leq 1/2$ of SHANGRLA, and the nonnegative supermartingale (NNSM) tests in ALPHA to improve the efficiency and flexibility of stratified RLAs. A simple, non-adaptive strategy for combining stratumwise NNSMs decreases the measured risk in the 2018 pilot hybrid audit in Kalamazoo, Michigan, USA by more than an order of magnitude, from 0.037 for SUITE to 0.003 for our method. We give a simple, computationally inexpensive, adaptive rule for deciding which stratum to sample next that reduces audit workload by as much as 74% in examples. We also present NNSM-based tests that are computationally tractable even when there are many strata, illustrated with a simulated audit stratified across California's 58 counties. △ Less

Submitted 25 July, 2022; v1 submitted 7 July, 2022; originally announced July 2022.

Comments: 16 pages, 1 figure, 3 tables, accepted to E-Vote ID 2022

Optimal sampling and assay for soil organic carbon estimation

Authors: Jacob V Spertus

Abstract: The world needs around 150 Pg of negative carbon emissions to mitigate climate change. Global soils may provide a stable, sizeable reservoir to help achieve this goal by sequestering atmospheric carbon dioxide as soil organic carbon (SOC). In turn, SOC can support healthy soils and provide a multitude of ecosystem benefits. To support SOC sequestration, researchers and policy makers must be able t… ▽ More The world needs around 150 Pg of negative carbon emissions to mitigate climate change. Global soils may provide a stable, sizeable reservoir to help achieve this goal by sequestering atmospheric carbon dioxide as soil organic carbon (SOC). In turn, SOC can support healthy soils and provide a multitude of ecosystem benefits. To support SOC sequestration, researchers and policy makers must be able to precisely measure the amount of SOC in a given plot of land. SOC measurement is typically accomplished by taking soil cores selected at random from the plot under study, mixing (compositing) some of them together, and analyzing (assaying) the composited samples in a laboratory. Compositing reduces assay costs, which can be substantial. Taking samples is also costly. Given uncertainties and costs in both sampling and assay along with a desired estimation precision, there is an optimal composite size that will minimize the budget required to achieve that precision. Conversely, given a fixed budget, there is a composite size that minimizes uncertainty. In this paper, we describe and formalize sampling and assay for SOC and derive the optima for three commonly used assay methods: dry combustion in an elemental analyzer, loss-on-ignition, and mid-infrared spectroscopy. We demonstrate the utility of this approach using data from a soil survey conducted in California. We give recommendations for practice and provide software to implement our framework. △ Less

Submitted 30 August, 2021; v1 submitted 18 January, 2021; originally announced January 2021.

Comments: 30 pages, 3 figures

More style, less work: card-style data decrease risk-limiting audit sample sizes

Authors: Amanda K. Glazer, Jacob V. Spertus, Philip B. Stark

Abstract: U.S. elections rely heavily on computers such as voter registration databases, electronic pollbooks, voting machines, scanners, tabulators, and results reporting websites. These introduce digital threats to election outcomes. Risk-limiting audits (RLAs) mitigate threats to some of these systems by manually inspecting random samples of ballot cards. RLAs have a large chance of correcting wrong outc… ▽ More U.S. elections rely heavily on computers such as voter registration databases, electronic pollbooks, voting machines, scanners, tabulators, and results reporting websites. These introduce digital threats to election outcomes. Risk-limiting audits (RLAs) mitigate threats to some of these systems by manually inspecting random samples of ballot cards. RLAs have a large chance of correcting wrong outcomes (by conducting a full manual tabulation of a trustworthy record of the votes), but can save labor when reported outcomes are correct. This efficiency is eroded when sampling cannot be targeted to ballot cards that contain the contest(s) under audit. If the sample is drawn from all cast cards, RLA sample sizes scale like the reciprocal of the fraction of ballot cards that contain the contest(s) under audit. That fraction shrinks as the number of cards per ballot grows (i.e., when elections contain more contests) and as the fraction of ballots that contain the contest decreases (i.e., when a smaller percentage of voters are eligible to vote in the contest). States that conduct RLAs of contests on multi-card ballots or of small contests can dramatically reduce sample sizes by using information about which ballot cards contain which contests -- by kee** track of card-style data (CSD). For instance, CSD reduces the expected number of draws needed to audit a single countywide contest on a 4-card ballot by 75%. Similarly, CSD reduces the expected number of draws by 95% or more for an audit of two contests with the same margin on a 4-card ballot if one contest is on every ballot and the other is on 10% of ballots. In realistic examples, the savings can be several orders of magnitude. △ Less

Submitted 6 December, 2020; originally announced December 2020.

Comments: 19 pages, 9 figures. In submission at Digital Threats: Research and Practice

Bayesian Meta-Analysis of Multiple Continuous Treatments: An Application to Antipsychotic Drugs

Authors: Jacob Spertus, Marcela Horvitz-Lennon, Sharon-Lise Normand

Abstract: Modeling dose-response relationships of drugs is essential to understanding their effect on patient outcomes under realistic circumstances. While intention-to-treat analyses of clinical trials provide the effect of assignment to a particular drug and dose, they do not capture observed exposure after factoring in non-adherence and dropout. We develop Bayesian methods to flexibly model dose-response… ▽ More Modeling dose-response relationships of drugs is essential to understanding their effect on patient outcomes under realistic circumstances. While intention-to-treat analyses of clinical trials provide the effect of assignment to a particular drug and dose, they do not capture observed exposure after factoring in non-adherence and dropout. We develop Bayesian methods to flexibly model dose-response relationships of binary outcomes with continuous treatment, allowing for treatment effect heterogeneity and a non-linear response surface. We use a hierarchical framework for meta-analysis with the explicit goal of combining information from multiple trials while accounting for heterogeneity. In an application, we examine the risk of excessive weight gain for patients with schizophrenia treated with the second generation antipsychotics paliperidone, risperidone, or olanzapine in 14 clinical trials. Averaging over the sample population, we found that olanzapine contributed to a 15.6% (95% CrI: 6.7, 27.1) excess risk of weight gain at a 500mg cumulative dose. Paliperidone conferred a 3.2% (95% CrI: 1.5, 5.2) and risperidone a 14.9% (95% CrI: 0.0, 38.7) excess risk at 500mg olanzapine equivalent cumulative doses. Blacks had an additional 6.8% (95% CrI: 1.0, 12.4) risk of weight gain over non-blacks at 1000mg olanzapine equivalent cumulative doses of paliperidone. △ Less

Submitted 14 February, 2018; originally announced February 2018.

Comments: 14 Pages, 2 Figures, 2 Tables, 2 Appendix Figures

Regularization and Hierarchical Prior Distributions for Adjustment with Health Care Claims Data: Rethinking Comorbidity Scores

Authors: Jacob Spertus, Samrachana Adhikari, Sharon-Lise Normand

Abstract: Health care claims data refer to information generated from interactions within health systems. They have been used in health services research for decades to assess effectiveness of interventions, determine the quality of medical care, predict disease prognosis, and monitor population health. While claims data are relatively cheap and ubiquitous, they are high-dimensional, sparse, and noisy, typi… ▽ More Health care claims data refer to information generated from interactions within health systems. They have been used in health services research for decades to assess effectiveness of interventions, determine the quality of medical care, predict disease prognosis, and monitor population health. While claims data are relatively cheap and ubiquitous, they are high-dimensional, sparse, and noisy, typically requiring dimension reduction. In health services research, the most common data reduction strategy involves use of a comorbidity index -- a single number summary reflecting overall patient health. We discuss Bayesian regularization strategies and a novel hierarchical prior distribution as better options for dimension reduction in claims data. The specifications are designed to work with a large number of codes while controlling variance by shrinking coefficients towards zero or towards a group-level mean. A comparison of drug-eluting to bare-metal coronary stents illustrates approaches. In our application, regularization and a hierarchical prior improved over comorbidity scores in terms of prediction and causal inference, as evidenced by out-of-sample fit and the ability to meet falsifiability endpoints. △ Less

Submitted 14 November, 2017; originally announced November 2017.

Comments: 13 pages (w/o references and appendix), 2 figures, methodological ties to arXiv:1710.03138

Bayesian Propensity Scores for High-Dimensional Causal Inference: A Comparison of Drug-Eluting to Bare-Metal Coronary Stents

Authors: Jacob Spertus, Sharon-Lise Normand

Abstract: High-dimensional data can be useful for causal inference by providing many confounders that may bolster the plausibility of the ignorability assumption. Propensity score methods are powerful tools for causal inference, are popular in health care research, and are particularly useful for high-dimensional data. Recent interest has surrounded a Bayesian formulation of these methods in order to flexib… ▽ More High-dimensional data can be useful for causal inference by providing many confounders that may bolster the plausibility of the ignorability assumption. Propensity score methods are powerful tools for causal inference, are popular in health care research, and are particularly useful for high-dimensional data. Recent interest has surrounded a Bayesian formulation of these methods in order to flexibly estimate propensity scores and summarize posterior quantities while incorporating variance from the (potentially high-dimensional) treatment model. We discuss methods for Bayesian propensity score analysis of binary treatments, focusing on modern methods for high-dimensional Bayesian regression and the propagation of uncertainty from the treatment regression. We introduce a novel and simple estimator for the average treatment effect that capitalizes on conjugancy of the beta and binomial distributions. Through simulations, we show the utility of horseshoe priors and Bayesian additive regression trees paired with our new estimator, while demonstrating the importance of including variance from the treatment and outcome models. Cardiac stent data with almost 500 confounders and 9000 patients illustrate approaches and compare among existing frequentist alternatives. △ Less

Submitted 9 October, 2017; originally announced October 2017.

Comments: 17 pages (without references/appendix), 2 figures, 4 tables