Analysis of the first Genetic Engineering Attribution Challenge
Authors:
Oliver M. Crook,
Kelsey Lane Warmbrod,
Greg Lipstein,
Christine Chung,
Christopher W. Bakerlee,
T. Greg McKelvey Jr.,
Shelly R. Holland,
Jacob L. Swett,
Kevin M. Esvelt,
Ethan C. Alley,
William J. Bradshaw
Abstract:
The ability to identify the designer of engineered biological sequences -- termed genetic engineering attribution (GEA) -- would help ensure due credit for biotechnological innovation, while holding designers accountable to the communities they affect. Here, we present the results of the first Genetic Engineering Attribution Challenge, a public data-science competition to advance GEA. Top-scoring…
▽ More
The ability to identify the designer of engineered biological sequences -- termed genetic engineering attribution (GEA) -- would help ensure due credit for biotechnological innovation, while holding designers accountable to the communities they affect. Here, we present the results of the first Genetic Engineering Attribution Challenge, a public data-science competition to advance GEA. Top-scoring teams dramatically outperformed previous models at identifying the true lab-of-origin of engineered sequences, including an increase in top-1 and top-10 accuracy of 10 percentage points. A simple ensemble of prizewinning models further increased performance. New metrics, designed to assess a model's ability to confidently exclude candidate labs, also showed major improvements, especially for the ensemble. Most winning teams adopted CNN-based machine-learning approaches; however, one team achieved very high accuracy with an extremely fast neural-network-free approach. Future work, including future competitions, should further explore a wide diversity of approaches for bringing GEA technology into practical use.
△ Less
Submitted 14 October, 2021;
originally announced October 2021.
