Facilitating Bioinformatics Reproducibility
Authors:
Christopher R. Keefe,
Matthew R. Dillon,
Chloe Herman,
Mary Jewell,
Colin V. Wood,
Evan Bolyen,
J. Gregory Caporaso
Abstract:
Study reproducibility is essential to corroborate, build on, and learn from the results of scientific research but is notoriously challenging in bioinformatics, which often involves large data sets and complex analytic workflows involving many different tools. Additionally many biologists aren't trained in how to effectively record their bioinformatics analysis steps to ensure reproducibility, so…
▽ More
Study reproducibility is essential to corroborate, build on, and learn from the results of scientific research but is notoriously challenging in bioinformatics, which often involves large data sets and complex analytic workflows involving many different tools. Additionally many biologists aren't trained in how to effectively record their bioinformatics analysis steps to ensure reproducibility, so critical information is often missing. Software tools used in bioinformatics can automate provenance tracking of the results they generate, removing most barriers to bioinformatics reproducibility. Here we present an implementation of that idea, Provenance Replay, a tool for generating new executable code from results generated with the QIIME 2 bioinformatics platform, and discuss considerations for bioinformatics developers who wish to implement similar functionality in their software.
△ Less
Submitted 18 May, 2023;
originally announced May 2023.
Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others
Authors:
Kanishk Gandhi,
Gala Stojnic,
Brenden M. Lake,
Moira R. Dillon
Abstract:
To achieve human-like common sense about everyday life, machine learning systems must understand and reason about the goals, preferences, and actions of other agents in the environment. By the end of their first year of life, human infants intuitively achieve such common sense, and these cognitive achievements lay the foundation for humans' rich and complex understanding of the mental states of ot…
▽ More
To achieve human-like common sense about everyday life, machine learning systems must understand and reason about the goals, preferences, and actions of other agents in the environment. By the end of their first year of life, human infants intuitively achieve such common sense, and these cognitive achievements lay the foundation for humans' rich and complex understanding of the mental states of others. Can machines achieve generalizable, commonsense reasoning about other agents like human infants? The Baby Intuitions Benchmark (BIB) challenges machines to predict the plausibility of an agent's behavior based on the underlying causes of its actions. Because BIB's content and paradigm are adopted from developmental cognitive science, BIB allows for direct comparison between human and machine performance. Nevertheless, recently proposed, deep-learning-based agency reasoning models fail to show infant-like reasoning, leaving BIB an open challenge.
△ Less
Submitted 11 February, 2022; v1 submitted 23 February, 2021;
originally announced February 2021.
