Croissant: A Metadata Format for ML-Ready Datasets
Authors:
Mubashara Akhtar,
Omar Benjelloun,
Costanza Conforti,
Pieter Gijsbers,
Joan Giner-Miguelez,
Nitisha Jain,
Michael Kuchnik,
Quentin Lhoest,
Pierre Marcenac,
Manil Maskey,
Peter Mattson,
Luis Oala,
Pierre Ruyssen,
Rajat Shinde,
Elena Simperl,
Goeffry Thomas,
Slava Tykhonov,
Joaquin Vanschoren,
Jos van der Velde,
Steffen Vogler,
Carole-Jean Wu
Abstract:
Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is…
▽ More
Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.
△ Less
Submitted 30 May, 2024; v1 submitted 28 March, 2024;
originally announced March 2024.
Blockchain for Genomics: A Systematic Literature Review
Authors:
Mohammed Alghazwi,
Fatih Turkmen,
Joeri van der Velde,
Dimka Karastoyanova
Abstract:
Human genomic data carry unique information about an individual and offer unprecedented opportunities for healthcare. The clinical interpretations derived from large genomic datasets can greatly improve healthcare and pave the way for personalized medicine. Sharing genomic datasets, however, pose major challenges, as genomic data is different from traditional medical data, indirectly revealing inf…
▽ More
Human genomic data carry unique information about an individual and offer unprecedented opportunities for healthcare. The clinical interpretations derived from large genomic datasets can greatly improve healthcare and pave the way for personalized medicine. Sharing genomic datasets, however, pose major challenges, as genomic data is different from traditional medical data, indirectly revealing information about descendants and relatives of the data owner and carrying valid information even after the owner passes away. Therefore, stringent data ownership and control measures are required when dealing with genomic data. In order to provide secure and accountable infrastructure, blockchain technologies offer a promising alternative to traditional distributed systems. Indeed, the research on blockchain-based infrastructures tailored to genomics is on the rise. However, there is a lack of a comprehensive literature review that summarizes the current state-of-the-art methods in the applications of blockchain in genomics. In this paper, we systematically look at the existing work both commercial and academic, and discuss the major opportunities and challenges. Our study is driven by five research questions that we aim to answer in our review. We also present our projections of future research directions which we hope the researchers interested in the area can benefit from.
△ Less
Submitted 16 September, 2022; v1 submitted 19 November, 2021;
originally announced November 2021.
