PMLB v1.0: An open source dataset collection for benchmarking machine learning methods
Authors:
Joseph D. Romano,
Trang T. Le,
William La Cava,
John T. Gregg,
Daniel J. Goldberg,
Natasha L. Ray,
Praneel Chakraborty,
Daniel Himmelstein,
Weixuan Fu,
Jason H. Moore
Abstract:
Motivation: Novel machine learning and statistical modeling studies rely on standardized comparisons to existing methods using well-studied benchmark datasets. Few tools exist that provide rapid access to many of these datasets through a standardized, user-friendly interface that integrates well with popular data science workflows.
Results: This release of PMLB provides the largest collection of…
▽ More
Motivation: Novel machine learning and statistical modeling studies rely on standardized comparisons to existing methods using well-studied benchmark datasets. Few tools exist that provide rapid access to many of these datasets through a standardized, user-friendly interface that integrates well with popular data science workflows.
Results: This release of PMLB provides the largest collection of diverse, public benchmark datasets for evaluating new machine learning and data science methods aggregated in one location. v1.0 introduces a number of critical improvements developed following discussions with the open-source community.
Availability: PMLB is available at https://github.com/EpistasisLab/pmlb. Python and R interfaces for PMLB can be installed through the Python Package Index and Comprehensive R Archive Network, respectively.
△ Less
Submitted 6 April, 2021; v1 submitted 30 November, 2020;
originally announced December 2020.
Jointly Predicting Job Performance, Personality, Cognitive Ability, Affect, and Well-Being
Authors:
Pablo Robles-Granda,
Suwen Lin,
Xian Wu,
Sidney D'Mello,
Gonzalo J. Martinez,
Koustuv Saha,
Kari Nies,
Gloria Mark,
Andrew T. Campbell,
Munmun De Choudhury,
Anind D. Dey,
Julie Gregg,
Ted Grover,
Stephen M. Mattingly,
Shayan Mirjafari,
Edward Moskal,
Aaron Striegel,
Nitesh V. Chawla
Abstract:
Assessment of job performance, personalized health and psychometric measures are domains where data-driven and ubiquitous computing exhibits the potential of a profound impact in the future. Existing techniques use data extracted from questionnaires, sensors (wearable, computer, etc.), or other traits, to assess well-being and cognitive attributes of individuals. However, these techniques can neit…
▽ More
Assessment of job performance, personalized health and psychometric measures are domains where data-driven and ubiquitous computing exhibits the potential of a profound impact in the future. Existing techniques use data extracted from questionnaires, sensors (wearable, computer, etc.), or other traits, to assess well-being and cognitive attributes of individuals. However, these techniques can neither predict individual's well-being and psychological traits in a global manner nor consider the challenges associated to processing the data available, that is incomplete and noisy. In this paper, we create a benchmark for predictive analysis of individuals from a perspective that integrates: physical and physiological behavior, psychological states and traits, and job performance. We design data mining techniques as benchmark and uses real noisy and incomplete data derived from wearable sensors to predict 19 constructs based on 12 standardized well-validated tests. The study included 757 participants who were knowledge workers in organizations across the USA with varied work roles. We developed a data mining framework to extract the meaningful predictors for each of the 19 variables under consideration. Our model is the first benchmark that combines these various instrument-derived variables in a single framework to understand people's behavior by leveraging real uncurated data from wearable, mobile, and social media sources. We verify our approach experimentally using the data obtained from our longitudinal study. The results show that our framework is consistently reliable and capable of predicting the variables under study better than the baselines when prediction is restricted to the noisy, incomplete data.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
