Showing 1–9 of 9 results for author: Hernandez, J G

Lexidate: Model Evaluation and Selection with Lexicase

Authors: Jose Guadalupe Hernandez, Anil Kumar Saini, Jason H. Moore

Abstract: Automated machine learning streamlines the task of finding effective machine learning pipelines by automating model training, evaluation, and selection. Traditional evaluation strategies, like cross-validation (CV), generate one value that averages the accuracy of a pipeline's predictions. This single value, however, may not fully describe the generalizability of the pipeline. Here, we present Lex… ▽ More Automated machine learning streamlines the task of finding effective machine learning pipelines by automating model training, evaluation, and selection. Traditional evaluation strategies, like cross-validation (CV), generate one value that averages the accuracy of a pipeline's predictions. This single value, however, may not fully describe the generalizability of the pipeline. Here, we present Lexicase-based Validation (lexidate), a method that uses multiple, independent prediction values for selection. Lexidate splits training data into a learning set and a selection set. Pipelines are trained on the learning set and make predictions on the selection set. The predictions are graded for correctness and used by lexicase selection to identify parent pipelines. Compared to 10-fold CV, lexicase reduces the training time. We test the effectiveness of three lexidate configurations within the Tree-based Pipeline Optimization Tool 2 (TPOT2) package on six OpenML classification tasks. In one configuration, we detected no difference in the accuracy of the final model returned from TPOT2 on most tasks compared to 10-fold CV. All configurations studied here returned similar or less complex final pipelines compared to 10-fold CV. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Prototipo de video juego activo basado en una cámara 3D para motivar la actividad física en niños y adultos mayores

Authors: Benjamín Ojeda Magaña, José Guadalupe Robledo Hernández, Leopoldo Gómez Barba, Victor Manuel Rangel Cobián

Abstract: This document describes the development of a video game prototype designed to encourage physical activity among children and older adults. The prototype consists of a laptop, a camera with 3D sensors, and optionally requires an LCD screen or a projector. The programming component of this prototype was developed in Scratch, a programming language geared towards children, which greatly facilitates t… ▽ More This document describes the development of a video game prototype designed to encourage physical activity among children and older adults. The prototype consists of a laptop, a camera with 3D sensors, and optionally requires an LCD screen or a projector. The programming component of this prototype was developed in Scratch, a programming language geared towards children, which greatly facilitates the creation of a game tailored to the users' preferences. The idea to create such a prototype originated from the desire to offer an option that promotes physical activity among children and adults, given that a lack of physical exercise is a predominant factor in the development of chronic degenerative diseases such as diabetes and hypertension, to name the most common. As a result of this initiative, an active video game prototype was successfully developed, based on a **-pong game, which allows both children and adults to interact in a fun way while encouraging the performance of physical activities that can positively impact the users' health. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Comments: 13 pages, in Spanish language, 11 figures

ACM Class: I.4.9

Phylogeny-informed fitness estimation

Authors: Alexander Lale**i, Matthew Andres Moreno, Jose Guadalupe Hernandez, Emily Dolson

Abstract: Phylogenies (ancestry trees) depict the evolutionary history of an evolving population. In evolutionary computing, a phylogeny can reveal how an evolutionary algorithm steers a population through a search space, illuminating the step-by-step process by which any solutions evolve. Thus far, phylogenetic analyses have primarily been applied as post-hoc analyses used to deepen our understanding of ex… ▽ More Phylogenies (ancestry trees) depict the evolutionary history of an evolving population. In evolutionary computing, a phylogeny can reveal how an evolutionary algorithm steers a population through a search space, illuminating the step-by-step process by which any solutions evolve. Thus far, phylogenetic analyses have primarily been applied as post-hoc analyses used to deepen our understanding of existing evolutionary algorithms. Here, we investigate whether phylogenetic analyses can be used at runtime to augment parent selection procedures during an evolutionary search. Specifically, we propose phylogeny-informed fitness estimation, which exploits a population's phylogeny to estimate fitness evaluations. We evaluate phylogeny-informed fitness estimation in the context of the down-sampled lexicase and cohort lexicase selection algorithms on two diagnostic analyses and four genetic programming (GP) problems. Our results indicate that phylogeny-informed fitness estimation can mitigate the drawbacks of down-sampled lexicase, improving diversity maintenance and search space exploration. However, the extent to which phylogeny-informed fitness estimation improves problem-solving success for GP varies by problem, subsampling method, and subsampling level. This work serves as an initial step toward improving evolutionary algorithms by exploiting runtime phylogenetic analysis. △ Less

Submitted 6 June, 2023; originally announced June 2023.

Comments: Submitted as contribution to GPTP XX

A suite of diagnostic metrics for characterizing selection schemes

Authors: Jose Guadalupe Hernandez, Alexander Lale**i, Charles Ofria

Abstract: Benchmark suites are crucial for assessing the performance of evolutionary algorithms, but the constituent problems are often too complex to provide clear intuition about an algorithm's strengths and weaknesses. To address this gap, we introduce DOSSIER ("Diagnostic Overview of Selection Schemes In Evolutionary Runs"), a diagnostic suite initially composed of eight handcrafted metrics. These metri… ▽ More Benchmark suites are crucial for assessing the performance of evolutionary algorithms, but the constituent problems are often too complex to provide clear intuition about an algorithm's strengths and weaknesses. To address this gap, we introduce DOSSIER ("Diagnostic Overview of Selection Schemes In Evolutionary Runs"), a diagnostic suite initially composed of eight handcrafted metrics. These metrics are designed to empirically measure specific capacities for exploitation, exploration, and their interactions. We consider exploitation both with and without constraints, and we divide exploration into two aspects: diversity exploration (the ability to simultaneously explore multiple pathways) and valley-crossing exploration (the ability to cross wider and wider fitness valleys). We apply DOSSIER to six popular selection schemes: truncation, tournament, fitness sharing, lexicase, nondominated sorting, and novelty search. Our results confirm that simple schemes (e.g., tournament and truncation) emphasized exploitation. For more sophisticated schemes, however, our diagnostics revealed interesting dynamics. Lexicase selection performed moderately well across all diagnostics that did not incorporate valley crossing, but faltered dramatically whenever valleys were present, performing worse than even random search. Fitness sharing was the only scheme to effectively contend with valley crossing but it struggled with the other diagnostics. Our study highlights the utility of using diagnostics to gain nuanced insights into selection scheme characteristics, which can inform the design of new selection methods. △ Less

Submitted 23 October, 2023; v1 submitted 28 April, 2022; originally announced April 2022.

Comments: Incorporated valley crossing diagnostics and results. Also refactored paper to focus on three key problem characteristics

Predicting the impact of treatments over time with uncertainty aware neural differential equations

Authors: Edward De Brouwer, Javier González Hernández, Stephanie Hyland

Abstract: Predicting the impact of treatments from observational data only still represents a majorchallenge despite recent significant advances in time series modeling. Treatment assignments are usually correlated with the predictors of the response, resulting in a lack of data support for counterfactual predictions and therefore in poor quality estimates. Developments in causal inference have lead to meth… ▽ More Predicting the impact of treatments from observational data only still represents a majorchallenge despite recent significant advances in time series modeling. Treatment assignments are usually correlated with the predictors of the response, resulting in a lack of data support for counterfactual predictions and therefore in poor quality estimates. Developments in causal inference have lead to methods addressing this confounding by requiring a minimum level of overlap. However,overlap is difficult to assess and usually notsatisfied in practice. In this work, we propose Counterfactual ODE (CF-ODE), a novel method to predict the impact of treatments continuously over time using Neural Ordinary Differential Equations equipped with uncertainty estimates. This allows to specifically assess which treatment outcomes can be reliably predicted. We demonstrate over several longitudinal data sets that CF-ODE provides more accurate predictions and more reliable uncertainty estimates than previously available methods. △ Less

Submitted 24 February, 2022; originally announced February 2022.

Journal ref: AISTATS 2022

What can phylogenetic metrics tell us about useful diversity in evolutionary algorithms?

Authors: Jose Guadalupe Hernandez, Alexander Lale**i, Emily Dolson

Abstract: It is generally accepted that "diversity" is associated with success in evolutionary algorithms. However, diversity is a broad concept that can be measured and defined in a multitude of ways. To date, most evolutionary computation research has measured diversity using the richness and/or evenness of a particular genotypic or phenotypic property. While these metrics are informative, we hypothesize… ▽ More It is generally accepted that "diversity" is associated with success in evolutionary algorithms. However, diversity is a broad concept that can be measured and defined in a multitude of ways. To date, most evolutionary computation research has measured diversity using the richness and/or evenness of a particular genotypic or phenotypic property. While these metrics are informative, we hypothesize that other diversity metrics are more strongly predictive of success. Phylogenetic diversity metrics are a class of metrics popularly used in biology, which take into account the evolutionary history of a population. Here, we investigate the extent to which 1) these metrics provide different information than those traditionally used in evolutionary computation, and 2) these metrics better predict the long-term success of a run of evolutionary computation. We find that, in most cases, phylogenetic metrics behave meaningfully differently from other diversity metrics. Moreover, our results suggest that phylogenetic diversity is indeed a better predictor of success. △ Less

Submitted 28 August, 2021; originally announced August 2021.

Comments: 21 page, 7 figures. Presented Genetic Programming in Theory and Practice, 2021

ACM Class: I.2.2

An Exploration of Exploration: Measuring the ability of lexicase selection to find obscure pathways to optimality

Authors: Jose Guadalupe Hernandez, Alexander Lale**i, Charles Ofria

Abstract: Parent selection algorithms (selection schemes) steer populations through a problem's search space, often trading off between exploitation and exploration. Understanding how selection schemes affect exploitation and exploration within a search space is crucial to tackling increasingly challenging problems. Here, we introduce an "exploration diagnostic" that diagnoses a selection scheme's capacity… ▽ More Parent selection algorithms (selection schemes) steer populations through a problem's search space, often trading off between exploitation and exploration. Understanding how selection schemes affect exploitation and exploration within a search space is crucial to tackling increasingly challenging problems. Here, we introduce an "exploration diagnostic" that diagnoses a selection scheme's capacity for search space exploration. We use our exploration diagnostic to investigate the exploratory capacity of lexicase selection and several of its variants: epsilon lexicase, down-sampled lexicase, cohort lexicase, and novelty-lexicase. We verify that lexicase selection out-explores tournament selection, and we show that lexicase selection's exploratory capacity can be sensitive to the ratio between population size and the number of test cases used for evaluating candidate solutions. Additionally, we find that relaxing lexicase's elitism with epsilon lexicase can further improve exploration. Both down-sampling and cohort lexicase -- two techniques for applying random subsampling to test cases -- degrade lexicase's exploratory capacity; however, we find that cohort partitioning better preserves lexicase's exploratory capacity than down-sampling. Finally, we find evidence that novelty-lexicase's addition of novelty test cases can degrade lexicase's capacity for exploration. Overall, our findings provide hypotheses for further exploration and actionable insights and recommendations for using lexicase selection. Additionally, this work demonstrates the value of selection scheme diagnostics as a complement to more conventional benchmarking approaches to selection scheme analysis. △ Less

Submitted 26 July, 2021; v1 submitted 20 July, 2021; originally announced July 2021.

Comments: Changes to the axis labels and added funding sources to acknowledgments

High-resolution spectroscopy of Boyajian's star during optical dimming events

Authors: M. J. Martínez González, C. González-Fernández, A. Asensio Ramos, H. Socas Navarro, C. Westendorp Plaza, T. S. Boyajian, J. T. Wright, A. Collier Cameron, J. González Hernández, G. Holgado, G. M. Kennedy, T. Masseron, E. Molinari, J. Saario, S. Simón-Díaz, B. Toledo-Padrón

Abstract: Boyajian's star is an apparently normal main sequence F-type star with a very unusual light curve. The dip** activity of the star, discovered during the Kepler mission, presents deep, asymmetric, and aperiodic events. Here we present high resolution spectroscopic follow-up during some dimming events recorded post-Kepler observations, from ground-based telescopes. We analise data from the HERMES,… ▽ More Boyajian's star is an apparently normal main sequence F-type star with a very unusual light curve. The dip** activity of the star, discovered during the Kepler mission, presents deep, asymmetric, and aperiodic events. Here we present high resolution spectroscopic follow-up during some dimming events recorded post-Kepler observations, from ground-based telescopes. We analise data from the HERMES, HARPS-N and FIES spectrographs to characterise the stellar atmosphere and to put some constraints on the hypotheses that have appeared in the literature concerning the occulting elements. The star's magnetism, if existing, is not extreme. The spots on the surface, if present, would occupy 0.02% of the area, at most. The chromosphere, irrespective of the epoch of observation, is hotter than the values expected from radiative equilibrium, meaning that the star has some degree of activity. We find no clear evidence of the interstellar medium nor exocoments being responsible for the dimmings of the light curve. However, we detect at 1-2 sigma level, a decrease of the radial velocity of the star during the first dip recorded after the \emph{\emph{Kepler}} observations. We claim the presence of an optically thick object with likely inclined and high impact parameter orbits that produces the observed Rossiter-McLaughlin effect. △ Less

Submitted 17 December, 2018; originally announced December 2018.

Comments: submitted to MNRAS

The EChO science case

Authors: Giovanna Tinetti, Pierre Drossart, Paul Eccleston, Paul Hartogh, Kate Isaak, Martin Linder, Christophe Lovis, Giusi Micela, Marc Ollivier, Ludovic Puig, Ignasi Ribas, Ignas Snellen, Bruce Swinyard. France Allard, Joanna Barstow, James Cho, Athena Coustenis, Charles Cockell, Alexandre Correia, Leen Decin, Remco de Kok, Pieter Deroo, Therese Encrenaz, Francois Forget, Alistair Glasse, Caitlin Griffith , et al. (326 additional authors not shown)

Abstract: The discovery of almost 2000 exoplanets has revealed an unexpectedly diverse planet population. Observations to date have shown that our Solar System is certainly not representative of the general population of planets in our Milky Way. The key science questions that urgently need addressing are therefore: What are exoplanets made of? Why are planets as they are? What causes the exceptional divers… ▽ More The discovery of almost 2000 exoplanets has revealed an unexpectedly diverse planet population. Observations to date have shown that our Solar System is certainly not representative of the general population of planets in our Milky Way. The key science questions that urgently need addressing are therefore: What are exoplanets made of? Why are planets as they are? What causes the exceptional diversity observed as compared to the Solar System? EChO (Exoplanet Characterisation Observatory) has been designed as a dedicated survey mission for transit and eclipse spectroscopy capable of observing a large and diverse planet sample within its four-year mission lifetime. EChO can target the atmospheres of super-Earths, Neptune-like, and Jupiter-like planets, in the very hot to temperate zones (planet temperatures of 300K-3000K) of F to M-type host stars. Over the next ten years, several new ground- and space-based transit surveys will come on-line (e.g. NGTS, CHEOPS, TESS, PLATO), which will specifically focus on finding bright, nearby systems. The current rapid rate of discovery would allow the target list to be further optimised in the years prior to EChO's launch and enable the atmospheric characterisation of hundreds of planets. Placing the satellite at L2 provides a cold and stable thermal environment, as well as a large field of regard to allow efficient time-critical observation of targets randomly distributed over the sky. A 1m class telescope is sufficiently large to achieve the necessary spectro-photometric precision. The spectral coverage (0.5-11 micron, goal 16 micron) and SNR to be achieved by EChO, thanks to its high stability and dedicated design, would enable a very accurate measurement of the atmospheric composition and structure of hundreds of exoplanets. △ Less

Submitted 19 February, 2015; originally announced February 2015.

Comments: 50 pages, 30 figures. Experimental Astronomy