Opportunities for machine learning in scientific discovery

Abstract

Technological advancements have substantially increased computational power and data availability, enabling the application of powerful machine-learning (ML) techniques across various fields. However, our ability to leverage ML methods for scientific discovery, i.e. to obtain fundamental and formalized knowledge about natural processes, is still in its infancy. In this review, we explore how the scientific community can increasingly leverage ML techniques to achieve scientific discoveries. We observe that the applicability and opportunity of ML depends strongly on the nature of the problem domain, and whether we have full (e.g., turbulence), partial (e.g., computational biochemistry), or no (e.g., neuroscience) a-priori knowledge about the governing equations and physical properties of the system. Although challenges remain, principled use of ML is opening up new avenues for fundamental scientific discoveries. Throughout these diverse fields, there is a theme that ML is enabling researchers to embrace complexity in observational data that was previously intractable to classic analysis and numerical investigations.

Keywords: machine learning (ML); deep learning (DL); artificial intelligence (AI); scientific discovery; physics; life sciences; computer science

Introduction

Machine learning (ML) has shown great potential to transform a broad range of domains [1, 2, 3, 4, 5], and it is increasingly being applied to problems in science and engineering. ML has been widely used for predictive tasks in these areas, and despite an initial promising phase where ML methods have outperformed well-established techniques [6, 7, 8, 9, 10], such predictive applications are starting to exhibit diminishing returns. There are, by contrast, increasing opportunities in the academic usage of ML for scientific discovery, i.e. answering challenging scientific questions while leveraging existing fundamental knowledge. Such focus on scientific discovery can move the frontiers of science forward when progress in more traditional methods has slowed. Furthermore, the development of novel and more powerful ML methods can help to tackle some open subjects in the context of predictions from scarce, noisy, or incomplete data, out-of-sample generalization, extreme-event predictions and predictions under uncertainty.

Science is fundamentally interested in identifying structure and explaining the world, the many systems that constitute it, and the laws that govern it. The development of the scientific method as we know it has taken time and been progressive, from early forms found in antiquity (Aristotle, first forms of causality [11], etc.) to rigorous methodologies based on objective, quantitative, and mathematical evidence [12]. This approach has encountered great success, although the “unreasonable effectiveness of mathematics in natural sciences” [13] increasingly appears to encounter difficulty making further progress with the “complexity” observed in more challenging problems and real-world data. Complexity is hard to define but often arises from non-linearity, high dimensionality, and multiscale dynamics in space and time, leading to a system comprising a very large number of parts and mechanisms that cannot easily be simplified or approximated. Traditional tools inherited from the rigorous mathematical tradition of the 1700s to late 1900s are challenged by these problems: still today, we can observe and, to some degree, simulate and reproduce in numerical models complex problems such as turbulence, processes in the brain, biological systems, etc. But arguably, we do not fully understand these phenomena at a deeper level.

Machine learning comprises a growing set of algorithms, enabled by high-performance computing and increasingly vast data, that show incredible promise for handling complexity [14, 15]. Interestingly, artificial neural networks are themselves “complex” artificial systems that can now perform “complex” tasks for which no traditional algorithms are known. Even though their governing laws are based on simple operations, they are fully observable, and they run deterministically on microprocessors, we cannot in general explain the decisions and outputs generated by neural networks. Regardless, neural networks are enabling novel discoveries with profound impact, such as the first new class of antibiotics in decades [16]. This gives rise to several challenges, such as the need for explainable artificial intelligence (XAI) [17, 18].

Symbolic approaches, such as gene-expression programming, sparse regression, and sparse Bayesian learning  [19, 20, 21, 22, 23], have led to successful approaches over the years (see e.g. the work on “machine scientists” from the 1980s [24]). Such approaches are limited by exponential complexity with the size of the search space, motivating recent attempts to combine symbolic and deep-learning-based strategies [25]; however, they can still be used to enable advances in science, including the discovery of novel materials [26]. This observation also raises several fundamental, quasi-philosophical questions: is a complex system complex enough to understand its complexity, or can it only understand lesser complexities? This is also related to the question of how much AI can discover that is not already contained in the training data [27]. These considerations naturally raise the fundamental question of what opportunities (and challenges) are offered by the growing impact of data-driven methods and machine learning in science: will we, as some authors have suggested, observe the “unreasonable effectiveness of data” [28], and of deep learning [29]? And as a consequence, what does this mean in practice for future scientific discoveries? How and to what degrees can we extract fundamental understanding, not just computational recipes, from modern machine learning, and by using which approaches? In this sense, and for this article, we do not consider optimization or automation tasks as “scientific discovery”. Still, being able to effectively solve governing equations (for instance, partial differential equations, PDEs) through ML would be considered as “scientific discovery”, as long as this enables addressing fundamental questions that cannot be answered with other tools.

There have been some recent studies on the possible impact of ML on science. For instance, Wang et al. [30] focus on the potential of self-supervised learning and generative models for generally achieving improvements in scientific experiments and simulations. Zenil et al. [31] are more focused on how AI can help formulate scientific questions and answer them, emphasizing the potential of large language models (LLMs) in this context. The latter are raising a number of open questions in terms of potential benefits and threats of their application in science [32]. Fajardo-Fontiveros et al. [33] discuss the interesting question of when it is possible to learn models from data and what is the maximum level of noise acceptable for learning the correct model.

In the present work we adopt a different approach, analyzing separately the potential of AI to enable/facilitate scientific discovery in three types of problems: (i) problems where the governing phenomenological equations are entirely known. This corresponds to cases where it would be directly possible, given enough computational power, to simulate and reproduce the system entirely. (ii) Cases where we have some partial knowledge regarding the governing equations and/or some physical properties that hold. (iii) Scenarios where nothing is known about the governing equations or the physical properties of the problem under study. We illustrate these categories by considering examples from two concrete application areas: Physical Sciences and Life Sciences. A summary of the various categories and applications is presented in Figure 1, where we discuss the examples of turbulent flows, dark matter, drug discovery and brain research, sorted from more to less knowledge of the governing equations and/or their underlying properties. While the proposed categorization is a convenient schematic view, the lines between full knowledge and partial or no knowledge become increasingly blurred as system complexity grows. Examples given in the following sections illustrate how different uses of ML may support scientific discovery by hel** to tackle complexity.

Complete information is available

There are several applications within Physical Sciences where the underlying governing equations are known perfectly but the high-level global dynamics are still not well understood. Even if, technically, we know the governing quantum equations of chemistry underlying the dynamics of bio-molecules, and subsequently of full organisms, complexity makes biology a partial-knowledge problem, because only very limited parts of complex biological systems can be actually measured or simulated, while extreme complexity makes it impossible to simulate the full human brain. Similarly, turbulent flows of continuous media (i.e. flows characterized by very small values of the Knudsen number $Kn=l_{\text{m.f.p.}}/L$, where $l_{\text{m.f.p.}}$ is the mean free path of molecules constituting the fluid and $L$ is a macroscopic length scale) are well described by the Navier–Stokes equations, which in turn can be derived from the Boltzmann equations describing the kinetics of gases at the microscopic level [34]. Nevertheless, the tiny and highly chaotic small scales in turbulent flows can no longer be measured or simulated as the advection forces become increasingly dominant over diffusion, i.e. as the Reynolds number increases, making turbulence a major unsolved problem [35]. On the other hand, the mean-flow properties or the largest, coherent scales can still be simulated and observed, thus providing a partial knowledge of the system, but we do not have a consistent theory to prove fundamental flow properties based on these, nor describe the detailed interactions among turbulent structures and phenomena they imply.

In this context, supervised ML holds promises for scientific discoveries, as summarized in Figure 2. Indeed, it is possible to generate large amounts of fully resolved DNS data, yielding large training datasets of high quality. Such data can be used, for example, to learn what the most important structures are through explainable deep learning [36], sensitivity analysis [37] or information theory [38]. In such a context, a broad range of ML techniques, such as symbolic regression, reduced-order modeling and autoencoders, can be leveraged to discover novel structures and relationships in flow features and to uncover previously unknown physical properties of turbulence. Similarly, turbulence closure modeling has seen significant advances using supervised-ML techniques [39, 40]. In particular, novel closures have been developed for Reynolds-averaged Navier–Stokes (RANS) [41, 42, 43] and large-eddy simulation (LES) [44] turbulence models. Importantly, these closures are analytic expressions that are interpretable and generalizable, built on sparse symbolic-regression techniques [22]. The broader family of symbolic-regression methods [45, 41, 46] promises to further aid in the development of these improved closure models. Furthermore, accelerated computational-fluid-dynamics codes are also being developed using machine learning, for example to discover improved stencils and numerical schemes for computations [47, 48]. In the context of astrophysical sciences, a supervised classifier denominated SPOCK has allowed to predict from short-term simulations the long-term stability of compact multi-planet systems that required integration of the laws of gravitation over billions of orbital periods [49]. Not only SPOCK was able to predict the long-term stability of small systems similar to those used for its training, but it was able to generalize to large multi-planet systems.

An additional challenge associated with complex physical and biological systems is to perform optimal control, which is a key aspect of many scientific questions as well as industrial applications [50, 51]. In this context there are also many promising applications of ML: specifically, deep reinforcement learning (DRL) is now leading to similar discoveries as AlphaGo [52], but in Physics. It is common to use simulation data, based on the known governing equations of the system, to define the environment with which the DRL agent interacts to develop the best-performing policy. In the case of experimental environments, numerical data can also be used to enhance the measurements. The use of DRL has spread across different physics applications in recent years. Several examples can be found in quantum physics, where DRL was able to find the ground state and describe the unitary time evolution of complex interacting quantum systems [53]. Reinforcement learning has also been employed in astronomy for the adaptive control of astronomy systems [54]. In the field of turbulence research, DRL has allowed to discover novel and previously unknown flow-control strategies which outperform previous state of the art [55]. Similar advances have been done in, e.g., tokamak-instability control [14, 56]. This allows the discovery of novel and more effective ways to tune instability control, which is of great importance in practice. But this may also allow to understand better complexity arising in turbulent systems. By performing an a-posteriori analysis of the novel control laws using traditional scientific methods, one can analyze and explain how such control laws work, providing novel insights into the underlying physics. Moreover, novel physical regimes that are not spontaneously observed, for example, because they lie behind an energy barrier or they are intrinsically unstable, can be discovered through DRL. For instance, application of RL in thermodynamics [57] has allowed to learn previously unknown thermodynamic cycles.

Intractable simulations and optimization of complex systems with known governing equations can be dramatically accelerated using ML. For example, autoencoders can be leveraged to discover effective latent-space representations of complex systems [58]. Such latent spaces can then be used to design effective time integrators and faster simulations [59], or to perform optimization at a lower computational cost [60]. While speeding up simulations is not, per se, a scientific discovery, the improved understanding that comes from faster simulations can be used to perform systematic studies and lead to breakthroughs. For example, in high-energy physics, particle discovery relies on the ability to accurately compare observed detector response data with expectations based on physical models [61]. While the processes of subatomic particle interactions with matter are known, the analytical calculation of the detector response is analytically intractable, and Monte-Carlo methods must be used to simulate the propagation of particles in detectors for comparison with the data [62, 63]. As a consequence, recent advances in high-fidelity fast generative models, such as generative adversarial networks (GANs) [64] or variational autoencoders (VAEs) [65], offer a promising alternative for simulation, gaining orders of magnitude in simulation speed over existing techniques, provided that these methodologies can be developed to achieve the required accuracy, which is a subject of ongoing research [66]. A striking example is given by the recent introduction of LLMs for weather and climate modeling, which is not only revolutionizing weather forecasting [67], but has also the potential to accelerate scientific discoveries in climate change. Studies on Paleoclimates can also be accelerated by leveraging AI to replace or complement the cumbersome coupled resolution of several complex climate processes (e.g. convection, clouds, atmospheric chemistry), and by enabling access to the finer resolutions required to understand regional to local changes in climate [15].

In applied Mathematics and scientific computing, approaches based on DRL similar to AlphaGo [52] can also lead to discoveries. Fawzi et al. [8] discovered novel algorithmic optimizations to accelerate matrix operations. This is important for both practical applications (faster programs, in particular for numerical simulations) and scientific advancement, opening for new algorithmic optimization possibilities that were previously unknown. Large language models (LLMs) are now also being leveraged to generate possible algorithms to approach solutions for classical problems in combinatorics, e.g. the cap set problem [5]. Similar to the matrix-acceleration task, since the validation of suggested solutions can be performed with a classical algorithm, this allows the discovery of strategies and algorithms that are both previously unknown and that can be formally validated and proved for correctness. In both cases, AI is proving useful as a heuristic method to provide good candidate solutions for a problem where the verification of a solution candidate can be done cheaply, e.g. deterministic linear [8] or polynomial [5] time. Note however that deciding how to find, or discover, an adequate candidate is challenging.

Partial information is available

In contrast to the previous section, the following discussion illustrates scientific-discovery techniques enabled by ML where only partial information is available about the governing equations and/or the mechanisms of the phenomenon under study. This occurs in physical or biological systems for which descriptions of the macroscopic behavior are possible according to first principles, however, the microscopic structure is too complex to be completely described or observed. This is the case for instance of complex materials (i.e., composite or textured) or certain types of fluids (e.g., granular flows, multiphase flows, flows with particles or sediments, flows through canopies, etc.). Conversely, some other systems may exhibit elementary microscopic behaviors leading however to extremely complex and/or chaotic macroscopic behaviors that cannot be easily described. In Biology this can be the case of the spread of infectious diseases, whereby the underlying mechanisms can be described by e.g. cellular automata [68], while the collective behavior can be highly complex and hardly predictable [69]. Another example is given by active matter, generally of biological origin, which exhibits a form of “turbulent” spectrum despite being described at the microscopic level by linear equations. This leads to highly chaotic behaviors known as “active turbulence” [70]. In all these cases, only the macroscopic or microscopic governing equations are known, while the rest is unknown or hard to describe.

In such cases, inductive biases [71, 72], i.e. the known information about the system behavior, can be incorporated into the architecture of data-driven methods to exploit established physical or biological knowledge. Such biases may consist of frame invariance, equivariance, or symmetry-based constraints, thereby reducing the sample complexity of the discovery process. Physical constraints, e.g. flow incompressibility, thermodynamic properties, etc., can also be incorparated, leading to the discovery of more compact and interpretable models with better generalization properties (universality). For example, Moya et al. [73] propose a framework for training neural networks where the learning is constrained by the known thermodynamic properties of the system in such a way that the model is general and can be applied to several physical systems (this particular study focuses on fluid mechanics). This enables exploring phenomena through digital twins, where only some data are available and makes it possible to combine data-driven aspects with simulations while preserving generalization properties [74]. Another example of taking advantage of known physical properties of the system is embedding symmetries in autoencoders, intending to develop reduced-order models (ROMs) in physical systems invariant to input transformations [75]. This process makes it possible to train ROMs more effectively without having to learn the symmetries and it enables examining the physics of the system in the latent space more effectively. A general framework to impose and discover symmetries in physical systems for ML applications was proposed by Otto et al. [76]. Such approaches, combining data-driven methods and physics, are essential to achieve novel techniques for scientific discovery. Recent work has shown the potential of novel deep-learning methods for geometric reasoning [9], a possibility that can be critical when dealing with symmetries and other physical properties in systems containing intrinsic structures.

Machine learning can also be used to obtain physical knowledge where partial knowledge is available through the discovery of constitutive laws. Many materials are characterized by complex rheologies that are difficult or impossible to describe using standard modeling approaches. However, costly high-fidelity or ab initio simulations can be produced under various loading configurations and used to indirectly infer the constitutive equations or rheological behaviour. For instance, De Lorenzis and coworkers have recently proposed the EUCLID hybrid finite-element/neural-network framework for learning constitutive equations in hyperelastic solids [77], and the SpaRTA framework initially introduced in Ref. [41] for data-driven discovery of turbulence models has been adapted to the discovery of constitutive equations for elastic solids [78]. Sparse identification has also been used to discover constitutive equations of crystal structures, learning from ab initio calculations or interatomic potentials [79]. Finally, a deep-learning method incorporating strong inductive biases, such as objectivity, consistency, and stability, was developed by As’ad et al. [80] to learn constitutive laws for complex nonlinear materials. In all these examples, which are schematically represented in Figure 3, the macroscopic behaviour of the material follows the conservation laws of Mechanics, and only the constitutive equations are unknown. Mahmoudabadbozchelou et al. [81] introduced the notion of “digital rheometer twin”, where ML methodologies are leveraged to learn the hidden rheology of complex fluids through a limited number of experiments.

Multiscale modeling and stochastic simulations are other areas where learning from simulated data can lead to a real discovery. In multiscale simulations, an appropriate model is available at a small scale (e.g., the fundamental laws of molecular dynamics), and the goal is to learn a model at another scale (e.g., a continuum-scale partial differential equation) from the data generated at the first scale. In stochastic simulations, the governing equations contain uncertain parameters or are driven by randomly fluctuating forcing terms representing subgrid variability and processes (e.g., Langevin equations). Solutions to such problems are typically characterized using probability density functions (PDFs). The goal is to learn the deterministic dynamics of either the PDF of a system state or its statistical moments. While PDF equations can be exact under certain conditions, their derivation requires closure approximations based on field-specific knowledge and can introduce uncontrollable errors. Machine learning can then infer closure terms from the databases, e.g. based on sparse regression [82]. The D-CIPHER method has recently been shown that it can discover many ordinary and partial differential equations [83]. Similarly, when studying biomolecules, their interactions, and generally their functions that serve the mechanisms of a cell, Newton’s laws of motion are commonly used to model molecular dynamics at the atomistic level. However, in these cases, the exact energy functions are not known as these are generally optimized in smaller systems using a combination of techniques [84]. Therefore, the exact energy function governing these simulations always contains some noise and includes necessary compromises to achieve computational efficiency. Furthermore, even if the precise energy functions were known, such simulations are prohibitively expensive [85], and we do not yet have complete predictive rules for larger-scale interactions at the size of biological macromolecules. Here, ML methods provide alternative paths to analyse the energies and dynamics of these systems, with both molecular-dynamics simulations and experiments contributing to generate databases for learning large-scale interactions [86]. They can also serve as a regulariser for ML models, thus enabling the use of a more coarse-grained, and therefore faster, representation [87, 88, 89].

In the context of Life Sciences, structural biology is a very important example where only partial knowledge of the phenomena is available. Yet, ML is leading to significant new scientific discoveries. Fast identification of free-energy minima has been developed for small molecules [90], proteins [7], ligand-binding [91], and mixtures of these [10]. More concretely, one notable recent progress is AlphaFold [7], which employs a complex deep architecture with several embedded inductive biases to fold proteins from their one-dimensional amino-acids sequence into their three-dimensional (3D) native form. Although AlphaFold does not provide any insights into how proteins fold, it has been proven to be very valuable for the structural-biology community. Among AlphaFold’s architectural biases, the most notable are: (i) multiple sequence alignment (MSA), a source of co-evolutionary signal of protein folding. In this context, a novel transformer architecture, the EvoFormer, is used, and (ii) structural equivariance is guaranteed by using a 3D equivariant transformer trained to predict a novel loss function, the FAPE loss.

There are several additional examples of ML leading to scientific discovery in systems where partial information is available, for instance through generative models. Such models constitute a powerful tool which has crossed over into popular culture in recent years, especially due to their capability of generating artistic pictures or videos. Recently, generative AI has been used to learn physical models from large datasets by incorporating prior knowledge expressed as constraints on the functional form of the learned model or from axiomatic knowledge and experimental data by combining logical reasoning with symbolic regression [92]. The discovered models enable generalizing known phenomena to new configurations, for example, new geometries or operating conditions. This can help to shed light on the physical phenomena in new scenarios beyond those in which data is available. When it comes to quantum systems, deep reinforcement learning (DRL) has enabled the discovery of novel approaches to put a quantum system in a given state, which provides novel insights into the underlying physics [93, 94]. Furthermore, ML is currently allowing to reduce the noise in quantum-computing systems, while quantum computing is allowing in turn to improve ML performance with reduced energy consumption [95]. Another example is climate, where ML is hel** to develop climate models enabling the characterization of novel physics. This includes establishing new large-eddy simulation (LES) models for climate ensuring stable behavior for long-term forecast [96]. Note that in LES only the larger scales are resolved, whereas the smaller ones need to be represented by a model, which, in this case, is developed through ML. There are also several studies which reflect how ML can help to improve classical weather-prediction systems [97, 98].

AlphaFold [7] has also spurred many innovations in the ML field, e.g. single-sequence methods such as ESMfold [99]. These single-sequence-based methods are based on foundation models trained to recover a masked region of a protein sequence [100], a technique also used in AlphaFold to improve its performance. They have also been used for protein design [101], but have lately been replaced by generative models that take both sequence and structure into account [102]. This highlights that a method not trained directly to uncover scientific insights can be utilised to provide such insights in another scientific discipline. Another such example is the use of diffusion models [103, 104] to generate protein backbone structures suitable for protein design that are routinely tested using AlphaFold. Lately, diffusion and flow-matching models can also be used to altogether bypass the need for simulations when generating a molecular ensemble [105].

No information is available

In various scientific fields, certain phenomena exist whose origins and descriptions remain largely unknown, and we lack known governing equations or foundational physical models to accurately capture their critical aspects. For example, the field of neuroscience has no governing equations from first principles, as there are no known conservation laws, symmetries, or other physics that may be used to derive generalizable differential equations. Even if we have access to equations to describe the behavior of individual molecules or cells, the complexity of the brain makes it unfeasible to simulate at every scale; furthermore, it is not tractable to take enough measurements to initialize such a simulation. At the same time, rapid progress is being made that advances our ability to acquire neural data. For instance, large-scale neural recording and imaging techniques now routinely produce datasets of unprecedented spatial and temporal resolution [106, 107], and advances in connectomics offer detailed views of cell types and neural circuit architecture [108, 109]. Taken together, the state of what is possible in data-driven modeling promises a new era of empirical models that leverage machine learning for discovery in our understanding of neuroscience and behavior.

Full numerical simulations of the brain and behavior of an animal are impossible to construct and infeasible to initialize, but data-driven models that recapitulate key input-output relationships and generate predictions can be crucial tools in future discovery. In scientific fields focused on complex phenomena for which we lack known governing equations, a common approach to gaining understanding is through perturbation experiments. For instance, in systems neuroscience, experimentally activating a specific population of neurons at a particular phase of a visual discrimination task can directly bias the animal’s perception. Such causal experiments allow us to gain insights into this part of the visual perception pathway. Thus, data-driven models are particularly useful when they are constructed in a space that matches experimentally measurable quantities and are capable of generating novel, testable hypotheses. What would happen if we combinatorially activated populations of neurons, and at several different phases of the task? Because data-driven models are not based on known physical equations, such predictions can diverge quite drastically from reality, so we emphasize the importance of a rapid iteration between generating hypotheses and performing experiments, which in turn generate data that continually refine models.

Another way in which ML contributes to discovery is through providing an approach to synthesize diverse experimental data that measure parts of the same system, but collected separately and at different resolutions. For instance, the MICrONS dataset [110] includes both functional-imaging data and structural reconstructions of the same cortical neurons. Although we know that morphological and physiological features of these neurons are related, each dataset is incomplete, and the relationships between them are difficult to describe. In such cases, a data-driven model is a valuable way to formalize these relationships.

Where no a-priori knowledge of governing equations is available, ML methods can serve as an approach to learn dynamics directly from observations. There is a long history of learning differential equations to model neural dynamics, exemplified by the famed Hodgkin–Huxley equations written to describe the electrical action potential based on changes in the conductance of ion channels [111]. There are several categories of modern ML approaches to learn dynamical models. For instance, the SINDy (sparse identification of non-linear dynamics) family of techniques [22] identifies the underlying dynamics of a system from the data and trajectories it generates by using sparse-promoting regression of the transitions matrix, or related sparsity-generating techniques. Related approaches have been used for the same purposes using either genetic algorithms [112] or reinforcement learning [113]. Where we have no governing equations but are aware of some underlying structure, incorporating such constraints to the learning process can yield faster and more robust discovery of equations describing the dynamics. Once a set of equations that govern a system are identified, they can be applied to discover novel mechanisms and to efficiently perform engineering tasks on complex systems.

Another approach to equation discovery is the neural ordinary differential equation model (NODE) [114]. Unlike traditional neural networks that map inputs to outputs through a series of layers with fixed parameters, NODEs formulate the transformation from input to output as the solution to an ODE and learn the dynamics of this transformation over continuous time. While NODE is a powerful and general method, it cannot provide direct scientific insights into the learned functional relationships from the dataset. To address this problem, there has been a recent interest in hybrid approaches that rely on transformer backbones [115] or on shallow neural-network architectures [116] inspired by SINDY [22] for learning differential equations from data. Note that interpretable models may lead to new scientific insights. For instance, such interpretable ML models can be trained on patient data with specific diagnoses or prognoses and then subsequently inspected to discover novel biomarkers for the detection of diseases or outcome-prediction of treatments [117, 118].

From a statistical perspective, a key element that enables the learning of the underlying mechanisms is whether, in addition to observational data (e.g. electronic-health records), interventional data are available. As an example of interventional data, perturbational CRISPR-Ko in single-cell biology [119] already includes genome-wide perturbational experiments, featuring millions of perturbations [120]. With increasingly automated setups becoming available [121, 122], a key research question is how to experimentally design and select the next intervention to facilitate identifying the underlying mechanisms and governing equations. While decades of research have focused on active learning for system identification in linear cases [123], more research is needed to consider learning an underlying equation from data in such a setting when the parametrized model is a NODE and in generally unknown contexts [124, 125, 126].

With no prior knowledge of the functional form of the equations, one often does not observe the variables of interest directly. In these cases, representation learning and latent-space identification are essential for understanding and predicting complex dynamical systems. A common assumption underlying the idea of representation learning is that while the observations might exhibit complex behaviour, the underlying dynamics might be expressible in a simple form in some abstract space. Thus, many techniques for representation learning from high-dimensional data rely on variational autoencoders and latent space embedding by a generative model [127, 128]. By identifying the correct latent spaces, one often hopes to achieve: i) dimensionality reduction, ii) simplifying system characterization, and iii) facilitating interpretation by identifying some form of causal structure [129].

Causality and causal formulations of dynamical systems [130] have been of significant interest in the past years, especially in Earth sciences [131, 132, 133] and molecular Biology [134, 135]. Using data from Biology, Lippe et al. [136] identified causal ODEs using invariance from heterogeneous experiments as a learning signal, while other approaches focus on understanding latent causal factors of a dynamical system by learning disentangled representations from time series [137, 138]. These approaches aim to not only model the latent space but also the temporal dependencies within a sequence [139, 140]. Learning these structured latent spaces is of crucial importance since it provides an effective coordinate system in which the dynamics have a simple representation, which is a key requirement for generalization and interpretability [141]. In Figure 4 we provide a schematic representation of the identification of an underlying causal structure from observations where the variables of interest are not directly observed.

Orthogonal to the approaches of using AI to learn and identify dynamical systems is the current research trend of investigating different ODEs and solvers for improving the general class of diffusion models [142]. The model class of diffusion, denoising autoencoders, or flow-matching models [143] has the key advantage that it is amenable to theoretical investigations while providing state-of-the-art results in various domains. This is an example where not only systems identification can profit from machine learning but where general ML approaches profit from the insights and cross pollination with control theory and existing knowledge of dynamical systems [144, 145]. While these models have found widespread success not only in image generation [146] but in many scientific applications from protein modeling [147] to materials science [148], we currently have very limited causal or scientific insights from these large pre-trained models [149], which may provide ample opportunities for further research.

In acquiring data from complex systems with unknown governing equations, our measurements are often indirect and incomplete, so that they require extensive processing before they are suitable for modeling and learning using the approaches described above. In such cases, applications of ML have critically facilitated data collection and have potential to further advance scientific discovery, by expanding the realm of the observable. For instance, where neural recordings are incomplete or corrupted, machine learning approaches to imputation and generative modeling can produce more complete timeseries data that improve downstream applications [150, 151, 152]. In large-scale image analysis, several notable examples highlight how once laborious manual annotation has been successfully automated by advanced computer vision, turning large quantities of unstructured images or movies into structured scientific data. In microscopy, automated segmentation and profiling have made tractable the analysis of heterogenous cell population as well as their development in time [153, 154]. Automated segmentation and image analysis also enabled the assembly of high value datasets to the community, including brain connectomes at cellular and synaptic resolution [155, 108, 156, 109, 110] . In animal behavior and ethology, tracking the body movements of one or many individuals transform video data into analyzable kinematics and poses, which can then be related to the underlying neural computational and to their impact on social interactions and behavior [157, 158, 159, 160, 161]. Taken together, it is hard to overstate the ongoing impact of machine learning as a critical tool that, when used in conjunction with other approaches, catalyzes scientific discovery.

Conclusions and outlook

Machine-learning methods have already enabled several scientific and technological advancements that were still considered out of reach just a few years back, from effectively solving image classification and segmentation to winning the game of Go. In this article, we have illustrated how contemporary developments in data-driven methods now provide unique opportunities enabling fundamental scientific discovery, focusing on state-of-the-art methods that provide means for pushing the frontiers of science beyond what has been achievable so far. To provide concrete examples of this potential, we have categorised available approaches according to the level of classical scientific knowledge available in the problems to address. These range from settings where governing equations are well known and partially understood to settings where virtually nothing is formally known about the underlying governing equations and principles. As highlighted in the summarizing Table 1, the potential of ML for performing new scientific discoveries spans various methodological approaches. These can enable scientific discovery across a wide spectrum, such as revealing underlying governing principles for physical systems, enabling parametric evaluation of highly complex systems, inducing unknown behaviour in systems for which we only have partial deterministic understanding, and causally inferring principles in systems of complex multiscale nature. The potential of ML for performing scientific discoveries covers a wide range of application areas, from theoretical to applied settings, across Physics, Mathematics, Chemistry and Life Sciences. We argue that continued focus on such discovering tasks represents one of the more fruitful and beneficial usage of machine learning in modern-day scientific practice, enabling scientific leaps in time frames previously unimaginable.

Pushing the scientific frontiers using data-driven methods, however, comes with its own set of challenges. Whilst a clear benefit of ML approaches is associated with their ability to learn and efficiently represent complex systems, this more often than not requires a significant amount of training data to successfully capture the underlying phenomena. In some instances data availability might not be a primary hurdle, however, in instances of scientific discovery data scarcity is frequently an obstacle, for example when causally describing infrequent astrophysical events, inferring descriptions of yet-to-be-tested (or yet-to-be-discovered) pharmaceutical agents, or achieving a mechanistic understanding of pathophysiological phenomena of rare disease. Interestingly, complementary ML approaches can either facilitate the creation of suitable training data (e.g. speeding up computational simulations), or avoid the need for extensive datasets (e.g. by self-supervised out-of-distribution generalization, dimension folding, etc.). As a consequence, even ML techniques that do not lead to scientific discovery per se might facilitate discovery in instances where available training data are scarce. The validation of ML-driven scientific discovery is another challenge, where the underlying governing principles might not exist to confirm output descriptions. However, validation of ML-driven discoveries should not be viewed any differently than validation of deterministically driven discoveries, where hypothesis-driven investigation, observational confirmation and comparative benchmarking can all help to confirm inferred behaviour or descriptions.

Lastly, when employing ML for scientific discovery, a challenge lies in overcoming the black-box nature of standard applied deep-learning frameworks to gain formal knowledge. A black-box setup utilizing a trained network is by construction not directly open for human insight [18]: in particular, internalized neural-network data are hidden in the network weights and not easily interpreted, making it challenging to gain formal knowledge and understanding. In the context of scientific discovery, this is a major challenge for the formalization of knowledge, as well as for public dissemination and acceptance. Both explainable [36] and interpretable [162] machine learning offer an alternative in which human oversight is preserved, however, the development and applicability of these methods is still challenging in many applications. Suppose explainable and/or interpretable ML can be deployed. In that case, the discovered new knowledge or proposed scientific advances can be grounded and validated in existing notions and know-hows, securing scientifically sound knowledge progression.

Scientific communities across a wide range of disciplines are now, following the rapid development of ML techniques, benefiting from incorporating these methods into their research methodology and toolbox, opening new opportunities for scientific discoveries. While this comes with new challenges, we believe that the recent development shows that we can expect further evolution of ML techniques adapted to various specific needs across disciplines. These are likely to provide opportunities to mitigate the limitations outlined in recent works, enabling new discoveries to take place.

Author contributions

RV and HA initiated the idea for this article following a workshop celebrated in November 2022 at KTH. All the authors contributed equally to the rest of this work.

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figure 1: Schematic representation of the various applications of ML for scientific discovery, depending on the amount of knowledge available in each category. A number of examples are provided, including turbulent flows [163], dark matter [164], drug discovery [165] and brain research. Other relevant figures were adapted from Refs. [166, 71].

Figure 2: Schematic representation of ML directions to enable scientific discoveries when complete information about the governing equations is available. In such a case, both supervised, unsupervised, and reinforcement-learning methodologies can be leveraged. Supervised and unsupervised methodologies are made possible by the ability to generate large datasets of synthetic data simulated from the governing equations. This allows to deploy a variety of ML techniques that can discover complex hidden relations, nonlinear coordinate systems, hidden dynamics or solve problems that are otherwise intractable. Reinforcement learning can also be used by coupling it to the physics simulator, which has already proven successful at discovering previously unknown control strategies and regimes of complex systems, or to generate high-quality heuristic guesses that can be tested in the case of problems where solution verification is easy, but the suggestion of good candidate solutions is hard. Some panels were adapted from Refs [167, 7].

Figure 3: Example of machine learning applied to a case where partial knowledge is available about the underlying system, illustrating a model (for instance a flow with complex rheology or a flow through a porous medium) which depends on a set of known inputs $\mathbf{x}$ (e.g. geometry, boundary conditions, etc.) as well as on a set of hidden (unobservable) variables describing, e.g., the fluid constitutive behavior. The latter may involve small-scale phenomena that can be difficult or impossible to describe. In such conditions, experimental or numerical data for observable quantities (e.g. velocity fields or stresses) can be used to infer the unknown field by training a machine learning model (here represented as a neural network, although other ML approaches are possible). The model is subjected to a set of available physical constraints (e.g. positivity, symmetries or invariances). The whole process allows, on one hand, to train a data-driven closure model for the hidden variables $\alpha$ and, on the other hand, to gain a-posteriori physical knowledge on the fluid constitutive properties.

Figure 4: Schematic representation of a model (for instance the observed symptoms of an unknown or complex disease within a population, or observed opinion dynamics within a social network) where the behavior as observed in data depends on an unknown dynamic or causal structure. The observed behavior or dynamics might occur on several different spatial and temporal scales, and the observed data might reflect more or less aspects of the underlying system. In such conditions, representation-learning methods can be employed to distill out an explanation of the observed data, in the form of a system of ODEs, or as a causal-graph representation.

Table 1: Summarizing overview of the opportunities for machine learning in scientific discovery. Based on the differentiation presented in our work, the level of prior, deterministic, knowledge (left) can be used to differentiate methods (second right) and applications (right) across which scientific advancements can be made by means of dedicated AI systems. This also allows for various modes of discovery (second left), ranging from cases where machine learning enables discovery by allowing for efficient computational usage, parametrics sweeps, etc.; to cases where machine learning is used to causally infer underlying mechanistic behaviours in complex multidisciplinary systems.