-
Benchmarking tools for a priori identifiability analysis
Authors:
Xabier Rey Barreiro,
Alejandro F. Villaverde
Abstract:
The structural identifiability and the observability of a model determine the possibility of inferring its parameters and states by observing its outputs. These properties should be analysed before attempting to calibrate a model. Unfortunately, such \textit{a priori} analysis can be challenging, since it requires symbolic calculations that often have a high computational cost. In recent years a n…
▽ More
The structural identifiability and the observability of a model determine the possibility of inferring its parameters and states by observing its outputs. These properties should be analysed before attempting to calibrate a model. Unfortunately, such \textit{a priori} analysis can be challenging, since it requires symbolic calculations that often have a high computational cost. In recent years a number of software tools have been developed for this task, mostly in the systems biology community but also in other disciplines. These tools have vastly different features and capabilities, and a critical assessment of their performance is still lacking. Here we present a comprehensive study of the computational resources available for analysing structural identifiability. We consider 12 software tools developed in 7 programming languages (Matlab, Maple, Mathematica, Julia, Python, Reduce, and Maxima), and evaluate their performance using a set of 25 case studies created from 21 models. Our results reveal their strengths and weaknesses, provide guidelines for choosing the most appropriate tool for a given problem, and highlight opportunities for future developments.
△ Less
Submitted 17 October, 2022; v1 submitted 20 July, 2022;
originally announced July 2022.
-
STRIKE-GOLDD 4.0: user-friendly, efficient analysis of structural identifiability and observability
Authors:
Sandra Díaz-Seoane,
Xabier Rey-Barreiro,
Alejandro F. Villaverde
Abstract:
Structural identifiability and observability are desirable properties of systems biology models. Many software toolboxes have been developed for their analysis in the last decades. STRIKE-GOLDD is a generally applicable tool that can analyse non-linear, non-rational ODE models with unknown inputs. However, this generality comes at the expense of a lower computational efficiency than other tools. H…
▽ More
Structural identifiability and observability are desirable properties of systems biology models. Many software toolboxes have been developed for their analysis in the last decades. STRIKE-GOLDD is a generally applicable tool that can analyse non-linear, non-rational ODE models with unknown inputs. However, this generality comes at the expense of a lower computational efficiency than other tools. Here we present STRIKE-GOLDD 4.0, which includes a new algorithm, ProbObsTest, specifically designed for the analysis of rational models. ProbObsTest is significantly faster than the FISPO algorithm - which was already available in older versions of the toolbox - when applied to computationally expensive models. An important feature of both algorithms is their ability to analyse models with unknown inputs. Thus, their coexistence in the same toolbox provides a combination of general applicability and computational efficiency. STRIKE-GOLDD 4.0 is implemented as a free and open-source Matlab toolbox with a user-friendly graphical interface. It is available under a GPLv3 license and it can be downloaded from GitHub at https://github.com/afvillaverde/strike-goldd.
△ Less
Submitted 15 July, 2022;
originally announced July 2022.
-
A protocol for dynamic model calibration
Authors:
Alejandro F. Villaverde,
Dilan Pathirana,
Fabian Fröhlich,
Jan Hasenauer,
Julio R. Banga
Abstract:
Ordinary differential equation models are nowadays widely used for the mechanistic description of biological processes and their temporal evolution. These models typically have many unknown and non-measurable parameters, which have to be determined by fitting the model to experimental data. In order to perform this task, known as parameter estimation or model calibration, the modeller faces challe…
▽ More
Ordinary differential equation models are nowadays widely used for the mechanistic description of biological processes and their temporal evolution. These models typically have many unknown and non-measurable parameters, which have to be determined by fitting the model to experimental data. In order to perform this task, known as parameter estimation or model calibration, the modeller faces challenges such as poor parameter identifiability, lack of sufficiently informative experimental data, and the existence of local minima in the objective function landscape. These issues tend to worsen with larger model sizes, increasing the computational complexity and the number of unknown parameters. An incorrectly calibrated model is problematic because it may result in inaccurate predictions and misleading conclusions. For non-expert users, there are a large number of potential pitfalls. Here, we provide a protocol that guides the user through all the steps involved in the calibration of dynamic models. We illustrate the methodology with two models, and provide all the code required to reproduce the results and perform the same analysis on new models. Our protocol provides practitioners and researchers in biological modelling with a one-stop guide that is at the same time compact and sufficiently comprehensive to cover all aspects of the problem.
△ Less
Submitted 26 May, 2021; v1 submitted 25 May, 2021;
originally announced May 2021.
-
Repairing dynamic models: a method to obtain identifiable and observable reparameterizations with mechanistic insights
Authors:
Gemma Massonis,
Julio R. Banga,
Alejandro F. Villaverde
Abstract:
Mechanistic dynamic models allow for a quantitative and systematic interpretation of data and the generation of testable hypotheses. However, these models are often over-parameterized, leading to non-identifiability and non-observability, i.e. the impossibility of inferring their parameters and state variables. The lack of structural identifiability and observability (SIO) compromises a model's ab…
▽ More
Mechanistic dynamic models allow for a quantitative and systematic interpretation of data and the generation of testable hypotheses. However, these models are often over-parameterized, leading to non-identifiability and non-observability, i.e. the impossibility of inferring their parameters and state variables. The lack of structural identifiability and observability (SIO) compromises a model's ability to make predictions and provide insight. Here we present a methodology, AutoRepar, that corrects SIO deficiencies automatically, yielding reparameterized models that are structurally identifiable and observable. The reparameterization preserves the mechanistic meaning of selected variables, and has the exact same dynamics and input-output map** as the original model. We implement AutoRepar as an extension of the STRIKE-GOLDD software toolbox for SIO analysis, applying it to several models from the literature to demonstrate its ability to repair their structural deficiencies. AutoRepar increases the applicability of mechanistic models, enabling them to provide reliable information about their parameters and dynamics.
△ Less
Submitted 18 December, 2020; v1 submitted 17 December, 2020;
originally announced December 2020.
-
Structural Identifiability and Observability of Compartmental Models of the COVID-19 Pandemic
Authors:
Gemma Massonis,
Julio R. Banga,
Alejandro F. Villaverde
Abstract:
The recent coronavirus disease (COVID-19) outbreak has dramatically increased the public awareness and appreciation of the utility of dynamic models. At the same time, the dissemination of contradictory model predictions has highlighted their limitations. If some parameters and/or state variables of a model cannot be determined from output measurements, its ability to yield correct insights -- as…
▽ More
The recent coronavirus disease (COVID-19) outbreak has dramatically increased the public awareness and appreciation of the utility of dynamic models. At the same time, the dissemination of contradictory model predictions has highlighted their limitations. If some parameters and/or state variables of a model cannot be determined from output measurements, its ability to yield correct insights -- as well as the possibility of controlling the system -- may be compromised. Epidemic dynamics are commonly analysed using compartmental models, and many variations of such models have been used for analysing and predicting the evolution of the COVID-19 pandemic. In this paper we survey the different models proposed in the literature, assembling a list of 36 model structures and assessing their ability to provide reliable information. We address the problem using the control theoretic concepts of structural identifiability and observability. Since some parameters can vary during the course of an epidemic, we consider both the constant and time-varying parameter assumptions. We analyse the structural identifiability and observability of all of the models, considering all plausible choices of outputs and time-varying parameters, which leads us to analyse 255 different model versions. We classify the models according to their structural identifiability and observability under the different assumptions and discuss the implications of the results. We also illustrate with an example several alternative ways of remedying the lack of observability of a model. Our analyses provide guidelines for choosing the most informative model for each purpose, taking into account the available knowledge and measurements.
△ Less
Submitted 25 June, 2020;
originally announced June 2020.
-
Nonlinear observability algorithms with known and unknown inputs: analysis and implementation
Authors:
Nerea Martínez,
Alejandro F. Villaverde
Abstract:
The observability of a dynamical system is affected by the presence of external inputs, either known (such as control actions) or unknown (disturbances). Inputs of unknown magnitude are especially detrimental for observability, and they also complicate its analysis. Hence the availability of computational tools capable of analysing the observability of nonlinear systems with unknown inputs has bee…
▽ More
The observability of a dynamical system is affected by the presence of external inputs, either known (such as control actions) or unknown (disturbances). Inputs of unknown magnitude are especially detrimental for observability, and they also complicate its analysis. Hence the availability of computational tools capable of analysing the observability of nonlinear systems with unknown inputs has been limited until lately. Two symbolic algorithms based on differential geometry, ORC-DF and FISPO, have been recently proposed for this task, but their critical analysis and comparison is still lacking. Here we perform an analytical comparison of both algorithms and evaluate their performance on a set of problems, discussing their strengths and limitations. Additionally, we use these analyses to provide insights about certain aspects of the relationship between inputs and observability. We find that, while ORC-DF and FISPO follow a similar approach, they differ in key aspects that can have a substantial influence on their applicability and computational cost. The FISPO algorithm is more generally applicable, since it can analyse any nonlinear ODE model. The ORC-DF algorithm analyses models that are affine in the inputs, and if those models have known inputs it is sometimes more efficient. Thus, the optimal choice of a method depends on the characteristics of the problem under consideration. To facilitate the use of both algorithms we implement the ORC-DF algorithm in a new version of STRIKE-GOLDD, a MATLAB toolbox for structural identifiability and observability analysis. Since this software tool already had an implementation of the FISPO algorithm, the new release allows modellers and model users the convenience of choosing between different algorithms in a single tool, without changing the coding of their model.
△ Less
Submitted 1 June, 2020;
originally announced June 2020.
-
PEtab -- interoperable specification of parameter estimation problems in systems biology
Authors:
Leonard Schmiester,
Yannik Schälte,
Frank T. Bergmann,
Tacio Camba,
Erika Dudkin,
Janine Egert,
Fabian Fröhlich,
Lara Fuhrmann,
Adrian L. Hauber,
Svenja Kemmer,
Polina Lakrisenko,
Carolin Loos,
Simon Merkt,
Wolfgang Müller,
Dilan Pathirana,
Elba Raimúndez,
Lukas Refisch,
Marcus Rosenblatt,
Paul L. Stapor,
Philipp Städter,
Dantong Wang,
Franz-Georg Wieland,
Julio R. Banga,
Jens Timmer,
Alejandro F. Villaverde
, et al. (4 additional authors not shown)
Abstract:
Reproducibility and reusability of the results of data-based modeling studies are essential. Yet, there has been -- so far -- no broadly supported format for the specification of parameter estimation problems in systems biology. Here, we introduce PEtab, a format which facilitates the specification of parameter estimation problems using Systems Biology Markup Language (SBML) models and a set of ta…
▽ More
Reproducibility and reusability of the results of data-based modeling studies are essential. Yet, there has been -- so far -- no broadly supported format for the specification of parameter estimation problems in systems biology. Here, we introduce PEtab, a format which facilitates the specification of parameter estimation problems using Systems Biology Markup Language (SBML) models and a set of tab-separated value files describing the observation model and experimental data as well as parameters to be estimated. We already implemented PEtab support into eight well-established model simulation and parameter estimation toolboxes with hundreds of users in total. We provide a Python library for validation and modification of a PEtab problem and currently 20 example parameter estimation problems based on recent studies. Specifications of PEtab, the PEtab Python library, as well as links to examples, and all supporting software tools are available at https://github.com/PEtab-dev/PEtab, a snapshot is available at https://doi.org/10.5281/zenodo.3732958. All original content is available under permissive licenses.
△ Less
Submitted 7 August, 2020; v1 submitted 2 April, 2020;
originally announced April 2020.
-
Observability and Structural Identifiability of Nonlinear Biological Systems
Authors:
Alejandro F. Villaverde
Abstract:
Observability is a modelling property that describes the possibility of inferring the internal state of a system from observations of its output. A related property, structural identifiability, refers to the theoretical possibility of determining the parameter values from the output. In fact, structural identifiability becomes a particular case of observability if the parameters are considered as…
▽ More
Observability is a modelling property that describes the possibility of inferring the internal state of a system from observations of its output. A related property, structural identifiability, refers to the theoretical possibility of determining the parameter values from the output. In fact, structural identifiability becomes a particular case of observability if the parameters are considered as constant state variables. It is possible to simultaneously analyse the observability and structural identifiability of a model using the conceptual tools of differential geometry. Many complex biological processes can be described by systems of nonlinear ordinary differential equations, and can therefore be analysed with this approach. The purpose of this review article is threefold: (I) to serve as a tutorial on observability and structural identifiability of nonlinear systems, using the differential geometry approach for their analysis; (II) to review recent advances in the field; and (III) to identify open problems and suggest new avenues for research in this area.
△ Less
Submitted 11 December, 2018;
originally announced December 2018.
-
Dynamical compensation and structural identifiability: analysis, implications, and reconciliation
Authors:
Alejandro F. Villaverde,
Julio R. Banga
Abstract:
The concept of dynamical compensation has been recently introduced to describe the ability of a biological system to keep its output dynamics unchanged in the face of varying parameters. Here we show that, according to its original definition, dynamical compensation is equivalent to lack of structural identifiability. This is relevant if model parameters need to be estimated, which is often the ca…
▽ More
The concept of dynamical compensation has been recently introduced to describe the ability of a biological system to keep its output dynamics unchanged in the face of varying parameters. Here we show that, according to its original definition, dynamical compensation is equivalent to lack of structural identifiability. This is relevant if model parameters need to be estimated, which is often the case in biological modelling. This realization prompts us to warn that care should we taken when using an unidentifiable model to extract biological insight: the estimated values of structurally unidentifiable parameters are meaningless, and model predictions about unmeasured state variables can be wrong. Taking this into account, we explore alternative definitions of dynamical compensation that do not necessarily imply structural unidentifiability. Accordingly, we show different ways in which a model can be made identifiable while exhibiting dynamical compensation. Our analyses enable the use of the new concept of dynamical compensation in the context of parameter identification, and reconcile it with the desirable property of structural identifiability.
△ Less
Submitted 24 March, 2017;
originally announced March 2017.
-
Dynamical compensation in biological systems as a particular case of structural non-identifiability
Authors:
Alejandro F. Villaverde,
Julio R. Banga
Abstract:
Dynamical compensation (DC) has been recently defined as the ability of a biological system to keep its output dynamics unchanged in the face of varying parameters. This concept is purported to describe a design principle that provides robustness to physiological circuits. Here we note the similitude between DC and Structural Identifiability (SI), and we argue that the former can be explained in t…
▽ More
Dynamical compensation (DC) has been recently defined as the ability of a biological system to keep its output dynamics unchanged in the face of varying parameters. This concept is purported to describe a design principle that provides robustness to physiological circuits. Here we note the similitude between DC and Structural Identifiability (SI), and we argue that the former can be explained in terms of (lack of) the latter. We propose to exploit this fact by using currently existing tools for SI analysis to perform DC analysis. We demonstrate the feasibility of this approach with four physiological circuits, for which we confirm the correspondence between DC and lack of SI. We also warn that care should we taken when using an unidentifiable model to extract biological insight, since lack of SI can be the result of an inappropriate choice of model structure and therefore not necessarily a sign of biological robustness.
△ Less
Submitted 10 January, 2017;
originally announced January 2017.
-
BioPreDyn-bench: benchmark problems for kinetic modelling in systems biology
Authors:
Alejandro F Villaverde,
David Henriques,
Kieran Smallbone,
Sophia Bongard,
Joachim Schmid,
Damjan Cicin-Sain,
Anton Crombach,
Julio Saez-Rodriguez,
Klaus Mauch,
Eva Balsa-Canto,
Pedro Mendes,
Johannes Jaeger,
Julio R Banga
Abstract:
Dynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to fac…
▽ More
Dynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to facilitate these tasks. However, there is a lack of suitable benchmark problems which allow a fair and systematic evaluation and comparison of these contributions. Here we present BioPreDyn-bench, a set of challenging parameter estimation problems which aspire to serve as reference test cases in this area. This set comprises six problems including medium and large-scale kinetic models of the bacterium E. coli, baker's yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The level of description includes metabolism, transcription, signal transduction, and development. For each problem we provide (i) a basic description and formulation, (ii) implementations ready-to-run in several formats, (iii) computational results obtained with specific solvers, (iv) a basic analysis and interpretation. This suite of benchmark problems can be readily used to evaluate and compare parameter estimation methods. Further, it can also be used to build test problems for sensitivity and identifiability analysis, model reduction and optimal experimental design methods. The suite, including codes and documentation, can be freely downloaded from http://www.iim.csic.es/%7egingproc/biopredynbench/.
△ Less
Submitted 22 July, 2014;
originally announced July 2014.
-
MEIGO: an open-source software suite based on metaheuristics for global optimization in systems biology and bioinformatics
Authors:
Jose A Egea,
David Henriques,
Thomas Cokelaer,
Alejandro F Villaverde,
Julio R Banga,
Julio Saez-Rodriguez
Abstract:
Optimization is key to solve many problems in computational biology. Global optimization methods provide a robust methodology, and metaheuristics in particular have proven to be the most efficient methods for many applications. Despite their utility, there is limited availability of metaheuristic tools. We present MEIGO, an R and Matlab optimization toolbox (also available in Python via a wrapper…
▽ More
Optimization is key to solve many problems in computational biology. Global optimization methods provide a robust methodology, and metaheuristics in particular have proven to be the most efficient methods for many applications. Despite their utility, there is limited availability of metaheuristic tools. We present MEIGO, an R and Matlab optimization toolbox (also available in Python via a wrapper of the R version), that implements metaheuristics capable of solving diverse problems arising in systems biology and bioinformatics: enhanced scatter search method (eSS) for continuous nonlinear programming (cNLP) and mixed-integer programming (MINLP) problems, and variable neighborhood search (VNS) for Integer Programming (IP) problems. Both methods can be run on a single-thread or in parallel using a cooperative strategy. The code is supplied under GPLv3 and is available at \url{http://www.iim.csic.es/~gingproc/meigo.html}. Documentation and examples are included. The R package has been submitted to Bioconductor. We evaluate MEIGO against optimization benchmarks, and illustrate its applicability to a series of case studies in bioinformatics and systems biology, outperforming other state-of-the-art methods. MEIGO provides a free, open-source platform for optimization, that can be applied to multiple domains of systems biology and bioinformatics. It includes efficient state of the art metaheuristics, and its open and modular structure allows the addition of further methods.
△ Less
Submitted 22 November, 2013;
originally announced November 2013.
