-
Accelerating Experimental Design by Incorporating Experimenter Hunches
Authors:
Cheng Li,
Santu Rana,
Sunil Gupta,
Vu Nguyen,
Svetha Venkatesh,
Alessandra Sutti,
David Rubin,
Teo Slezak,
Murray Height,
Mazher Mohammed,
Ian Gibson
Abstract:
Experimental design is a process of obtaining a product with target property via experimentation. Bayesian optimization offers a sample-efficient tool for experimental design when experiments are expensive. Often, expert experimenters have 'hunches' about the behavior of the experimental system, offering potentials to further improve the efficiency. In this paper, we consider per-variable monotoni…
▽ More
Experimental design is a process of obtaining a product with target property via experimentation. Bayesian optimization offers a sample-efficient tool for experimental design when experiments are expensive. Often, expert experimenters have 'hunches' about the behavior of the experimental system, offering potentials to further improve the efficiency. In this paper, we consider per-variable monotonic trend in the underlying property that results in a unimodal trend in those variables for a target value optimization. For example, sweetness of a candy is monotonic to the sugar content. However, to obtain a target sweetness, the utility of the sugar content becomes a unimodal function, which peaks at the value giving the target sweetness and falls off both ways. In this paper, we propose a novel method to solve such problems that achieves two main objectives: a) the monotonicity information is used to the fullest extent possible, whilst ensuring that b) the convergence guarantee remains intact. This is achieved by a two-stage Gaussian process modeling, where the first stage uses the monotonicity trend to model the underlying property, and the second stage uses `virtual' samples, sampled from the first, to model the target value optimization function. The process is made theoretically consistent by adding appropriate adjustment factor in the posterior computation, necessitated because of using the `virtual' samples. The proposed method is evaluated through both simulations and real world experimental design problems of a) new short polymer fiber with the target length, and b) designing of a new three dimensional porous scaffolding with a target porosity. In all scenarios our method demonstrates faster convergence than the basic Bayesian optimization approach not using such `hunches'.
△ Less
Submitted 21 July, 2019;
originally announced July 2019.
-
Bayesian functional optimisation with shape prior
Authors:
Pratibha Vellanki,
Santu Rana,
Sunil Gupta,
David Rubin de Celis Leal,
Alessandra Sutti,
Murray Height,
Svetha Venkatesh
Abstract:
Real world experiments are expensive, and thus it is important to reach a target in minimum number of experiments. Experimental processes often involve control variables that changes over time. Such problems can be formulated as a functional optimisation problem. We develop a novel Bayesian optimisation framework for such functional optimisation of expensive black-box processes. We represent the c…
▽ More
Real world experiments are expensive, and thus it is important to reach a target in minimum number of experiments. Experimental processes often involve control variables that changes over time. Such problems can be formulated as a functional optimisation problem. We develop a novel Bayesian optimisation framework for such functional optimisation of expensive black-box processes. We represent the control function using Bernstein polynomial basis and optimise in the coefficient space. We derive the theory and practice required to dynamically adjust the order of the polynomial degree, and show how prior information about shape can be integrated. We demonstrate the effectiveness of our approach for short polymer fibre design and optimising learning rate schedules for deep networks.
△ Less
Submitted 24 February, 2020; v1 submitted 19 September, 2018;
originally announced September 2018.
-
Accelerated Bayesian Optimization throughWeight-Prior Tuning
Authors:
Alistair Shilton,
Sunil Gupta,
Santu Rana,
Pratibha Vellanki,
Laurence Park,
Cheng Li,
Svetha Venkatesh,
Alessandra Sutti,
David Rubin,
Thomas Dorin,
Alireza Vahid,
Murray Height,
Teo Slezak
Abstract:
Bayesian optimization (BO) is a widely-used method for optimizing expensive (to evaluate) problems. At the core of most BO methods is the modeling of the objective function using a Gaussian Process (GP) whose covariance is selected from a set of standard covariance functions. From a weight-space view, this models the objective as a linear function in a feature space implied by the given covariance…
▽ More
Bayesian optimization (BO) is a widely-used method for optimizing expensive (to evaluate) problems. At the core of most BO methods is the modeling of the objective function using a Gaussian Process (GP) whose covariance is selected from a set of standard covariance functions. From a weight-space view, this models the objective as a linear function in a feature space implied by the given covariance K, with an arbitrary Gaussian weight prior ${\bf w} \sim \mathcal{N} ({\bf 0}, {\bf I})$. In many practical applications there is data available that has a similar (covariance) structure to the objective, but which, having different form, cannot be used directly in standard transfer learning. In this paper we show how such auxiliary data may be used to construct a GP covariance corresponding to a more appropriate weight prior for the objective function. Building on this, we show that we may accelerate BO by modeling the objective function using this (learned) weight prior, which we demonstrate on both test functions and a practical application to short-polymer fibre manufacture.
△ Less
Submitted 6 February, 2020; v1 submitted 20 May, 2018;
originally announced May 2018.
-
Rapid Bayesian optimisation for synthesis of short polymer fiber materials
Authors:
Cheng Li,
David Rubin de Celis Leal,
Santu Rana,
Sunil Gupta,
Alessandra Sutti,
Stewart Greenhill,
Teo Slezak,
Murray Height,
Svetha Venkatesh
Abstract:
The discovery of processes for the synthesis of new materials involves many decisions about process design, operation, and material properties. Experimentation is crucial but as complexity increases, exploration of variables can become impractical using traditional combinatorial approaches. We describe an iterative method which uses machine learning to optimise process development, incorporating m…
▽ More
The discovery of processes for the synthesis of new materials involves many decisions about process design, operation, and material properties. Experimentation is crucial but as complexity increases, exploration of variables can become impractical using traditional combinatorial approaches. We describe an iterative method which uses machine learning to optimise process development, incorporating multiple qualitative and quantitative objectives. We demonstrate the method with a novel fluid processing platform for synthesis of short polymer fibers, and show how the synthesis process can be efficiently directed to achieve material and process objectives.
△ Less
Submitted 16 February, 2018;
originally announced February 2018.
-
Covariance Function Pre-Training with m-Kernels for Accelerated Bayesian Optimisation
Authors:
Alistair Shilton,
Sunil Gupta,
Santu Rana,
Pratibha Vellanki,
Cheng Li,
Laurence Park,
Svetha Venkatesh,
Alessandra Sutti,
David Rubin,
Thomas Dorin,
Alireza Vahid,
Murray Height
Abstract:
The paper presents a novel approach to direct covariance function learning for Bayesian optimisation, with particular emphasis on experimental design problems where an existing corpus of condensed knowledge is present. The method presented borrows techniques from reproducing kernel Banach space theory (specifically m-kernels) and leverages them to convert (or re-weight) existing covariance functio…
▽ More
The paper presents a novel approach to direct covariance function learning for Bayesian optimisation, with particular emphasis on experimental design problems where an existing corpus of condensed knowledge is present. The method presented borrows techniques from reproducing kernel Banach space theory (specifically m-kernels) and leverages them to convert (or re-weight) existing covariance functions into new, problem-specific covariance functions. The key advantage of this approach is that rather than relying on the user to manually select (with some hyperparameter tuning and experimentation) an appropriate covariance function it constructs the covariance function to specifically match the problem at hand. The technique is demonstrated on two real-world problems - specifically alloy design and short-polymer fibre manufacturing - as well as a selected test function.
△ Less
Submitted 12 March, 2018; v1 submitted 14 February, 2018;
originally announced February 2018.
