-
Sundials/ML: Connecting OCaml to the Sundials Numeric Solvers
Authors:
Timothy Bourke,
Jun Inoue,
Marc Pouzet
Abstract:
This paper describes the design and implementation of a comprehensive OCaml interface to the Sundials library of numeric solvers for ordinary differential equations, differential algebraic equations, and non-linear equations. The interface provides a convenient and memory-safe alternative to using Sundials directly from C and facilitates application development by integrating with higher-level lan…
▽ More
This paper describes the design and implementation of a comprehensive OCaml interface to the Sundials library of numeric solvers for ordinary differential equations, differential algebraic equations, and non-linear equations. The interface provides a convenient and memory-safe alternative to using Sundials directly from C and facilitates application development by integrating with higher-level language features, like garbage-collected memory management, algebraic data types, and exceptions. Our benchmark results suggest that the interface overhead is acceptable: the standard examples are rarely twice as slow in OCaml than in C, and often less than 50% slower. The challenges in interfacing with Sundials are to efficiently and safely share data structures between OCaml and C, to support multiple implementations of vector operations and linear solvers through a common interface, and to manage calls and error signalling to and from OCaml. We explain how we overcame these difficulties using a combination of standard techniques such as phantom types and polymorphic variants, and carefully crafted data representations.
△ Less
Submitted 30 December, 2018;
originally announced December 2018.
-
Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning
Authors:
Jun Inoue,
Yoriyuki Yamagata,
Yuqi Chen,
Christopher M. Poskitt,
Jun Sun
Abstract:
In this paper, we propose and evaluate the application of unsupervised machine learning to anomaly detection for a Cyber-Physical System (CPS). We compare two methods: Deep Neural Networks (DNN) adapted to time series data generated by a CPS, and one-class Support Vector Machines (SVM). These methods are evaluated against data from the Secure Water Treatment (SWaT) testbed, a scaled-down but fully…
▽ More
In this paper, we propose and evaluate the application of unsupervised machine learning to anomaly detection for a Cyber-Physical System (CPS). We compare two methods: Deep Neural Networks (DNN) adapted to time series data generated by a CPS, and one-class Support Vector Machines (SVM). These methods are evaluated against data from the Secure Water Treatment (SWaT) testbed, a scaled-down but fully operational raw water purification plant. For both methods, we first train detectors using a log generated by SWaT operating under normal conditions. Then, we evaluate the performance of both methods using a log generated by SWaT operating under 36 different attack scenarios. We find that our DNN generates fewer false positives than our one-class SVM while our SVM detects slightly more anomalies. Overall, our DNN has a slightly better F measure than our SVM. We discuss the characteristics of the DNN and one-class SVM used in this experiment, and compare the advantages and disadvantages of the two methods.
△ Less
Submitted 25 September, 2017; v1 submitted 15 September, 2017;
originally announced September 2017.
-
Operational Semantics of Process Monitors
Authors:
Jun Inoue,
Yoriyuki Yamagata
Abstract:
CSPe is a specification language for runtime monitors that can directly express concurrency in a bottom-up manner that composes the system from simpler, interacting components. It includes constructs to explicitly flag failures to the monitor, which unlike deadlocks and livelocks in conventional process algebras, propagate globally and aborts the whole system's execution. Although CSPe has a trace…
▽ More
CSPe is a specification language for runtime monitors that can directly express concurrency in a bottom-up manner that composes the system from simpler, interacting components. It includes constructs to explicitly flag failures to the monitor, which unlike deadlocks and livelocks in conventional process algebras, propagate globally and aborts the whole system's execution. Although CSPe has a trace semantics along with an implementation demonstrating acceptable performance, it lacks an operational semantics. An operational semantics is not only more accessible than trace semantics but also indispensable for ensuring the correctness of the implementation. Furthermore, a process algebra like CSPe admits multiple denotational semantics appropriate for different purposes, and an operational semantics is the basis for justifying such semantics' integrity and relevance. In this paper, we develop an SOS-style operational semantics for CSPe, which properly accounts for explicit failures and will serve as a basis for further study of its properties, its optimization, and its use in runtime verification.
△ Less
Submitted 15 May, 2017;
originally announced May 2017.
-
Online compressed sensing
Authors:
Paulo V. Rossi,
Yoshiyuki Kabashima,
Jun-ichi Inoue
Abstract:
In this paper, we explore the possibilities and limitations of recovering sparse signals in an online fashion. Employing a mean field approximation to the Bayes recursion formula yields an online signal recovery algorithm that can be performed with a computational cost that is linearly proportional to the signal length per update. Analysis of the resulting algorithm indicates that the online algor…
▽ More
In this paper, we explore the possibilities and limitations of recovering sparse signals in an online fashion. Employing a mean field approximation to the Bayes recursion formula yields an online signal recovery algorithm that can be performed with a computational cost that is linearly proportional to the signal length per update. Analysis of the resulting algorithm indicates that the online algorithm asymptotically saturates the optimal performance limit achieved by the offline method in the presence of Gaussian measurement noise, while differences in the allowable computational costs may result in fundamental gaps of the achievable performance in the absence of noise.
△ Less
Submitted 16 September, 2015;
originally announced September 2015.
-
PENCIL: Towards a Platform-Neutral Compute Intermediate Language for DSLs
Authors:
Riyadh Baghdadi,
Albert Cohen,
Serge Guelton,
Sven Verdoolaege,
Jun Inoue,
Tobias Grosser,
Georgia Kouveli,
Alexey Kravets,
Anton Lokhmotov,
Cedric Nugteren,
Fraser Waters,
Alastair F. Donaldson
Abstract:
We motivate the design and implementation of a platform-neutral compute intermediate language (PENCIL) for productive and performance-portable accelerator programming.
We motivate the design and implementation of a platform-neutral compute intermediate language (PENCIL) for productive and performance-portable accelerator programming.
△ Less
Submitted 22 February, 2013;
originally announced February 2013.
-
Pattern-recalling processes in quantum Hopfield networks far from saturation
Authors:
Jun-ichi Inoue
Abstract:
As a mathematical model of associative memories, the Hopfield model was now well-established and a lot of studies to reveal the pattern-recalling process have been done from various different approaches. As well-known, a single neuron is itself an uncertain, noisy unit with a finite unnegligible error in the input-output relation. To model the situation artificially, a kind of 'heat bath' that sur…
▽ More
As a mathematical model of associative memories, the Hopfield model was now well-established and a lot of studies to reveal the pattern-recalling process have been done from various different approaches. As well-known, a single neuron is itself an uncertain, noisy unit with a finite unnegligible error in the input-output relation. To model the situation artificially, a kind of 'heat bath' that surrounds neurons is introduced. The heat bath, which is a source of noise, is specified by the 'temperature'. Several studies concerning the pattern-recalling processes of the Hopfield model governed by the Glauber-dynamics at finite temperature were already reported. However, we might extend the 'thermal noise' to the quantum-mechanical variant. In this paper, in terms of the stochastic process of quantum-mechanical Markov chain Monte Carlo method (the quantum MCMC), we analytically derive macroscopically deterministic equations of order parameters such as 'overlap' in a quantum-mechanical variant of the Hopfield neural networks (let us call "quantum Hopfield model" or "quantum Hopfield networks"). For the case in which non-extensive number $p$ of patterns are embedded via asymmetric Hebbian connections, namely, $p/N \to 0$ for the number of neuron $N \to \infty$ ('far from saturation'), we evaluate the recalling processes for one of the built-in patterns under the influence of quantum-mechanical noise.
△ Less
Submitted 19 March, 2011;
originally announced March 2011.
-
Statistical mechanics of digital halftoning
Authors:
Jun-ichi Inoue,
Yohei Saika,
Masato Okada
Abstract:
We consider the problem of digital halftoning from the view point of statistical mechanics. The digital halftoning is a sort of image processing, namely, representing each grayscale in terms of black and white binary dots. The digital halftoning is achieved by making use of the threshold mask, namely, for each pixel, the halftoned binary pixel is determined as black if the original grayscale pixel…
▽ More
We consider the problem of digital halftoning from the view point of statistical mechanics. The digital halftoning is a sort of image processing, namely, representing each grayscale in terms of black and white binary dots. The digital halftoning is achieved by making use of the threshold mask, namely, for each pixel, the halftoned binary pixel is determined as black if the original grayscale pixel is greater than or equal to the mask value and is determined as white vice versa. To determine the optimal value of the mask on each pixel for a given original grayscale image, we first assume that the human-eyes might recognize the black and white binary halftoned image as the corresponding grayscale one by linear filters. The Hamiltonian is constructed as a distance between the original and the recognized images which is written in terms of the threshold mask. We are confirmed that the system described by the Hamiltonian is regarded as a kind of antiferromagnetic Ising model with quenched disorders. By searching the ground state of the Hamiltonian, we obtain the optimal threshold mask and the resulting halftoned binary dots simultaneously. From the power-spectrum analysis, we find that the binary dots image is physiologically plausible from the view point of human-eyes modulation properties. We also propose a theoretical framework to investigate statistical performance of inverse digital halftoning, that is, the inverse process of halftoning. From the Bayesian inference view point, we rigorously show that the Bayes-optimal inverse-halftoning is achieved on a specific condition which is very similar to the so-called Nishimori line in the research field of spin glasses.
△ Less
Submitted 8 November, 2010;
originally announced November 2010.
-
Optimization of artificial flockings by means of anisotropy measurements
Authors:
Motohiro Makiguchi,
Jun-ichi Inoue
Abstract:
An effective procedure to determine the optimal parameters appearing in artificial flockings is proposed in terms of optimization problems. We numerically examine genetic algorithms (GAs) to determine the optimal set of parameters such as the weights for three essential interactions in BOIDS by Reynolds (1987) under `zero-collision' and `no-breaking-up' constraints. As a fitness function (the ener…
▽ More
An effective procedure to determine the optimal parameters appearing in artificial flockings is proposed in terms of optimization problems. We numerically examine genetic algorithms (GAs) to determine the optimal set of parameters such as the weights for three essential interactions in BOIDS by Reynolds (1987) under `zero-collision' and `no-breaking-up' constraints. As a fitness function (the energy function) to be maximized by the GA, we choose the so-called the $γ$-value of anisotropy which can be observed empirically in typical flocks of starling. We confirm that the GA successfully finds the solution having a large $γ$-value leading-up to a strong anisotropy. The numerical experience shows that the procedure might enable us to make more realistic and efficient artificial flocking of starling even in our personal computers. We also evaluate two distinct types of interactions in agents, namely, metric and topological definitions of interactions. We confirmed that the topological definition can explain the empirical evidence much better than the metric definition does.
△ Less
Submitted 9 May, 2011; v1 submitted 1 November, 2010;
originally announced November 2010.
-
A Gibbs distribution that learns from GA dynamics
Authors:
Manabu Kitagata,
Jun-ichi Inoue
Abstract:
A general procedure of average-case performance evaluation for population dynamics such as genetic algorithms (GAs) is proposed and its validity is numerically examined. We introduce a learning algorithm of Gibbs distributions from training sets which are gene configurations (strings) generated by GA in order to figure out the statistical properties of GA from the view point of thermodynamics. The…
▽ More
A general procedure of average-case performance evaluation for population dynamics such as genetic algorithms (GAs) is proposed and its validity is numerically examined. We introduce a learning algorithm of Gibbs distributions from training sets which are gene configurations (strings) generated by GA in order to figure out the statistical properties of GA from the view point of thermodynamics. The learning algorithm is constructed by means of minimization of the Kullback-Leibler information between a parametric Gibbs distribution and the empirical distribution of gene configurations. The formulation is applied to the solvable probabilistic models having multi-valley energy landscapes, namely, the spin glass chain and the Sherrington-Kirkpatrick model. By using computer simulations, we discuss the asymptotic behaviour of the effective temperature scheduling and the residual energy induced by the GA dynamics.
△ Less
Submitted 21 April, 2010;
originally announced April 2010.
-
Simultaneous Bayesian inference of motion velocity fields and probabilistic models in successive video-frames described by spatio-temporal MRFs
Authors:
Yuya Inagaki,
Jun-ichi Inoue
Abstract:
We numerically investigate a mean-field Bayesian approach with the assistance of the Markov chain Monte Carlo method to estimate motion velocity fields and probabilistic models simultaneously in consecutive digital images described by spatio-temporal Markov random fields. Preliminary to construction of our procedure, we find that mean-field variables in the iteration diverge due to improper norm…
▽ More
We numerically investigate a mean-field Bayesian approach with the assistance of the Markov chain Monte Carlo method to estimate motion velocity fields and probabilistic models simultaneously in consecutive digital images described by spatio-temporal Markov random fields. Preliminary to construction of our procedure, we find that mean-field variables in the iteration diverge due to improper normalization factor of regularization terms appearing in the posterior. To avoid this difficulty, we rescale the regularization term by introducing a scaling factor and optimizing it by means of minimization of the mean-square error. We confirm that the optimal scaling factor stabilizes the mean-field iterative process of the motion velocity estimation. We next attempt to estimate the optimal values of hyper-parameters including the regularization term, which define our probabilistic model macroscopically, by using the Boltzmann-machine type learning algorithm based on gradient descent of marginal likelihood (type-II likelihood) with respect to the hyper-parameters. In our framework, one can estimate both the probabilistic model (hyper-parameters) and motion velocity fields simultaneously. We find that our motion estimation is much better than the result obtained by Zhang and Hanouer (1995) in which the hyper-parameters are set to some ad-hoc values without any theoretical justification.
△ Less
Submitted 21 April, 2010;
originally announced April 2010.
