-
NoX: a Compact Open-Source RISC-V Processor for Multi-Processor Systems-on-Chip
Authors:
Anderson I. Silva,
Altamiro Susin,
Fernanda L. Kastensmidt,
Antonio Carlos S. Beck,
Jose Rodrigo Azambuja
Abstract:
IoT applications are one of the driving forces in making systems energy and power-efficient, given their resource constraints. However, because of security, latency, and transmission, we advocate for local computing through multi-processor systems-on-chip (MPSoCs) for edge computing. The RISC-V ISA has grown in academia and industry due to its flexibility. Still, available open-source cores cannot…
▽ More
IoT applications are one of the driving forces in making systems energy and power-efficient, given their resource constraints. However, because of security, latency, and transmission, we advocate for local computing through multi-processor systems-on-chip (MPSoCs) for edge computing. The RISC-V ISA has grown in academia and industry due to its flexibility. Still, available open-source cores cannot be seamlessly integrated into MPSoCs for a fast time to market. This paper presents NoX, a compact open-source plug-and-play 32-bit RISC-V core designed in System Verilog for efficient data processing in MPSoCs. NoX has a 4-stage single-issue in-order pipeline with full bypass, providing an efficient resource-constrained architecture. Compared to industry and academia resource-constrained RISC-V cores, NoX offers a better resource usage and performance trade-off.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
A Two-Phase Infinite/Finite Low-Level Memory Model
Authors:
Calvin Beck,
Irene Yoon,
Hanxi Chen,
Yannick Zakowski,
Steve Zdancewic
Abstract:
This paper provides a novel approach to reconciling complex low-level memory model features, such as pointer--integer casts, with desired refinements that are needed to justify the correctness of program transformations. The idea is to use a "two-phased" memory model, one with and unbounded memory and corresponding unbounded integer type, and one with a finite memory; the connection between the tw…
▽ More
This paper provides a novel approach to reconciling complex low-level memory model features, such as pointer--integer casts, with desired refinements that are needed to justify the correctness of program transformations. The idea is to use a "two-phased" memory model, one with and unbounded memory and corresponding unbounded integer type, and one with a finite memory; the connection between the two levels is made explicit by our notion of refinement that handles out-of-memory behaviors. This approach allows for more optimizations to be performed and establishes a clear boundary between the idealized semantics of a program and the implementation of that program on finite hardware.
To demonstrate the utility of this idea in practice, we instantiate the two-phase memory model in the context of Zakowski et al.'s VIR semantics, yielding infinite and finite memory models of LLVM IR, including low-level features like undef and bitcast. Both the infinite and finite models, which act as specifications, can provably be refined to executable reference interpreters. The semantics justify optimizations, such as dead-alloca-elimination, that were previously impossible or difficult to prove correct.
△ Less
Submitted 24 April, 2024;
originally announced April 2024.
-
Domain-Independent Dynamic Programming
Authors:
Ryo Kuroiwa,
J. Christopher Beck
Abstract:
For combinatorial optimization problems, model-based paradigms such as mixed-integer programming (MIP) and constraint programming (CP) aim to decouple modeling and solving a problem: the `holy grail' of declarative problem solving. We propose domain-independent dynamic programming (DIDP), a new model-based paradigm based on dynamic programming (DP). While DP is not new, it has typically been imple…
▽ More
For combinatorial optimization problems, model-based paradigms such as mixed-integer programming (MIP) and constraint programming (CP) aim to decouple modeling and solving a problem: the `holy grail' of declarative problem solving. We propose domain-independent dynamic programming (DIDP), a new model-based paradigm based on dynamic programming (DP). While DP is not new, it has typically been implemented as a problem-specific method. We introduce Dynamic Programming Description Language (DyPDL), a formalism to define DP models based on a state transition system, inspired by AI planning. We show that heuristic search algorithms can be used to solve DyPDL models and propose seven DIDP solvers. We experimentally compare our DIDP solvers with commercial MIP and CP solvers (solving MIP and CP models, respectively) on common benchmark instances of eleven combinatorial optimization problem classes. We show that DIDP outperforms MIP in nine problem classes, CP also in nine problem classes, and both MIP and CP in seven.
△ Less
Submitted 31 May, 2024; v1 submitted 24 January, 2024;
originally announced January 2024.
-
PRP Rebooted: Advancing the State of the Art in FOND Planning
Authors:
Christian Muise,
Sheila A. McIlraith,
J. Christopher Beck
Abstract:
Fully Observable Non-Deterministic (FOND) planning is a variant of classical symbolic planning in which actions are nondeterministic, with an action's outcome known only upon execution. It is a popular planning paradigm with applications ranging from robot planning to dialogue-agent design and reactive synthesis. Over the last 20 years, a number of approaches to FOND planning have emerged. In this…
▽ More
Fully Observable Non-Deterministic (FOND) planning is a variant of classical symbolic planning in which actions are nondeterministic, with an action's outcome known only upon execution. It is a popular planning paradigm with applications ranging from robot planning to dialogue-agent design and reactive synthesis. Over the last 20 years, a number of approaches to FOND planning have emerged. In this work, we establish a new state of the art, following in the footsteps of some of the most powerful FOND planners to date. Our planner, PR2, decisively outperforms the four leading FOND planners, at times by a large margin, in 17 of 18 domains that represent a comprehensive benchmark suite. Ablation studies demonstrate the impact of various techniques we introduce, with the largest improvement coming from our novel FOND-aware heuristic.
△ Less
Submitted 19 December, 2023; v1 submitted 18 December, 2023;
originally announced December 2023.
-
Domain-Independent Dynamic Programming: Generic State Space Search for Combinatorial Optimization
Authors:
Ryo Kuroiwa,
J. Christopher Beck
Abstract:
For combinatorial optimization problems, model-based approaches such as mixed-integer programming (MIP) and constraint programming (CP) aim to decouple modeling and solving a problem: the 'holy grail' of declarative problem solving. We propose domain-independent dynamic programming (DIDP), a new model-based paradigm based on dynamic programming (DP). While DP is not new, it has typically been impl…
▽ More
For combinatorial optimization problems, model-based approaches such as mixed-integer programming (MIP) and constraint programming (CP) aim to decouple modeling and solving a problem: the 'holy grail' of declarative problem solving. We propose domain-independent dynamic programming (DIDP), a new model-based paradigm based on dynamic programming (DP). While DP is not new, it has typically been implemented as a problem-specific method. We propose Dynamic Programming Description Language (DyPDL), a formalism to define DP models, and develop Cost-Algebraic A* Solver for DyPDL (CAASDy), a generic solver for DyPDL using state space search. We formalize existing problem-specific DP and state space search methods for combinatorial optimization problems as DP models in DyPDL. Using CAASDy and commercial MIP and CP solvers, we experimentally compare the DP models with existing MIP and CP models, showing that, despite its nascent nature, CAASDy outperforms MIP and CP on a number of common problem classes.
△ Less
Submitted 1 March, 2023; v1 submitted 25 November, 2022;
originally announced November 2022.
-
Negation, Coordination, and Quantifiers in Contextualized Language Models
Authors:
Aikaterini-Lida Kalouli,
Rita Sevastjanova,
Christin Beck,
Maribel Romero
Abstract:
With the success of contextualized language models, much research explores what these models really learn and in which cases they still fail. Most of this work focuses on specific NLP tasks and on the learning outcome. Little research has attempted to decouple the models' weaknesses from specific tasks and focus on the embeddings per se and their mode of learning. In this paper, we take up this re…
▽ More
With the success of contextualized language models, much research explores what these models really learn and in which cases they still fail. Most of this work focuses on specific NLP tasks and on the learning outcome. Little research has attempted to decouple the models' weaknesses from specific tasks and focus on the embeddings per se and their mode of learning. In this paper, we take up this research opportunity: based on theoretical linguistic insights, we explore whether the semantic constraints of function words are learned and how the surrounding context impacts their embeddings. We create suitable datasets, provide new insights into the inner workings of LMs vis-a-vis function words and implement an assisting visual web interface for qualitative analysis.
△ Less
Submitted 16 September, 2022;
originally announced September 2022.
-
RCP: A Temporal Clustering Algorithm for Real-time Controller Placement in Mobile SDN Systems
Authors:
Reza Soleymanifar,
Carolyn Beck
Abstract:
Software Defined Networking (SDN) is a recent paradigm in telecommunication networks that disentangles data and control planes and brings more flexibility and efficiency to the network as a result. The Controller Placement (CP) problem in SDN, which is typically subject to specific optimality criteria, is one of the primary problems in the design of SDN systems. {\em Dynamic} Controller Placement…
▽ More
Software Defined Networking (SDN) is a recent paradigm in telecommunication networks that disentangles data and control planes and brings more flexibility and efficiency to the network as a result. The Controller Placement (CP) problem in SDN, which is typically subject to specific optimality criteria, is one of the primary problems in the design of SDN systems. {\em Dynamic} Controller Placement (DCP) enables a placement solution that is adaptable to inherent variability in network components (traffic, locations, etc.). DCP has gained much attention in recent years, yet despite this, most solutions proposed in the literature cannot be implemented in real-time, which is a critical concern especially in UAV/drone based SDN networks where mobility is high and split second updates are necessary. As current conventional methods fail to be relevant to such scenarios, in this work we propose a real-time control placement (RCP) algorithm. Namely, we propose a temporal clustering algorithm that provides real-time solutions for DCP, based on a control theoretic framework for which we show the solution exponentially converges to a near-optimal placement of controller devices. RCP has linear O(n) iteration computational complexity with respect to the underlying network size, n, i.e., the number of nodes, and also leverages the maximum entropy principle from information theory. This approach results in high quality solutions that are practically immune from getting stuck in poor local optima, a drawback that most works in the literature are susceptible to. We compare our work with a frame-by-frame approach and demonstrate its superiority, both in terms of speed and incurred cost, via simulations. According to our simulations RCP can be up to 25 times faster than the conventional frame-by-frame method.
△ Less
Submitted 6 December, 2021;
originally announced December 2021.
-
An overview on deep learning-based approximation methods for partial differential equations
Authors:
Christian Beck,
Martin Hutzenthaler,
Arnulf Jentzen,
Benno Kuckuck
Abstract:
It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learnin…
▽ More
It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.
△ Less
Submitted 18 November, 2022; v1 submitted 22 December, 2020;
originally announced December 2020.
-
Deep learning based numerical approximation algorithms for stochastic partial differential equations and high-dimensional nonlinear filtering problems
Authors:
Christian Beck,
Sebastian Becker,
Patrick Cheridito,
Arnulf Jentzen,
Ariel Neufeld
Abstract:
In this article we introduce and study a deep learning based approximation algorithm for solutions of stochastic partial differential equations (SPDEs). In the proposed approximation algorithm we employ a deep neural network for every realization of the driving noise process of the SPDE to approximate the solution process of the SPDE under consideration. We test the performance of the proposed app…
▽ More
In this article we introduce and study a deep learning based approximation algorithm for solutions of stochastic partial differential equations (SPDEs). In the proposed approximation algorithm we employ a deep neural network for every realization of the driving noise process of the SPDE to approximate the solution process of the SPDE under consideration. We test the performance of the proposed approximation algorithm in the case of stochastic heat equations with additive noise, stochastic heat equations with multiplicative noise, stochastic Black--Scholes equations with multiplicative noise, and Zakai equations from nonlinear filtering. In each of these SPDEs the proposed approximation algorithm produces accurate results with short run times in up to 50 space dimensions.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
On the Consistency of Maximum Likelihood Estimators for Causal Network Identification
Authors:
Xiaotian Xie,
Dimitrios Katselis,
Carolyn L. Beck,
R. Srikant
Abstract:
We consider the problem of identifying parameters of a particular class of Markov chains, called Bernoulli Autoregressive (BAR) processes. The structure of any BAR model is encoded by a directed graph. Incoming edges to a node in the graph indicate that the state of the node at a particular time instant is influenced by the states of the corresponding parental nodes in the previous time instant. T…
▽ More
We consider the problem of identifying parameters of a particular class of Markov chains, called Bernoulli Autoregressive (BAR) processes. The structure of any BAR model is encoded by a directed graph. Incoming edges to a node in the graph indicate that the state of the node at a particular time instant is influenced by the states of the corresponding parental nodes in the previous time instant. The associated edge weights determine the corresponding level of influence from each parental node. In the simplest setup, the Bernoulli parameter of a particular node's state variable is a convex combination of the parental node states in the previous time instant and an additional Bernoulli noise random variable. This paper focuses on the problem of edge weight identification using Maximum Likelihood (ML) estimation and proves that the ML estimator is strongly consistent for two variants of the BAR model. We additionally derive closed-form estimators for the aforementioned two variants and prove their strong consistency.
△ Less
Submitted 17 October, 2020;
originally announced October 2020.
-
Nonlinear Monte Carlo methods with polynomial runtime for high-dimensional iterated nested expectations
Authors:
Christian Beck,
Arnulf Jentzen,
Thomas Kruse
Abstract:
The approximative calculation of iterated nested expectations is a recurring challenging problem in applications. Nested expectations appear, for example, in the numerical approximation of solutions of backward stochastic differential equations (BSDEs), in the numerical approximation of solutions of semilinear parabolic partial differential equations (PDEs), in statistical physics, in optimal stop…
▽ More
The approximative calculation of iterated nested expectations is a recurring challenging problem in applications. Nested expectations appear, for example, in the numerical approximation of solutions of backward stochastic differential equations (BSDEs), in the numerical approximation of solutions of semilinear parabolic partial differential equations (PDEs), in statistical physics, in optimal stop** problems such as the approximative pricing of American or Bermudan options, in risk measure estimation in mathematical finance, or in decision-making under uncertainty. Nested expectations which arise in the above named applications often consist of a large number of nestings. However, the computational effort of standard nested Monte Carlo approximations for iterated nested expectations grows exponentially in the number of nestings and it remained an open question whether it is possible to approximately calculate multiply iterated high-dimensional nested expectations in polynomial time. In this article we tackle this problem by proposing and studying a new class of full-history recursive multilevel Picard (MLP) approximation schemes for iterated nested expectations. In particular, we prove under suitable assumptions that these MLP approximation schemes can approximately calculate multiply iterated nested expectations with a computational effort growing at most polynomially in the number of nestings $ K \in \mathbb{N} = \{1, 2, 3, \ldots \} $, in the problem dimension $ d \in \mathbb{N} $, and in the reciprocal $\frac{1}{\varepsilon}$ of the desired approximation accuracy $ \varepsilon \in (0, \infty) $.
△ Less
Submitted 29 September, 2020;
originally announced September 2020.
-
Proactive Aging Mitigation in CGRAs through Utilization-Aware Allocation
Authors:
Marcelo Brandalero,
Bernardo Neuhaus Lignati,
Antonio Carlos Schneider Beck,
Muhammad Shafique,
Michael Hübner
Abstract:
Resource balancing has been effectively used to mitigate the long-term aging effects of Negative Bias Temperature Instability (NBTI) in multi-core and Graphics Processing Unit (GPU) architectures. In this work, we investigate this strategy in Coarse-Grained Reconfigurable Arrays (CGRAs) with a novel application-to-CGRA allocation approach. By introducing important extensions to the reconfiguration…
▽ More
Resource balancing has been effectively used to mitigate the long-term aging effects of Negative Bias Temperature Instability (NBTI) in multi-core and Graphics Processing Unit (GPU) architectures. In this work, we investigate this strategy in Coarse-Grained Reconfigurable Arrays (CGRAs) with a novel application-to-CGRA allocation approach. By introducing important extensions to the reconfiguration logic and the datapath, we enable the dynamic movement of configurations throughout the fabric and allow overutilized Functional Units (FUs) to recover from stress-induced NBTI aging. Implementing the approach in a resource-constrained state-of-the-art CGRA reveals $2.2\times$ lifetime improvement with negligible performance overheads and less than $10\%$ increase in area.
△ Less
Submitted 5 November, 2020; v1 submitted 22 April, 2020;
originally announced April 2020.
-
A Clustering Approach to Edge Controller Placement in Software Defined Networks with Cost Balancing
Authors:
Reza Soleymanifar,
Amber Srivastava,
Carolyn Beck,
Srinivasa Salapaka
Abstract:
In this work we introduce two novel deterministic annealing based clustering algorithms to address the problem of Edge Controller Placement (ECP) in wireless edge networks. These networks lie at the core of the fifth generation (5G) wireless systems and beyond. These algorithms, ECP-LL and ECP-LB, address the dominant leader-less and leader-based controller placement topologies and have linear com…
▽ More
In this work we introduce two novel deterministic annealing based clustering algorithms to address the problem of Edge Controller Placement (ECP) in wireless edge networks. These networks lie at the core of the fifth generation (5G) wireless systems and beyond. These algorithms, ECP-LL and ECP-LB, address the dominant leader-less and leader-based controller placement topologies and have linear computational complexity in terms of network size, maximum number of clusters and dimensionality of data. Each algorithm tries to place controllers close to edge node clusters and not far away from other controllers to maintain a reasonable balance between synchronization and delay costs. While the ECP problem can be conveniently expressed as a multi-objective mixed integer non-linear program (MINLP), our algorithms outperform state of art MINLP solver, BARON both in terms of accuracy and speed. Our proposed algorithms have the competitive edge of avoiding poor local minima through a Shannon entropy term in the clustering objective function. Most ECP algorithms are highly susceptible to poor local minima and greatly depend on initialization.
△ Less
Submitted 5 December, 2019;
originally announced December 2019.
-
Full error analysis for the training of deep neural networks
Authors:
Christan Beck,
Arnulf Jentzen,
Benno Kuckuck
Abstract:
Deep learning algorithms have been applied very successfully in recent years to a range of problems out of reach for classical solution paradigms. Nevertheless, there is no completely rigorous mathematical error and convergence analysis which explains the success of deep learning algorithms. The error of a deep learning algorithm can in many situations be decomposed into three parts, the approxima…
▽ More
Deep learning algorithms have been applied very successfully in recent years to a range of problems out of reach for classical solution paradigms. Nevertheless, there is no completely rigorous mathematical error and convergence analysis which explains the success of deep learning algorithms. The error of a deep learning algorithm can in many situations be decomposed into three parts, the approximation error, the generalization error, and the optimization error. In this work we estimate for a certain deep learning algorithm each of these three errors and combine these three error estimates to obtain an overall error analysis for the deep learning algorithm under consideration. In particular, we thereby establish convergence with a suitable convergence speed for the overall error of the deep learning algorithm under consideration. Our convergence speed analysis is far from optimal and the convergence speed that we establish is rather slow, increases exponentially in the dimensions, and, in particular, suffers from the curse of dimensionality. The main contribution of this work is, instead, to provide a full error analysis (i) which covers each of the three different sources of errors usually emerging in deep learning algorithms and (ii) which merges these three sources of errors into one overall error estimate for the considered deep learning algorithm.
△ Less
Submitted 30 January, 2020; v1 submitted 30 September, 2019;
originally announced October 2019.
-
Deep splitting method for parabolic PDEs
Authors:
Christian Beck,
Sebastian Becker,
Patrick Cheridito,
Arnulf Jentzen,
Ariel Neufeld
Abstract:
In this paper we introduce a numerical method for nonlinear parabolic PDEs that combines operator splitting with deep learning. It divides the PDE approximation problem into a sequence of separate learning problems. Since the computational graph for each of the subproblems is comparatively small, the approach can handle extremely high-dimensional PDEs. We test the method on different examples from…
▽ More
In this paper we introduce a numerical method for nonlinear parabolic PDEs that combines operator splitting with deep learning. It divides the PDE approximation problem into a sequence of separate learning problems. Since the computational graph for each of the subproblems is comparatively small, the approach can handle extremely high-dimensional PDEs. We test the method on different examples from physics, stochastic control and mathematical finance. In all cases, it yields very good results in up to 10,000 dimensions with short run times.
△ Less
Submitted 21 June, 2021; v1 submitted 8 July, 2019;
originally announced July 2019.
-
Solving the Kolmogorov PDE by means of deep learning
Authors:
Christian Beck,
Sebastian Becker,
Philipp Grohs,
Nor Jaafari,
Arnulf Jentzen
Abstract:
Stochastic differential equations (SDEs) and the Kolmogorov partial differential equations (PDEs) associated to them have been widely used in models from engineering, finance, and the natural sciences. In particular, SDEs and Kolmogorov PDEs, respectively, are highly employed in models for the approximative pricing of financial derivatives. Kolmogorov PDEs and SDEs, respectively, can typically not…
▽ More
Stochastic differential equations (SDEs) and the Kolmogorov partial differential equations (PDEs) associated to them have been widely used in models from engineering, finance, and the natural sciences. In particular, SDEs and Kolmogorov PDEs, respectively, are highly employed in models for the approximative pricing of financial derivatives. Kolmogorov PDEs and SDEs, respectively, can typically not be solved explicitly and it has been and still is an active topic of research to design and analyze numerical methods which are able to approximately solve Kolmogorov PDEs and SDEs, respectively. Nearly all approximation methods for Kolmogorov PDEs in the literature suffer under the curse of dimensionality or only provide approximations of the solution of the PDE at a single fixed space-time point. In this paper we derive and propose a numerical approximation method which aims to overcome both of the above mentioned drawbacks and intends to deliver a numerical approximation of the Kolmogorov PDE on an entire region $[a,b]^d$ without suffering from the curse of dimensionality. Numerical results on examples including the heat equation, the Black-Scholes model, the stochastic Lorenz equation, and the Heston model suggest that the proposed approximation algorithm is quite effective in high dimensions in terms of both accuracy and speed.
△ Less
Submitted 14 July, 2021; v1 submitted 1 June, 2018;
originally announced June 2018.
-
Comparing and Integrating Constraint Programming and Temporal Planning for Quantum Circuit Compilation
Authors:
Kyle E. C. Booth,
Minh Do,
J. Christopher Beck,
Eleanor Rieffel,
Davide Venturelli,
Jeremy Frank
Abstract:
Recently, the makespan-minimization problem of compiling a general class of quantum algorithms into near-term quantum processors has been introduced to the AI community. The research demonstrated that temporal planning is a strong approach for a class of quantum circuit compilation (QCC) problems. In this paper, we explore the use of constraint programming (CP) as an alternative and complementary…
▽ More
Recently, the makespan-minimization problem of compiling a general class of quantum algorithms into near-term quantum processors has been introduced to the AI community. The research demonstrated that temporal planning is a strong approach for a class of quantum circuit compilation (QCC) problems. In this paper, we explore the use of constraint programming (CP) as an alternative and complementary approach to temporal planning. We extend previous work by introducing two new problem variations that incorporate important characteristics identified by the quantum computing community. We apply temporal planning and CP to the baseline and extended QCC problems as both stand-alone and hybrid approaches. Our hybrid methods use solutions found by temporal planning to warm start CP, leveraging the ability of the former to find satisficing solutions to problems with a high degree of task optionality, an area that CP typically struggles with. The CP model, benefiting from inferred bounds on planning horizon length and task counts provided by the warm start, is then used to find higher quality solutions. Our empirical evaluation indicates that while stand-alone CP is only competitive for the smallest problems, CP in our hybridization with temporal planning out-performs stand-alone temporal planning in the majority of problem classes.
△ Less
Submitted 18 March, 2018;
originally announced March 2018.
-
Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
Authors:
Christian Beck,
Weinan E,
Arnulf Jentzen
Abstract:
High-dimensional partial differential equations (PDE) appear in a number of models from the financial industry, such as in derivative pricing models, credit valuation adjustment (CVA) models, or portfolio optimization models. The PDEs in such applications are high-dimensional as the dimension corresponds to the number of financial assets in a portfolio. Moreover, such PDEs are often fully nonlinea…
▽ More
High-dimensional partial differential equations (PDE) appear in a number of models from the financial industry, such as in derivative pricing models, credit valuation adjustment (CVA) models, or portfolio optimization models. The PDEs in such applications are high-dimensional as the dimension corresponds to the number of financial assets in a portfolio. Moreover, such PDEs are often fully nonlinear due to the need to incorporate certain nonlinear phenomena in the model such as default risks, transaction costs, volatility uncertainty (Knightian uncertainty), or trading constraints in the model. Such high-dimensional fully nonlinear PDEs are exceedingly difficult to solve as the computational effort for standard approximation methods grows exponentially with the dimension. In this work we propose a new method for solving high-dimensional fully nonlinear second-order PDEs. Our method can in particular be used to sample from high-dimensional nonlinear expectations. The method is based on (i) a connection between fully nonlinear second-order PDEs and second-order backward stochastic differential equations (2BSDEs), (ii) a merged formulation of the PDE and the 2BSDE problem, (iii) a temporal forward discretization of the 2BSDE and a spatial approximation via deep neural nets, and (iv) a stochastic gradient descent-type optimization procedure. Numerical results obtained using ${\rm T{\small ENSOR}F{\small LOW}}$ in ${\rm P{\small YTHON}}$ illustrate the efficiency and the accuracy of the method in the cases of a $100$-dimensional Black-Scholes-Barenblatt equation, a $100$-dimensional Hamilton-Jacobi-Bellman equation, and a nonlinear expectation of a $ 100 $-dimensional $ G $-Brownian motion.
△ Less
Submitted 18 September, 2017;
originally announced September 2017.
-
Data Clustering and Graph Partitioning via Simulated Mixing
Authors:
Shahzad Bhatti,
Carolyn Beck,
Angelia Nedic
Abstract:
Spectral clustering approaches have led to well-accepted algorithms for finding accurate clusters in a given dataset. However, their application to large-scale datasets has been hindered by computational complexity of eigenvalue decompositions. Several algorithms have been proposed in the recent past to accelerate spectral clustering, however they compromise on the accuracy of the spectral cluster…
▽ More
Spectral clustering approaches have led to well-accepted algorithms for finding accurate clusters in a given dataset. However, their application to large-scale datasets has been hindered by computational complexity of eigenvalue decompositions. Several algorithms have been proposed in the recent past to accelerate spectral clustering, however they compromise on the accuracy of the spectral clustering to achieve faster speed. In this paper, we propose a novel spectral clustering algorithm based on a mixing process on a graph. Unlike the existing spectral clustering algorithms, our algorithm does not require computing eigenvectors. Specifically, it finds the equivalent of a linear combination of eigenvectors of the normalized similarity matrix weighted with corresponding eigenvalues. This linear combination is then used to partition the dataset into meaningful clusters. Simulations on real datasets show that partitioning datasets based on such linear combinations of eigenvectors achieves better accuracy than standard spectral clustering methods as the number of clusters increase. Our algorithm can easily be implemented in a distributed setting.
△ Less
Submitted 15 March, 2016;
originally announced March 2016.
-
Estimator Selection: End-Performance Metric Aspects
Authors:
Dimitrios Katselis,
Cristian R. Rojas,
Carolyn L. Beck
Abstract:
Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs based on usual pointwise functional distances of the estimated system from the true…
▽ More
Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs based on usual pointwise functional distances of the estimated system from the true one. The separation of the system estimator from the experiment design is done within this new framework by choosing and fixing the estimation method to either a maximum likelihood (ML) approach or a Bayesian estimator such as the minimum mean square error (MMSE). Since the MMSE estimator delivers a system estimate with lower mean square error (MSE) than the ML estimator for finite-length experiments, it is usually considered the best choice in practice in signal processing and control applications. Within the application-oriented framework a related meaningful question is: Are there end-performance metrics for which the ML estimator outperforms the MMSE when the experiment is finite-length? In this paper, we affirmatively answer this question based on a simple linear Gaussian regression example.
△ Less
Submitted 26 July, 2015;
originally announced July 2015.
-
Scheduling a Dynamic Aircraft Repair Shop with Limited Repair Resources
Authors:
Maliheh Aramon Bajestani,
J. Christopher Beck
Abstract:
We address a dynamic repair shop scheduling problem in the context of military aircraft fleet management where the goal is to maintain a full complement of aircraft over the long-term. A number of flights, each with a requirement for a specific number and type of aircraft, are already scheduled over a long horizon. We need to assign aircraft to flights and schedule repair activities while consider…
▽ More
We address a dynamic repair shop scheduling problem in the context of military aircraft fleet management where the goal is to maintain a full complement of aircraft over the long-term. A number of flights, each with a requirement for a specific number and type of aircraft, are already scheduled over a long horizon. We need to assign aircraft to flights and schedule repair activities while considering the flights requirements, repair capacity, and aircraft failures. The number of aircraft awaiting repair dynamically changes over time due to failures and it is therefore necessary to rebuild the repair schedule online. To solve the problem, we view the dynamic repair shop as successive static repair scheduling sub-problems over shorter time periods. We propose a complete approach based on the logic-based Benders decomposition to solve the static sub-problems, and design different rescheduling policies to schedule the dynamic repair shop. Computational experiments demonstrate that the Benders model is able to find and prove optimal solutions on average four times faster than a mixed integer programming model. The rescheduling approach having both aspects of scheduling over a longer horizon and quickly adjusting the schedule increases aircraft available in the long term by 10% compared to the approaches having either one of the aspects alone.
△ Less
Submitted 3 February, 2014;
originally announced February 2014.
-
Represent MOD function by low degree polynomial with unbounded one-sided error
Authors:
Chris Beck,
Yuan Li
Abstract:
In this paper, we prove tight lower bounds on the smallest degree of a nonzero polynomial in the ideal generated by $MOD_q$ or $\neg MOD_q$ in the polynomial ring $F_p[x_1, \ldots, x_n]/(x_1^2 = x_1, \ldots, x_n^2 = x_n)$, $p,q$ are coprime, which is called \emph{immunity} over $F_p$. The immunity of $MOD_q$ is lower bounded by $\lfloor (n+1)/2 \rfloor$, which is achievable when $n$ is a multiple…
▽ More
In this paper, we prove tight lower bounds on the smallest degree of a nonzero polynomial in the ideal generated by $MOD_q$ or $\neg MOD_q$ in the polynomial ring $F_p[x_1, \ldots, x_n]/(x_1^2 = x_1, \ldots, x_n^2 = x_n)$, $p,q$ are coprime, which is called \emph{immunity} over $F_p$. The immunity of $MOD_q$ is lower bounded by $\lfloor (n+1)/2 \rfloor$, which is achievable when $n$ is a multiple of $2q$; the immunity of $\neg MOD_q$ is exactly $\lfloor (n+q-1)/q \rfloor$ for every $q$ and $n$. Our result improves the previous bound $\lfloor \frac{n}{2(q-1)} \rfloor$ by Green.
We observe how immunity over $F_p$ is related to $\acc$ circuit lower bound. For example, if the immunity of $f$ over $F_p$ is lower bounded by $n/2 - o(\sqrt{n})$, and $|1_f| = Ω(2^n)$, then $f$ requires $\acc$ circuit of exponential size to compute.
△ Less
Submitted 2 April, 2013;
originally announced April 2013.
-
A Constraint Programming Approach for Solving a Queueing Control Problem
Authors:
Daria Terekhov,
J. Christopher Beck
Abstract:
In a facility with front room and back room operations, it is useful to switch workers between the rooms in order to cope with changing customer demand. Assuming stochastic customer arrival and service times, we seek a policy for switching workers such that the expected customer waiting time is minimized while the expected back room staffing is sufficient to perform all work. Three novel constrain…
▽ More
In a facility with front room and back room operations, it is useful to switch workers between the rooms in order to cope with changing customer demand. Assuming stochastic customer arrival and service times, we seek a policy for switching workers such that the expected customer waiting time is minimized while the expected back room staffing is sufficient to perform all work. Three novel constraint programming models and several shaving procedures for these models are presented. Experimental results show that a model based on closed-form expressions together with a combination of shaving procedures is the most efficient. This model is able to find and prove optimal solutions for many problem instances within a reasonable run-time. Previously, the only available approach was a heuristic algorithm. Furthermore, a hybrid method combining the heuristic and the best constraint programming method is shown to perform as well as the heuristic in terms of solution quality over time, while achieving the same performance in terms of proving optimality as the pure constraint programming model. This is the first work of which we are aware that solves such queueing-based problems with constraint programming.
△ Less
Submitted 31 October, 2011;
originally announced November 2011.
-
Solution-Guided Multi-Point Constructive Search for Job Shop Scheduling
Authors:
J. C. Beck
Abstract:
Solution-Guided Multi-Point Constructive Search (SGMPCS) is a novel constructive search technique that performs a series of resource-limited tree searches where each search begins either from an empty solution (as in randomized restart) or from a solution that has been encountered during the search. A small number of these "elite solutions is maintained during the search. We introduce the techniqu…
▽ More
Solution-Guided Multi-Point Constructive Search (SGMPCS) is a novel constructive search technique that performs a series of resource-limited tree searches where each search begins either from an empty solution (as in randomized restart) or from a solution that has been encountered during the search. A small number of these "elite solutions is maintained during the search. We introduce the technique and perform three sets of experiments on the job shop scheduling problem. First, a systematic, fully crossed study of SGMPCS is carried out to evaluate the performance impact of various parameter settings. Second, we inquire into the diversity of the elite solution set, showing, contrary to expectations, that a less diverse set leads to stronger performance. Finally, we compare the best parameter setting of SGMPCS from the first two experiments to chronological backtracking, limited discrepancy search, randomized restart, and a sophisticated tabu search algorithm on a set of well-known benchmark problems. Results demonstrate that SGMPCS is significantly better than the other constructive techniques tested, though lags behind the tabu search.
△ Less
Submitted 12 October, 2011;
originally announced October 2011.
-
Proactive Algorithms for Job Shop Scheduling with Probabilistic Durations
Authors:
J. C. Beck,
N. Wilson
Abstract:
Most classical scheduling formulations assume a fixed and known duration for each activity. In this paper, we weaken this assumption, requiring instead that each duration can be represented by an independent random variable with a known mean and variance. The best solutions are ones which have a high probability of achieving a good makespan. We first create a theoretical framework, formally showi…
▽ More
Most classical scheduling formulations assume a fixed and known duration for each activity. In this paper, we weaken this assumption, requiring instead that each duration can be represented by an independent random variable with a known mean and variance. The best solutions are ones which have a high probability of achieving a good makespan. We first create a theoretical framework, formally showing how Monte Carlo simulation can be combined with deterministic scheduling algorithms to solve this problem. We propose an associated deterministic scheduling problem whose solution is proved, under certain conditions, to be a lower bound for the probabilistic problem. We then propose and investigate a number of techniques for solving such problems based on combinations of Monte Carlo simulation, solutions to the associated deterministic problem, and either constraint programming or tabu search. Our empirical results demonstrate that a combination of the use of the associated deterministic problem and Monte Carlo simulation results in algorithms that scale best both in terms of problem size and uncertainty. Further experiments point to the correlation between the quality of the deterministic solution and the quality of the probabilistic solution as a major factor responsible for this success.
△ Less
Submitted 12 October, 2011;
originally announced October 2011.
