Showing 1–50 of 63 results for author: Powell, W

Association of neighborhood disadvantage with cognitive function and cortical disorganization in an unimpaired cohort

Authors: Apoorva Safai, Erin Jonaitis, Rebecca E Langhough, William R Buckingham, Sterling C. Johnson, W. Ryan Powell, Amy J. H. Kind, Barbara B. Bendlin, Pallavi Tiwari

Abstract: Neighborhood disadvantage is associated with worse health and cognitive outcomes. Morphological similarity network (MSN) is a promising approach to elucidate cortical network patterns underlying complex cognitive functions. We hypothesized that MSNs could capture changes in cortical patterns related to neighborhood disadvantage and cognitive function. This cross-sectional study included cognitivel… ▽ More Neighborhood disadvantage is associated with worse health and cognitive outcomes. Morphological similarity network (MSN) is a promising approach to elucidate cortical network patterns underlying complex cognitive functions. We hypothesized that MSNs could capture changes in cortical patterns related to neighborhood disadvantage and cognitive function. This cross-sectional study included cognitively unimpaired participants from two large Alzheimers studies at University of Wisconsin-Madison. Neighborhood disadvantage status was obtained using the Area Deprivation Index (ADI). Cognitive performance was assessed on memory, processing speed and executive function. Morphological Similarity Networks (MSN) were constructed for each participant based on the similarity in distribution of cortical thickness of brain regions, followed by computation of local and global network features. Association of ADI with cognitive scores and MSN features were examined using linear regression and mediation analysis. ADI showed negative association with category fluency,implicit learning speed, story recall and modified pre-clinical Alzheimers cognitive composite scores, indicating worse cognitive function among those living in more disadvantaged neighborhoods. Local network features of frontal and temporal regions differed based on ADI status. Centrality of left lateral orbitofrontal region showed a partial mediating effect between association of neighborhood disadvantage and story recall performance. Our preliminary findings suggest differences in local cortical organization by neighborhood disadvantage, which partially mediated the relationship between ADI and cognitive performance, providing a possible network-based mechanism to, in-part, explain the risk for poor cognitive functioning associated with disadvantaged neighborhoods. △ Less

Submitted 19 June, 2024; originally announced June 2024.

The AGORA High-resolution Galaxy Simulations Comparison Project IV: Halo and Galaxy Mass Assembly in a Cosmological Zoom-in Simulation at $z\le2$

Authors: Santi Roca-Fàbrega, Ji-hoon Kim, Joel R. Primack, Minyong Jung, Anna Genina, Loic Hausammann, Hyeonyong Kim, Alessandro Lupi, Kentaro Nagamine, Johnny W. Powell, Yves Revaz, Ikkoh Shimizu, Clayton Strawn, Héctor Velázquez, Tom Abel, Daniel Ceverino, Bili Dong, Thomas R. Quinn, Eun-** Shin, Alvaro Segovia-Otero, Oscar Agertz, Kirk S. S. Barrow, Corentin Cadiou, Avishai Dekel, Cameron Hummels , et al. (3 additional authors not shown)

Abstract: In this fourth paper from the AGORA Collaboration, we study the evolution down to redshift $z=2$ and below of a set of cosmological zoom-in simulations of a Milky Way mass galaxy by eight of the leading hydrodynamic simulation codes. We also compare this CosmoRun suite of simulations with dark matter-only simulations by the same eight codes. We analyze general properties of the halo and galaxy at… ▽ More In this fourth paper from the AGORA Collaboration, we study the evolution down to redshift $z=2$ and below of a set of cosmological zoom-in simulations of a Milky Way mass galaxy by eight of the leading hydrodynamic simulation codes. We also compare this CosmoRun suite of simulations with dark matter-only simulations by the same eight codes. We analyze general properties of the halo and galaxy at $z=4$ and 3, and before the last major merger, focusing on the formation of well-defined rotationally-supported disks, the mass-metallicity relation, the specific star formation rate, the gas metallicity gradients, and the non-axisymmetric structures in the stellar disks. Codes generally converge well to the stellar-to-halo mass ratios predicted by semi-analytic models at $z\sim$2. We see that almost all the hydro codes develop rotationally-supported structures at low redshifts. Most agree within 0.5 dex with the observed MZR at high and intermediate redshifts, and reproduce the gas metallicity gradients obtained from analytical models and low-redshift observations. We confirm that the inter-code differences in the halo assembly history reported in the first paper of the collaboration also exist in CosmoRun, making the code-to-code comparison more difficult. We show that such differences are mainly due to variations in code-dependent parameters that control the time-step** strategy of the gravity solver. We find that variations in the early stellar feedback can also result in differences in the timing of the low-redshift mergers. All the simulation data down to $z=2$ and the auxiliary data will be made publicly available. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Comments: Submitted to ApJ. Visit the AGORA Collaboration website (www.agorasimulations.org <http://www.agorasimulations.org/>) for more information

The AGORA High-resolution Galaxy Simulations Comparison Project. V: Satellite Galaxy Populations In A Cosmological Zoom-in Simulation of A Milky Way-mass Halo

Authors: Minyong Jung, Santi Roca-Fàbrega, Ji-hoon Kim, Anna Genina, Loic Hausammann, Hyeonyong Kim, Alessandro Lupi, Kentaro Nagamine, Johnny W. Powell, Yves Revaz, Ikkoh Shimizu, Héctor Velázquez, Daniel Ceverino, Joel R. Primack, Thomas R. Quinn, Clayton Strawn, Tom Abel, Avishai Dekel, Bili Dong, Boon Kiat Oh, Romain Teyssier

Abstract: We analyze and compare the satellite halo populations at $z\sim2$ in the high-resolution cosmological zoom-in simulations of a $10^{12}\,{\rm M}_{\odot}$ target halo ($z=0$ mass) carried out on eight widely-used astrophysical simulation codes ({\sc Art-I}, {\sc Enzo}, {\sc Ramses}, {\sc Changa}, {\sc Gadget-3}, {\sc Gear}, {\sc Arepo-t}, and {\sc Gizmo}) for the {\it AGORA} High-resolution Galaxy… ▽ More We analyze and compare the satellite halo populations at $z\sim2$ in the high-resolution cosmological zoom-in simulations of a $10^{12}\,{\rm M}_{\odot}$ target halo ($z=0$ mass) carried out on eight widely-used astrophysical simulation codes ({\sc Art-I}, {\sc Enzo}, {\sc Ramses}, {\sc Changa}, {\sc Gadget-3}, {\sc Gear}, {\sc Arepo-t}, and {\sc Gizmo}) for the {\it AGORA} High-resolution Galaxy Simulations Comparison Project. We use slightly different redshift epochs near $z=2$ for each code (hereafter ``$z\sim2$') at which the eight simulations are in the same stage in the target halo's merger history. After identifying the matched pairs of halos between the {\it CosmoRun} simulations and the DMO simulations, we discover that each {\it CosmoRun} halo tends to be less massive than its DMO counterpart. When we consider only the halos containing stellar particles at $z\sim2$, the number of satellite {\it galaxies} is significantly fewer than that of dark matter halos in all participating {\it AGORA} simulations, and is comparable to the number of present-day satellites near the Milky Way or M31. The so-called ``missing satellite problem' is fully resolved across all participating codes simply by implementing the common baryonic physics adopted in {\it AGORA} and the stellar feedback prescription commonly used in each code, with sufficient numerical resolution ($\lesssim100$ proper pc at $z=2$). We also compare other properties such as the stellar mass$-$halo mass relation and the mass$-$metallicity relation. Our work highlights the value of comparison studies such as {\it AGORA}, where outstanding problems in galaxy formation theory are studied simultaneously on multiple numerical platforms. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: Accepted for publication to the ApJ, 19 pages, 10 figures. Visit the AGORA Collaboration website (www.agorasimulations.org) for more information. For summary video, please see https://www.youtube.com/watch?v=HShRJYvPe1k

Journal ref: ApJ 964 123 (2024)

The AGORA High-resolution Galaxy Simulations Comparison Project. VI. Similarities and Differences in the Circumgalactic Medium

Authors: Clayton Strawn, Santi Roca-Fàbrega, Joel R. Primack, Ji-hoon Kim, Anna Genina, Loic Hausammann, Hyeonyong Kim, Alessandro Lupi, Kentaro Nagamine, Johnny W. Powell, Yves Revaz, Ikkoh Shimizu, Héctor Velázquez, Tom Abel, Daniel Ceverino, Bili Dong, Minyong Jung, Thomas R. Quinn, Eun-** Shin, Kirk S. S. Barrow, Avishai Dekel, Boon Kiat Oh, Nir Mandelker, Romain Teyssier, Cameron Hummels , et al. (4 additional authors not shown)

Abstract: We analyze the circumgalactic medium (CGM) for eight commonly-used cosmological codes in the AGORA collaboration. The codes are calibrated to use identical initial conditions, cosmology, heating and cooling, and star formation thresholds, but each evolves with its own unique code architecture and stellar feedback implementation. Here, we analyze the results of these simulations in terms of the str… ▽ More We analyze the circumgalactic medium (CGM) for eight commonly-used cosmological codes in the AGORA collaboration. The codes are calibrated to use identical initial conditions, cosmology, heating and cooling, and star formation thresholds, but each evolves with its own unique code architecture and stellar feedback implementation. Here, we analyze the results of these simulations in terms of the structure, composition, and phase dynamics of the CGM. We show properties such as metal distribution, ionization levels, and kinematics are effective tracers of the effects of the different code feedback and implementation methods, and as such they can be highly divergent between simulations. This is merely a fiducial set of models, against which we will in the future compare multiple feedback recipes for each code. Nevertheless, we find that the large parameter space these simulations establish can help disentangle the different variables that affect observable quantities in the CGM, e.g., showing that abundances for ions with higher ionization energy are more strongly determined by the simulation's metallicity, while abundances for ions with lower ionization energy are more strongly determined by the gas density and temperature. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: 32 pages, 17 figures. Published in ApJ

Journal ref: ApJ 962 29 (2024)

Stochastic optimization with arbitrary recurrent data sampling

Authors: William G. Powell, Hanbaek Lyu

Abstract: For obtaining optimal first-order convergence guarantee for stochastic optimization, it is necessary to use a recurrent data sampling algorithm that samples every data point with sufficient frequency. Most commonly used data sampling algorithms (e.g., i.i.d., MCMC, random reshuffling) are indeed recurrent under mild assumptions. In this work, we show that for a particular class of stochastic optim… ▽ More For obtaining optimal first-order convergence guarantee for stochastic optimization, it is necessary to use a recurrent data sampling algorithm that samples every data point with sufficient frequency. Most commonly used data sampling algorithms (e.g., i.i.d., MCMC, random reshuffling) are indeed recurrent under mild assumptions. In this work, we show that for a particular class of stochastic optimization algorithms, we do not need any other property (e.g., independence, exponential mixing, and reshuffling) than recurrence in data sampling algorithms to guarantee the optimal rate of first-order convergence. Namely, using regularized versions of Minimization by Incremental Surrogate Optimization (MISO), we show that for non-convex and possibly non-smooth objective functions, the expected optimality gap converges at an optimal rate $O(n^{-1/2})$ under general recurrent sampling schemes. Furthermore, the implied constant depends explicitly on the `speed of recurrence', measured by the expected amount of time to visit a given data point either averaged (`target time') or supremized (`hitting time') over the current location. We demonstrate theoretically and empirically that convergence can be accelerated by selecting sampling algorithms that cover the data set most effectively. We discuss applications of our general framework to decentralized optimization and distributed non-negative matrix factorization. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Comments: 41 pages, 3 figures, 1 table

Generalising the Yagi-Uda Antenna: Designing Disordered Metamaterials to Manipulate Antenna Radiation

Authors: J. R. Capers, L. D. Stanfield, J. R. Sambles, S. J. Boyes, A. W. Powell, A. P. Hibbins, S. A. R. Horsley

Abstract: Next generation microwave communications systems face several challenges, particularly from congested communications frequencies and complex propagation environments. Taking inspiration from the Yagi-Uda antenna, we present, and experimentally test, a framework based on the coupled dipole approximation for designing structures composed of a single simple emitter with a passive disordered scatterin… ▽ More Next generation microwave communications systems face several challenges, particularly from congested communications frequencies and complex propagation environments. Taking inspiration from the Yagi-Uda antenna, we present, and experimentally test, a framework based on the coupled dipole approximation for designing structures composed of a single simple emitter with a passive disordered scattering structure of rods that is optimised to provide a desired radiation pattern. Our numerical method provides an efficient way to model, and then design and test, otherwise inaccessibly large scattering systems. △ Less

Submitted 30 June, 2023; originally announced June 2023.

Entropy Minimization for Optimization of Expensive, Unimodal Functions

Authors: Xiaohe Luo, Warren B. Powell

Abstract: Maximization of an expensive, unimodal function under random observations has been an important problem in hyperparameter tuning. It features expensive function evaluations (which means small budgets) and a high level of noise. We develop an algorithm based on entropy reduction of a probabilistic belief about the optimum. The algorithm provides an efficient way of estimating the computationally in… ▽ More Maximization of an expensive, unimodal function under random observations has been an important problem in hyperparameter tuning. It features expensive function evaluations (which means small budgets) and a high level of noise. We develop an algorithm based on entropy reduction of a probabilistic belief about the optimum. The algorithm provides an efficient way of estimating the computationally intractable surrogate objective in the general Entropy Search algorithm by leveraging a sampled belief model and designing a metric that measures the information value of any search point. △ Less

Submitted 22 February, 2023; originally announced February 2023.

Simulating gravitational motion, gas dynamics, and structure in the cosmos

Authors: J. W. Powell, L. Caudill, O. Young

Abstract: We provide introductory explanations and illustrations of the $N$-body hydrodynamics code Charm N-body GrAvity solver (ChaNGa). ChaNGa simulates the gravitational motion and gas dynamics of matter in space, with the goal of modeling galactic and/or cosmological structure and evolution. We discuss the algorithm for leapfrog integration and smoothed particle hydrodynamics and computer science concep… ▽ More We provide introductory explanations and illustrations of the $N$-body hydrodynamics code Charm N-body GrAvity solver (ChaNGa). ChaNGa simulates the gravitational motion and gas dynamics of matter in space, with the goal of modeling galactic and/or cosmological structure and evolution. We discuss the algorithm for leapfrog integration and smoothed particle hydrodynamics and computer science concepts used by the program, including the binary data structure for the particle positions. Our presentation borrows from the doctoral dissertation of J.\ G.\ Stadel. Problems are provided in order to use ChaNGa to learn or solidify some cosmological concepts. △ Less

Submitted 9 March, 2023; v1 submitted 3 February, 2023; originally announced February 2023.

Comments: Accepted by the American Journal of Physics. Two important supplementary documents are included

An Information-Collecting Drone Management Problem for Wildfire Mitigation

Authors: Lawrence Thul, Warren B Powell

Abstract: We present a formal mathematical multi-agent modeling framework for autonomously combating a wildland fire with unmanned aerial vehicles. The problem is formulated as a collaboration between a drone and a helicopter equipped with a tanker. The modeling solutions are designed to capture the communication between agents and the information processes between the agents and their environment. The dron… ▽ More We present a formal mathematical multi-agent modeling framework for autonomously combating a wildland fire with unmanned aerial vehicles. The problem is formulated as a collaboration between a drone and a helicopter equipped with a tanker. The modeling solutions are designed to capture the communication between agents and the information processes between the agents and their environment. The drone is used to make partial observations and use them to update probability distributions over the uncertain state of the world. We design a parameterized direct lookahead approximation policy to guide the drone through the region and promote active learning. The helicopter is used to extinguish the fire. We design a helicopter policy which anticipates the zones which will burn through the belief modeling and extinguish them. We simulate the modeling and policy solutions using a wildland fire simulation in an area of Northern California. △ Less

Submitted 17 January, 2023; originally announced January 2023.

Comments: 42 pages, 6 figures

The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response

Authors: Lina Al-Kanj, Warren B. Powell, Belgacem Bouzaiene-Ayari

Abstract: We address the problem of mitigating damage to a power grid following a storm by managing a vehicle that has to be routed while simultaneously performing two tasks: learning about damage from the grid (which requires direct observation) and repairing damage that it observes. The learning process is assisted by calls from customers notifying the utility that they have lost power (``lights-out calls… ▽ More We address the problem of mitigating damage to a power grid following a storm by managing a vehicle that has to be routed while simultaneously performing two tasks: learning about damage from the grid (which requires direct observation) and repairing damage that it observes. The learning process is assisted by calls from customers notifying the utility that they have lost power (``lights-out calls''). However, when a tree falls and damages a line, it triggers the first upstream circuit breaker, which results in power outages for everyone on the grid below the circuit breaker. We present a dynamic routing model that captures observable state variables such as the location of the truck and the state of the grid on segments the truck has visited, and beliefs about outages on segments that have not been visited. Trucks are routed over a physical transportation network, but the pattern of outages is governed by the structure of the power grid. We introduce a form of Monte Carlo tree search based on information relaxation that we call {\it optimistic MCTS} which improves its application to problems with larger action spaces. We show that the method significantly outperforms standard escalation heuristics used in industry.} △ Less

Submitted 16 January, 2023; originally announced January 2023.

Comments: 44 pages, 7 figures. arXiv admin note: substantial text overlap with arXiv:1605.05711

Testing for relics of past strong buckling events in edge-on galaxies: Simulation predictions and data from S$^{4}$G

Authors: V. Cuomo, V. P. Debattista, S. Racz, S. R. Anderson, P. Erwin, O. A. Gonzalez, J. W. Powell, E. M. Corsini, L. Morelli, M. A. Norris

Abstract: The short-lived buckling instability is responsible for the formation of at least some box/peanut (B/P) shaped bulges, which are observed in most massive, $z=0$, barred galaxies. Nevertheless, it has also been suggested that B/P bulges form via the slow trap** of stars onto vertically extended resonant orbits. The key difference between these two scenarios is that when the bar buckles, symmetry… ▽ More The short-lived buckling instability is responsible for the formation of at least some box/peanut (B/P) shaped bulges, which are observed in most massive, $z=0$, barred galaxies. Nevertheless, it has also been suggested that B/P bulges form via the slow trap** of stars onto vertically extended resonant orbits. The key difference between these two scenarios is that when the bar buckles, symmetry about the mid-plane is broken for a period of time. We use a suite of simulations (with and without gas) to show that when the buckling is sufficiently strong, a residual mid-plane asymmetry persists for several Gyrs after the end of the buckling phase, and is visible in simulation images. On the other hand, images of B/P bulges formed through resonant trap** and/or weak buckling remain symmetric about the mid-plane. We develop two related diagnostics to identify and quantify mid-plane asymmetry in simulation images of galaxies that are within 3° of edge-on orientation, allowing us to test whether the presence of a B/P-shaped bulge can be explained by a past buckling event. We apply our diagnostics to two nearly edge-on galaxies with B/P bulges from the ${\it Spitzer}$ Survey of Stellar Structure in Galaxies, finding no mid-plane asymmetry, implying these galaxies formed their bulges either by resonant trap** or by buckling more than $\sim 5$ Gyr ago. We conclude that the formation of B/P bulges through strong buckling may be a rare event in the past $\sim 5$ Gyr. △ Less

Submitted 21 October, 2022; originally announced October 2022.

Comments: Accepted for pubblication in MNRAS

Microwave Demonstration of Purcell Effect Enhanced Radiation Efficiency

Authors: L. D. Stanfield, A. W. Powell, S. A. R. Horsley, J. R. Sambles, A. P. Hibbins

Abstract: We experimentally demonstrate a Purcell effect-based design technique for improved impedance matching, and thus enhanced radiation efficiency from a small microwave emitter. Using an iterative process centred on comparing the phase of the radiated field of the emitter in air with that of the emitter in a dielectric environment, we optimise the structure of a dielectric hemisphere above a ground pl… ▽ More We experimentally demonstrate a Purcell effect-based design technique for improved impedance matching, and thus enhanced radiation efficiency from a small microwave emitter. Using an iterative process centred on comparing the phase of the radiated field of the emitter in air with that of the emitter in a dielectric environment, we optimise the structure of a dielectric hemisphere above a ground plane surrounding a small monopolar microwave emitter in order to maximise its radiation efficiency. The optimised system shows very strong coupling between the emitter and two omnidirectional radiation modes at 2.00 GHz and 2.84 GHz, yielding Purcell enhancement factors of 8360 and 430 times increase respectively, and near perfect radiation efficiency. △ Less

Submitted 29 July, 2022; originally announced September 2022.

Numerical reconstruction for 3D nonlinear SAR imaging via a version of the convexification method

Authors: Vo Anh Khoa, Michael Victor Klibanov, William Grayson Powell, Loc Hoang Nguyen

Abstract: This work extends the applicability of our recent convexification-based algorithm for constructing images of the dielectric constant of buried or occluded target. We are orientated towards the detection of explosive-like targets such as antipersonnel land mines and improvised explosive devices in the non-invasive inspections of buildings. In our previous work, the method is posed in the perspectiv… ▽ More This work extends the applicability of our recent convexification-based algorithm for constructing images of the dielectric constant of buried or occluded target. We are orientated towards the detection of explosive-like targets such as antipersonnel land mines and improvised explosive devices in the non-invasive inspections of buildings. In our previous work, the method is posed in the perspective that we use multiple source locations running along a line of source to get a 2D image of the dielectric function. Mathematically, we solve a 1D coefficient inverse problem for a hyperbolic equation for each source location. Different from any conventional Born approximation-based technique for synthetic-aperture radar, this method does not need any linearization. In this paper, we attempt to verify the method using several 3D numerical tests with simulated data. We revisit the global convergence of the gradient descent method of our computational approach. △ Less

Submitted 19 June, 2022; originally announced June 2022.

Comments: 23 pages, 11 figures

MSC Class: 78A46; 65L70; 65C20

Stochastic Search for a Parametric Cost Function Approximation: Energy storage with rolling forecasts

Authors: Saeed Ghadimi, Warren B. Powell

Abstract: Rolling forecasts have been almost overlooked in the renewable energy storage literature. In this paper, we provide a new approach for handling uncertainty not just in the accuracy of a forecast, but in the evolution of forecasts over time. Our approach shifts the focus from modeling the uncertainty in a lookahead model to accurate simulations in a stochastic base model. We develop a robust policy… ▽ More Rolling forecasts have been almost overlooked in the renewable energy storage literature. In this paper, we provide a new approach for handling uncertainty not just in the accuracy of a forecast, but in the evolution of forecasts over time. Our approach shifts the focus from modeling the uncertainty in a lookahead model to accurate simulations in a stochastic base model. We develop a robust policy for making energy storage decisions by creating a parametrically modified lookahead model, where the parameters are tuned in the stochastic base model. Since computing unbiased stochastic gradients with respect to the parameters require restrictive assumptions, we propose a simulation-based stochastic approximation algorithm based on numerical derivatives to optimize these parameters. While numerical derivatives, calculated based on the noisy function evaluations, provide biased gradient estimates, an online variance reduction technique built in the framework of our proposed algorithm, will enable us to control the accumulated bias errors and establish the finite-time rate of convergence of the algorithm. Our numerical experiments show the performance of this algorithm in finding policies outperforming the deterministic benchmark policy. △ Less

Submitted 14 April, 2022; originally announced April 2022.

On-Demand Activation of Photochromic Nanoheaters for High Color Purity 3D Printing

Authors: Alexander W. Powell, Alexandros Stavrinadis, Sotirios Christodoulou, Romain Quidant, Gerasimos Konstantatos

Abstract: The creation of white and multicoloured 3D-printed objects with high colour fidelity via powder sintering processes is currently limited by discolouration from thermal sensitizers used in the printing process. Here we circumvent this problem by using switchable, photochromic tungsten oxide nanoparticles, which are colourless even at high concentrations. Upon ultraviolet illumination, the tungsten… ▽ More The creation of white and multicoloured 3D-printed objects with high colour fidelity via powder sintering processes is currently limited by discolouration from thermal sensitizers used in the printing process. Here we circumvent this problem by using switchable, photochromic tungsten oxide nanoparticles, which are colourless even at high concentrations. Upon ultraviolet illumination, the tungsten oxide nanoparticles can be reversibly activated making them highly absorbing in the infrared. Their strong infrared absorption upon activation renders them efficient photothermal sensitizers that can act as fusing agents for polymer powders in sintering-based 3D printing. The WO3 nanoparticles show fast activation times, and when mixed with polyamide powders they exhibit a heating-to-colour-change ratio greatly exceeding other sensitizers in the literature. Upon mixing with coloured inks, powders containing WO3 display identical colouration to a pristine powder. This demonstrates the potential of WO3, and photochromic nanoparticles in general as a new class of material for advanced manufacturing. △ Less

Submitted 25 March, 2022; originally announced March 2022.

The Parametric Cost Function Approximation: A new approach for multistage stochastic programming

Authors: Warren B Powell, Saeed Ghadimi

Abstract: The most common approaches for solving multistage stochastic programming problems in the research literature have been to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the impact of a decision now on the future. By contrast, common industry practice is to use a deterministic approximation of the future which is easier to understand a… ▽ More The most common approaches for solving multistage stochastic programming problems in the research literature have been to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the impact of a decision now on the future. By contrast, common industry practice is to use a deterministic approximation of the future which is easier to understand and solve, but which is criticized for ignoring uncertainty. We show that a parameterized version of a deterministic optimization model can be an effective way of handling uncertainty without the complexity of either stochastic programming or dynamic programming. We present the idea of a parameterized deterministic optimization model, and in particular a deterministic lookahead model, as a powerful strategy for many complex stochastic decision problems. This approach can handle complex, high-dimensional state variables, and avoids the usual approximations associated with scenario trees or value function approximations. Instead, it introduces the offline challenge of designing and tuning the parameterization. We illustrate the idea by using a series of application settings, and demonstrate its use in a nonstationary energy storage problem with rolling forecasts. △ Less

Submitted 1 January, 2022; originally announced January 2022.

Comments: 3 figures

MSC Class: 68 ACM Class: F.2; I.2

Integrating On-chain and Off-chain Governance for Supply Chain Transparency and Integrity

Authors: Shoufeng Cao, Thomas Miller, Marcus Foth, Warwick Powell, Xavier Boyen, Charles Turner-Morris

Abstract: Integrating on-chain and off-chain data storage for decentralised and distributed information systems, such as blockchain, presents specific challenges for providing transparency of data governance and ensuring data integrity through stakeholder engagement. Current research on blockchain-based supply chains focuses on using on-chain governance rules developed for cryptocurrency blockchains to stor… ▽ More Integrating on-chain and off-chain data storage for decentralised and distributed information systems, such as blockchain, presents specific challenges for providing transparency of data governance and ensuring data integrity through stakeholder engagement. Current research on blockchain-based supply chains focuses on using on-chain governance rules developed for cryptocurrency blockchains to store some critical data points without designing tailored on-chain governance mechanisms and disclosing off-chain decision-making processes on data governance. In response to this research gap, this paper presents an integrated data governance framework that coordinates supply chain stakeholders with inter-linked on-chain and off-chain governance to disclose on-chain and off-chain rules and decision-making processes for supply chain transparency and integrity. We present a Proof-of-Concept (PoC) of our integrated data governance approach and suggest future research to strengthen scaling up and supply chain-based use cases based on our learnings. △ Less

Submitted 11 November, 2021; originally announced November 2021.

Comments: The 5th Symposium on Distributed Ledger Technology

Journal ref: Proc. 5th Symp. Distributed Ledger Technology, Brisbane, Australia, 23 Nov 2021

COMAP Early Science: II. Pathfinder Instrument

Authors: James W. Lamb, Kieran A. Cleary, David P. Woody, Morgan Catha, Dongwoo T. Chung, Joshua Ott Gundersen, Stuart E. Harper, Andrew I. Harris, Richard Hobbs, Håvard T. Ihle, Jonathon Kocz, Timothy J. Pearson, Liju Philip, Travis W. Powell, Lilian Basoalto, J. Richard Bond, Jowita Borowska, Patrick C. Breysse, Sarah E. Church, Clive Dickinson, Delaney A. Dunne, Hans Kristian Eriksen, Marie Kristine Foss, Todd Gaier, Junhan Kim , et al. (10 additional authors not shown)

Abstract: Line intensity map** (LIM) is a new technique for tracing the global properties of galaxies over cosmic time. Detection of the very faint signals from redshifted carbon monoxide (CO), a tracer of star formation, pushes the limits of what is feasible with a total-power instrument. The CO Map** Project (COMAP) Pathfinder is a first-generation instrument aiming to prove the concept and develop th… ▽ More Line intensity map** (LIM) is a new technique for tracing the global properties of galaxies over cosmic time. Detection of the very faint signals from redshifted carbon monoxide (CO), a tracer of star formation, pushes the limits of what is feasible with a total-power instrument. The CO Map** Project (COMAP) Pathfinder is a first-generation instrument aiming to prove the concept and develop the technology for future experiments, as well as delivering early science products. With 19 receiver channels in a hexagonal focal plane arrangement on a 10.4 m antenna, and an instantaneous 26-34 GHz frequency range with 2 MHz resolution, it is ideally suited to measuring CO($J$=1-0) from $z\sim3$. In this paper we discuss strategies for designing and building the Pathfinder and the challenges that were encountered. The design of the instrument prioritized LIM requirements over those of ancillary science. After a couple of years of operation, the instrument is well understood, and the first year of data is already yielding useful science results. Experience with this Pathfinder will drive the design of the next generations of experiments. △ Less

Submitted 29 November, 2021; v1 submitted 10 November, 2021; originally announced November 2021.

Comments: Paper 2 of 7 in series. 27 pages, 28 figures, submitted to ApJ

COMAP Early Science: I. Overview

Authors: Kieran A. Cleary, Jowita Borowska, Patrick C. Breysse, Morgan Catha, Dongwoo T. Chung, Sarah E. Church, Clive Dickinson, Hans Kristian Eriksen, Marie Kristine Foss, Joshua Ott Gundersen, Stuart E. Harper, Andrew I. Harris, Richard Hobbs, Håvard, T. Ihle, Junhan Kim, Jonathon Kocz, James W. Lamb, Jonas G. S. Lunde, Hamsa Padmanabhan, Timothy J. Pearson, Liju Philip, Travis W. Powell, Maren Rasmussen, Anthony C. S. Readhead , et al. (18 additional authors not shown)

Abstract: The CO Map** Array Project (COMAP) aims to use line intensity map** of carbon monoxide (CO) to trace the distribution and global properties of galaxies over cosmic time, back to the Epoch of Reionization (EoR). To validate the technologies and techniques needed for this goal, a Pathfinder instrument has been constructed and fielded. Sensitive to CO(1-0) emission from $z=2.4$-$3.4$ and a fainte… ▽ More The CO Map** Array Project (COMAP) aims to use line intensity map** of carbon monoxide (CO) to trace the distribution and global properties of galaxies over cosmic time, back to the Epoch of Reionization (EoR). To validate the technologies and techniques needed for this goal, a Pathfinder instrument has been constructed and fielded. Sensitive to CO(1-0) emission from $z=2.4$-$3.4$ and a fainter contribution from CO(2-1) at $z=6$-8, the Pathfinder is surveying $12$ deg$^2$ in a 5-year observing campaign to detect the CO signal from $z\sim3$. Using data from the first 13 months of observing, we estimate $P_\mathrm{CO}(k) = -2.7 \pm 1.7 \times 10^4μ\mathrm{K}^2 \mathrm{Mpc}^3$ on scales $k=0.051-0.62 \mathrm{Mpc}^{-1}$ - the first direct 3D constraint on the clustering component of the CO(1-0) power spectrum. Based on these observations alone, we obtain a constraint on the amplitude of the clustering component (the squared mean CO line temperature-bias product) of $\langle Tb\rangle^2<49$ $μ$K$^2$ - nearly an order-of-magnitude improvement on the previous best measurement. These constraints allow us to rule out two models from the literature. We forecast a detection of the power spectrum after 5 years with signal-to-noise ratio (S/N) 9-17. Cross-correlation with an overlap** galaxy survey will yield a detection of the CO-galaxy power spectrum with S/N of 19. We are also conducting a 30 GHz survey of the Galactic plane and present a preliminary map. Looking to the future of COMAP, we examine the prospects for future phases of the experiment to detect and characterize the CO signal from the EoR. △ Less

Submitted 29 November, 2021; v1 submitted 10 November, 2021; originally announced November 2021.

Comments: Paper 1 of 7 in series. 18 pages, 16 figures, submitted to ApJ

The AGORA High-resolution Galaxy Simulations Comparison Project. III: Cosmological zoom-in simulation of a Milky Way-mass halo

Authors: Santi Roca-Fàbrega, Ji-hoon Kim, Loic Hausammann, Kentaro Nagamine, Johnny W. Powell, Ikkoh Shimizu, Daniel Ceverino, Alessandro Lupi, Joel R. Primack, Thomas Quinn, Yves Revaz, Héctor Velázquez, Tom Abel, Michael Buehlmann, Avishai Dekel, Bili Dong, Oliver Hahn, Cameron B. Hummels, Ki-won Kim, Britton D. Smith, Clayton J. Strawn, Romain Teyssier, Matthew Turk

Abstract: We present a suite of high-resolution cosmological zoom-in simulations to $z=4$ of a $10^{12}\,{\rm M}_{\odot}$ halo at $z=0$, obtained using seven contemporary astrophysical simulation codes widely used in the numerical galaxy formation community. Physics prescriptions for gas cooling, heating, and star formation, are similar to the ones used in our previous {\it AGORA} disk comparison but now ac… ▽ More We present a suite of high-resolution cosmological zoom-in simulations to $z=4$ of a $10^{12}\,{\rm M}_{\odot}$ halo at $z=0$, obtained using seven contemporary astrophysical simulation codes widely used in the numerical galaxy formation community. Physics prescriptions for gas cooling, heating, and star formation, are similar to the ones used in our previous {\it AGORA} disk comparison but now account for the effects of cosmological processes. In this work, we introduce the most careful comparison yet of galaxy formation simulations run by different code groups, together with a series of four calibration steps each of which is designed to reduce the number of tunable simulation parameters adopted in the final run. After all the participating code groups successfully completed the calibration steps, we reach a suite of cosmological simulations with similar mass assembly histories down to $z=4$. With numerical accuracy that resolves the internal structure of a target halo, we find that the codes overall agree well with one another in e.g., gas and stellar properties, but also show differences in e.g., circumgalactic medium properties. We argue that, if adequately tested in accordance with our proposed calibration steps and common parameters, the results of high-resolution cosmological zoom-in simulations can be robust and reproducible. New code groups are invited to join this comparison by generating equivalent models by adopting the common initial conditions, the common easy-to-implement physics package, and the proposed calibration steps. Further analyses of the simulations presented here will be in forthcoming reports from our Collaboration. △ Less

Submitted 17 June, 2021; originally announced June 2021.

Comments: Accepted for publication to the ApJ. The simulation snapshots used in this paper will be made publicly available in a user-friendly interface. The announcement of the data release will be posted on the AGORA Project webpage (www.agorasimulations.org). Early access to the data can be provided upon request to the authors, or to the project coordinator ([email protected], [email protected])

Stochastic Optimization for Vaccine and Testing Kit Allocation for the COVID-19 Pandemic

Authors: Lawrence Thul, Warren Powell

Abstract: The pandemic caused by the SARS-CoV-2 virus has exposed many flaws in the decision-making strategies used to distribute resources to combat global health crises. In this paper, we leverage reinforcement learning and optimization to improve upon the allocation strategies for various resources. In particular, we consider a problem where a central controller must decide where to send testing kits to… ▽ More The pandemic caused by the SARS-CoV-2 virus has exposed many flaws in the decision-making strategies used to distribute resources to combat global health crises. In this paper, we leverage reinforcement learning and optimization to improve upon the allocation strategies for various resources. In particular, we consider a problem where a central controller must decide where to send testing kits to learn about the uncertain states of the world (active learning); then, use the new information to construct beliefs about the states and decide where to allocate resources. We propose a general model coupled with a tunable lookahead policy for making vaccine allocation decisions without perfect knowledge about the state of the world. The lookahead policy is compared to a population-based myopic policy which is more likely to be similar to the present strategies in practice. Each vaccine allocation policy works in conjunction with a testing kit allocation policy to perform active learning. Our simulation results demonstrate that an optimization-based lookahead decision making strategy will outperform the presented myopic policy. △ Less

Submitted 4 January, 2021; originally announced January 2021.

The quasi-reversibility method to numerically solve an inverse source problem for hyperbolic equations

Authors: Thuy T. Le, Loc H. Nguyen, Thi-Phong Nguyen, William Powell

Abstract: We propose a numerical method to solve an inverse source problem of computing the initial condition of hyperbolic equations from the measurements of Cauchy data. This problem arises in thermo- and photo- acoustic tomography in a bounded cavity, in which the reflection of the wave makes the widely-used approaches, such as the time reversal method, not applicable. In order to solve this inverse sour… ▽ More We propose a numerical method to solve an inverse source problem of computing the initial condition of hyperbolic equations from the measurements of Cauchy data. This problem arises in thermo- and photo- acoustic tomography in a bounded cavity, in which the reflection of the wave makes the widely-used approaches, such as the time reversal method, not applicable. In order to solve this inverse source problem, we approximate the solution to the hyperbolic equation by its Fourier series with respect to a special orthogonal basis of $L^2$. Then, we derive a coupled system of elliptic equations for the corresponding Fourier coefficients. We solve it by the quasi-reversibility method. The desired initial condition follows. We rigorously prove the convergence of the quasi-reversibility method as the noise level tends to 0. Some numerical examples are provided. In addition, we numerically prove that the use of the special basic above is significant. △ Less

Submitted 11 January, 2021; v1 submitted 9 November, 2020; originally announced November 2020.

The S$π$RIT Time Projection Chamber

Authors: J. Barney, J. Estee, W. G. Lynch, T. Isobe, G. Jhang, M. Kurata-Nishimura, A. B. McIntosh, T. Murakami, R. Shane, S. Tangwancharoen, M. B. Tsang, G. Cerizza, M. Kaneko, J. W. Lee, C. Y. Tsang, R. Wang, C. Anderson, H. Baba, Z. Chajecki, M. Famiano, R. Hodges-Showalter, B. Hong, T. Kobayashi, P. Lasko, J. Łukasik , et al. (15 additional authors not shown)

Abstract: The SAMURAI Pion Reconstruction and Ion-Tracker Time Projection Chamber (S$π$RIT TPC) was designed to enable measurements of heavy ion collisions with the SAMURAI spectrometer at the RIKEN Radioactive Isotope Beam Factory and provide constraints on the Equation of State of neutron-rich nuclear matter. The S$π$RIT TPC has a 50.5 cm drift length and an 86.4 cm $\times$ 134.4 cm pad plane with 12,096… ▽ More The SAMURAI Pion Reconstruction and Ion-Tracker Time Projection Chamber (S$π$RIT TPC) was designed to enable measurements of heavy ion collisions with the SAMURAI spectrometer at the RIKEN Radioactive Isotope Beam Factory and provide constraints on the Equation of State of neutron-rich nuclear matter. The S$π$RIT TPC has a 50.5 cm drift length and an 86.4 cm $\times$ 134.4 cm pad plane with 12,096 pads that are equipped with the Generic Electronics for TPCs readout electronics. The S$π$RIT TPC allows excellent reconstruction of particles and provides isotopic resolution for pions and other light charged particles across a wide range of energy losses and momenta. Details of the S$π$RIT TPC are presented, along with discussion of the TPC performance based on cosmic ray and experimental data. △ Less

Submitted 21 May, 2020; originally announced May 2020.

Comments: 12 pages, 20 figures

Optimal Learning for Sequential Decisions in Laboratory Experimentation

Authors: Kristopher Reyes, Warren B Powell

Abstract: The process of discovery in the physical, biological and medical sciences can be painstakingly slow. Most experiments fail, and the time from initiation of research until a new advance reaches commercial production can span 20 years. This tutorial is aimed to provide experimental scientists with a foundation in the science of making decisions. Using numerical examples drawn from the experiences of… ▽ More The process of discovery in the physical, biological and medical sciences can be painstakingly slow. Most experiments fail, and the time from initiation of research until a new advance reaches commercial production can span 20 years. This tutorial is aimed to provide experimental scientists with a foundation in the science of making decisions. Using numerical examples drawn from the experiences of the authors, the article describes the fundamental elements of any experimental learning problem. It emphasizes the important role of belief models, which include not only the best estimate of relationships provided by prior research, previous experiments and scientific expertise, but also the uncertainty in these relationships. We introduce the concept of a learning policy, and review the major categories of policies. We then introduce a policy, known as the knowledge gradient, that maximizes the value of information from each experiment. We bring out the importance of reducing uncertainty, and illustrate this process for different belief models. △ Less

Submitted 13 April, 2020; v1 submitted 11 April, 2020; originally announced April 2020.

On State Variables, Bandit Problems and POMDPs

Authors: Warren B Powell

Abstract: State variables are easily the most subtle dimension of sequential decision problems. This is especially true in the context of active learning problems (bandit problems") where decisions affect what we observe and learn. We describe our canonical framework that models {\it any} sequential decision problem, and present our definition of state variables that allows us to claim: Any properly modeled… ▽ More State variables are easily the most subtle dimension of sequential decision problems. This is especially true in the context of active learning problems (bandit problems") where decisions affect what we observe and learn. We describe our canonical framework that models {\it any} sequential decision problem, and present our definition of state variables that allows us to claim: Any properly modeled sequential decision problem is Markovian. We then present a novel two-agent perspective of partially observable Markov decision problems (POMDPs) that allows us to then claim: Any model of a real decision problem is (possibly) non-Markovian. We illustrate these perspectives using the context of observing and treating flu in a population, and provide examples of all four classes of policies in this setting. We close with an indication of how to extend this thinking to multiagent problems. △ Less

Submitted 14 February, 2020; originally announced February 2020.

Risk Directed Importance Sampling in Stochastic Dual Dynamic Programming with Hidden Markov Models for Grid Level Energy Storage

Authors: Joseph L. Durante, Juliana Nascimento, Warren B. Powell

Abstract: Power systems that need to integrate renewables at a large scale must account for the high levels of uncertainty introduced by these power sources. This can be accomplished with a system of many distributed grid-level storage devices. However, develo** a cost-effective and robust control policy in this setting is a challenge due to the high dimensionality of the resource state and the highly vol… ▽ More Power systems that need to integrate renewables at a large scale must account for the high levels of uncertainty introduced by these power sources. This can be accomplished with a system of many distributed grid-level storage devices. However, develo** a cost-effective and robust control policy in this setting is a challenge due to the high dimensionality of the resource state and the highly volatile stochastic processes involved. We first model the problem using a carefully calibrated power grid model and a specialized hidden Markov stochastic model for wind power which replicates crossing times. We then base our control policy on a variant of stochastic dual dynamic programming, an algorithm well suited for certain high dimensional control problems, that is modified to accommodate hidden Markov uncertainty in the stochastics. However, the algorithm may be impractical to use as it exhibits relatively slow convergence. To accelerate the algorithm, we apply both quadratic regularization and a risk-directed importance sampling technique for sampling the outcome space at each time step in the backward pass of the algorithm. We show that the resulting policies are more robust than those developed using classical SDDP modeling assumptions and algorithms. △ Less

Submitted 1 February, 2020; v1 submitted 16 January, 2020; originally announced January 2020.

Comments: 42 pages, 7 figures, Replacement: added additional references

Reinforcement Learning via Parametric Cost Function Approximation for Multistage Stochastic Programming

Authors: Saeed Ghadimi, Raymond T. Perkins, Warren B. Powell

Abstract: The most common approaches for solving stochastic resource allocation problems in the research literature is to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the impact of a decision now on the future. By contrast, common industry practice is to use a deterministic approximation of the future which is easier to understand and solve,… ▽ More The most common approaches for solving stochastic resource allocation problems in the research literature is to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the impact of a decision now on the future. By contrast, common industry practice is to use a deterministic approximation of the future which is easier to understand and solve, but which is criticized for ignoring uncertainty. We show that a parameterized version of a deterministic lookahead can be an effective way of handling uncertainty, while enjoying the computational simplicity of a deterministic lookahead. We present the parameterized lookahead model as a form of policy for solving a stochastic base model, which is used as the basis for optimizing the parameterized policy. This approach can handle complex, high-dimensional state variables, and avoids the usual approximations associated with scenario trees. We formalize this approach and demonstrate its use in the context of a complex, nonstationary energy storage problem. △ Less

Submitted 2 January, 2020; originally announced January 2020.

Comments: arXiv admin note: text overlap with arXiv:1703.04644

Zeroth-order Stochastic Compositional Algorithms for Risk-Aware Learning

Authors: Dionysios S. Kalogerias, Warren B. Powell

Abstract: We present $\textit{Free-MESSAGE}^{p}$, the first zeroth-order algorithm for (weakly-)convex mean-semideviation-based risk-aware learning, which is also the first three-level zeroth-order compositional stochastic optimization algorithm whatsoever. Using a non-trivial extension of Nesterov's classical results on Gaussian smoothing, we develop the $\textit{Free-MESSAGE}^{p}$ algorithm from first pri… ▽ More We present $\textit{Free-MESSAGE}^{p}$, the first zeroth-order algorithm for (weakly-)convex mean-semideviation-based risk-aware learning, which is also the first three-level zeroth-order compositional stochastic optimization algorithm whatsoever. Using a non-trivial extension of Nesterov's classical results on Gaussian smoothing, we develop the $\textit{Free-MESSAGE}^{p}$ algorithm from first principles, and show that it essentially solves a smoothed surrogate to the original problem, the former being a uniform approximation of the latter, in a useful, convenient sense. We then present a complete analysis of the $\textit{Free-MESSAGE}^{p}$ algorithm, which establishes convergence in a user-tunable neighborhood of the optimal solutions of the original problem for convex costs, as well as explicit convergence rates for convex, weakly convex, and strongly convex costs, and in a unified way. Orderwise, and for fixed problem parameters, our results demonstrate no sacrifice in convergence speed as compared to existing first-order methods, while striking a certain balance among the condition of the problem, its dimensionality, as well as the accuracy of the obtained results, naturally extending previous results in zeroth-order risk-neutral learning. △ Less

Submitted 13 December, 2021; v1 submitted 19 December, 2019; originally announced December 2019.

Comments: 31 pages, major revision of the first version

From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions

Authors: Warren B Powell

Abstract: There are over 15 distinct communities that work in the general area of sequential decisions and information, often referred to as decisions under uncertainty or stochastic optimization. We focus on two of the most important fields: stochastic optimal control, with its roots in deterministic optimal control, and reinforcement learning, with its roots in Markov decision processes. Building on prior… ▽ More There are over 15 distinct communities that work in the general area of sequential decisions and information, often referred to as decisions under uncertainty or stochastic optimization. We focus on two of the most important fields: stochastic optimal control, with its roots in deterministic optimal control, and reinforcement learning, with its roots in Markov decision processes. Building on prior work, we describe a unified framework that covers all 15 different communities, and note the strong parallels with the modeling framework of stochastic optimal control. By contrast, we make the case that the modeling framework of reinforcement learning, inherited from discrete Markov decision processes, is quite limited. Our framework (and that of stochastic control) is based on the core problem of optimizing over policies. We describe four classes of policies that we claim are universal, and show that each of these two fields have, in their own way, evolved to include examples of each of these four classes. △ Less

Submitted 18 December, 2019; v1 submitted 7 December, 2019; originally announced December 2019.

Comments: 47 pages, 6 figures

Approximate Dynamic Programming for Planning a Ride-Sharing System using Autonomous Fleets of Electric Vehicles

Authors: Lina Al-Kanj, Juliana Nascimento, Warren B. Powell

Abstract: Within a decade, almost every major auto company, along with fleet operators such as Uber, have announced plans to put autonomous vehicles on the road. At the same time, electric vehicles are quickly emerging as a next-generation technology that is cost effective, in addition to offering the benefits of reducing the carbon footprint. The combination of a centrally managed fleet of driverless vehic… ▽ More Within a decade, almost every major auto company, along with fleet operators such as Uber, have announced plans to put autonomous vehicles on the road. At the same time, electric vehicles are quickly emerging as a next-generation technology that is cost effective, in addition to offering the benefits of reducing the carbon footprint. The combination of a centrally managed fleet of driverless vehicles, along with the operating characteristics of electric vehicles, is creating a transformative new technology that offers significant cost savings with high service levels. This problem involves a dispatch problem for assigning riders to cars, a surge pricing problem for deciding on the price per trip and a planning problem for deciding on the fleet size. We use approximate dynamic programming to develop high-quality operational dispatch strategies to determine which car is best for a particular trip, when a car should be recharged, and when it should be re-positioned to a different zone which offers a higher density of trips. We prove that the value functions are monotone in the battery and time dimensions and use hierarchical aggregation to get better estimates of the value functions with a small number of observations. Then, surge pricing is discussed using an adaptive learning approach to decide on the price for each trip. Finally, we discuss the fleet size problem which depends on the previous two problems. △ Less

Submitted 11 December, 2018; v1 submitted 18 October, 2018; originally announced October 2018.

Wrangling Rogues: Managing Experimental Post-Moore Architectures

Authors: Will Powell, Jason Riedy, Jeffrey S. Young, Thomas M. Conte

Abstract: The Rogues Gallery is a new experimental testbed that is focused on tackling "rogue" architectures for the Post-Moore era of computing. While some of these devices have roots in the embedded and high-performance computing spaces, managing current and emerging technologies provides a challenge for system administration that are not always foreseen in traditional data center environments. We prese… ▽ More The Rogues Gallery is a new experimental testbed that is focused on tackling "rogue" architectures for the Post-Moore era of computing. While some of these devices have roots in the embedded and high-performance computing spaces, managing current and emerging technologies provides a challenge for system administration that are not always foreseen in traditional data center environments. We present an overview of the motivations and design of the initial Rogues Gallery testbed and cover some of the unique challenges that we have seen and foresee with upcoming hardware prototypes for future post-Moore research. Specifically, we cover the networking, identity management, scheduling of resources, and tools and sensor access aspects of the Rogues Gallery and techniques we have developed to manage these new platforms. △ Less

Submitted 1 August, 2019; v1 submitted 20 August, 2018; originally announced August 2018.

Recursive Optimization of Convex Risk Measures: Mean-Semideviation Models

Authors: Dionysios S. Kalogerias, Warren B. Powell

Abstract: We develop recursive, data-driven, stochastic subgradient methods for optimizing a new, versatile, and application-driven class of convex risk measures, termed here as mean-semideviations, strictly generalizing the well-known and popular mean-upper-semideviation. We introduce the MESSAGEp algorithm, which is an efficient compositional subgradient procedure for iteratively solving convex mean-semid… ▽ More We develop recursive, data-driven, stochastic subgradient methods for optimizing a new, versatile, and application-driven class of convex risk measures, termed here as mean-semideviations, strictly generalizing the well-known and popular mean-upper-semideviation. We introduce the MESSAGEp algorithm, which is an efficient compositional subgradient procedure for iteratively solving convex mean-semideviation risk-averse problems to optimality. We analyze the asymptotic behavior of the MESSAGEp algorithm under a flexible and structure-exploiting set of problem assumptions. In particular: 1) Under appropriate stepsize rules, we establish pathwise convergence of the MESSAGEp algorithm in a strong technical sense, confirming its asymptotic consistency. 2) Assuming a strongly convex cost, we show that, for fixed semideviation order $p>1$ and for $ε\in\left[0,1\right)$, the MESSAGEp algorithm achieves a squared-${\cal L}_{2}$ solution suboptimality rate of the order of ${\cal O}(n^{-\left(1-ε\right)/2})$ iterations, where, for $ε>0$, pathwise convergence is simultaneously guaranteed. This result establishes a rate of order arbitrarily close to ${\cal O}(n^{-1/2})$, while ensuring strongly stable pathwise operation. For $p\equiv1$, the rate order improves to ${\cal O}(n^{-2/3})$, which also suffices for pathwise convergence, and matches previous results. 3) Likewise, in the general case of a convex cost, we show that, for any $ε\in\left[0,1\right)$, the MESSAGEp algorithm with iterate smoothing achieves an ${\cal L}_{1}$ objective suboptimality rate of the order of ${\cal O}(n^{-\left(1-ε\right)/\left(4\bf{1}_{\left\{ p>1\right\} }+4\right)})$ iterations. This result provides maximal rates of ${\cal O}(n^{-1/4})$, if $p\equiv1$, and ${\cal O}(n^{-1/8})$, if $p>1$, matching the state of the art, as well. △ Less

Submitted 29 October, 2018; v1 submitted 2 April, 2018; originally announced April 2018.

Comments: 90 pages, 3 figures. Update: Substantial revision of the technical content, with an additional fully detailed analysis in regard to the rate of convergence of the MESSAGEp algorithm. NOTE: Please open in browser to see the math in the abstract!

Reinforcement Learning for Dynamic Bidding in Truckload Markets: an Application to Large-Scale Fleet Management with Advance Commitments

Authors: Yingfei Wang, Juliana Martins Do Nascimento, Warren Powell

Abstract: Truckload brokerages, a $100 billion/year industry in the U.S., plays the critical role of matching shippers with carriers, often to move loads several days into the future. Brokerages not only have to find companies that will agree to move a load, the brokerage often has to find a price that both the shipper and carrier will agree to. The price not only varies by shipper and carrier, but also by… ▽ More Truckload brokerages, a $100 billion/year industry in the U.S., plays the critical role of matching shippers with carriers, often to move loads several days into the future. Brokerages not only have to find companies that will agree to move a load, the brokerage often has to find a price that both the shipper and carrier will agree to. The price not only varies by shipper and carrier, but also by the traffic lanes and other variables such as commodity type. Brokerages have to learn about shipper and carrier response functions by offering a price and observing whether each accepts the quote. We propose a knowledge gradient policy with bootstrap aggregation for high-dimensional contextual settings to guide price experimentation by maximizing the value of information. The learning policy is tested using a carefully calibrated fleet simulator that includes a stochastic lookahead policy that simulates fleet movements, as well as the stochastic modeling of driver assignments and the carrier's load commitment policies with advance booking. △ Less

Submitted 4 June, 2019; v1 submitted 25 February, 2018; originally announced February 2018.

Backward Approximate Dynamic Programming with Hidden Semi-Markov Stochastic Models in Energy Storage Optimization

Authors: Joseph L. Durante, Juliana Nascimento, Warren B. Powell

Abstract: We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power forecast errors are modeled using a novel hidden semi-Markov model that accurately replicates not just the distribution of the errors, but also crossing times, cap… ▽ More We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power forecast errors are modeled using a novel hidden semi-Markov model that accurately replicates not just the distribution of the errors, but also crossing times, capturing the amount of time each process stays above or below some benchmark such as the forecast. This is an important property of stochastic processes involved in storage problems. We show that we achieve more robust solutions using this model than when more common stochastic models are considered. The new model introduces some additional complexity to the problem as its information states are partially hidden, forming a partially observable Markov decision process. We derive a near-optimal time-dependent policy using backward approximate dynamic programming, which overcomes the computational hurdles of classical (exact) backward dynamic programming, with higher quality solutions than the more familiar forward approximate dynamic programming methods. △ Less

Submitted 1 February, 2020; v1 submitted 11 October, 2017; originally announced October 2017.

Comments: 36 pages, 7 figures, replacement: up to date version, additional references added

Optimal Learning for Sequential Decision Making for Expensive Cost Functions with Stochastic Binary Feedbacks

Authors: Yingfei Wang, Chu Wang, Warren Powell

Abstract: We consider the problem of sequentially making decisions that are rewarded by "successes" and "failures" which can be predicted through an unknown relationship that depends on a partially controllable vector of attributes for each instance. The learner takes an active role in selecting samples from the instance pool. The goal is to maximize the probability of success in either offline (training) o… ▽ More We consider the problem of sequentially making decisions that are rewarded by "successes" and "failures" which can be predicted through an unknown relationship that depends on a partially controllable vector of attributes for each instance. The learner takes an active role in selecting samples from the instance pool. The goal is to maximize the probability of success in either offline (training) or online (testing) phases. Our problem is motivated by real-world applications where observations are time-consuming and/or expensive. We develop a knowledge gradient policy using an online Bayesian linear classifier to guide the experiment by maximizing the expected value of information of labeling each alternative. We provide a finite-time analysis of the estimated error and show that the maximum likelihood estimator based produced by the KG policy is consistent and asymptotically normal. We also show that the knowledge gradient policy is asymptotically optimal in an offline setting. This work further extends the knowledge gradient to the setting of contextual bandits. We report the results of a series of experiments that demonstrate its efficiency. △ Less

Submitted 13 September, 2017; originally announced September 2017.

Comments: arXiv admin note: text overlap with arXiv:1510.02354

MOLTE: a Modular Optimal Learning Testing Environment

Authors: Yingfei Wang, Warren Powell

Abstract: We address the relative paucity of empirical testing of learning algorithms (of any type) by introducing a new public-domain, Modular, Optimal Learning Testing Environment (MOLTE) for Bayesian ranking and selection problem, stochastic bandits or sequential experimental design problems. The Matlab-based simulator allows the comparison of a number of learning policies (represented as a series of .m… ▽ More We address the relative paucity of empirical testing of learning algorithms (of any type) by introducing a new public-domain, Modular, Optimal Learning Testing Environment (MOLTE) for Bayesian ranking and selection problem, stochastic bandits or sequential experimental design problems. The Matlab-based simulator allows the comparison of a number of learning policies (represented as a series of .m modules) in the context of a wide range of problems (each represented in its own .m module) which makes it easy to add new algorithms and new test problems. State-of-the-art policies and various problem classes are provided in the package. The choice of problems and policies is guided through a spreadsheet-based interface. Different graphical metrics are included. MOLTE is designed to be compatible with parallel computing to scale up from local desktop to clusters and clouds. We offer MOLTE as an easy-to-use tool for the research community that will make it possible to perform much more comprehensive testing, spanning a broader selection of algorithms and test problems. We demonstrate the capabilities of MOLTE through a series of comparisons of policies on a starter library of test problems. We also address the problem of tuning and constructing priors that have been largely overlooked in optimal learning literature. We envision MOLTE as a modest spur to provide researchers an easy environment to study interesting questions involved in optimal learning. △ Less

Submitted 13 September, 2017; originally announced September 2017.

Monte Carlo Tree Search with Sampled Information Relaxation Dual Bounds

Authors: Daniel R. Jiang, Lina Al-Kanj, Warren B. Powell

Abstract: Monte Carlo Tree Search (MCTS), most famously used in game-play artificial intelligence (e.g., the game of Go), is a well-known strategy for constructing approximate solutions to sequential decision problems. Its primary innovation is the use of a heuristic, known as a default policy, to obtain Monte Carlo estimates of downstream values for states in a decision tree. This information is used to it… ▽ More Monte Carlo Tree Search (MCTS), most famously used in game-play artificial intelligence (e.g., the game of Go), is a well-known strategy for constructing approximate solutions to sequential decision problems. Its primary innovation is the use of a heuristic, known as a default policy, to obtain Monte Carlo estimates of downstream values for states in a decision tree. This information is used to iteratively expand the tree towards regions of states and actions that an optimal policy might visit. However, to guarantee convergence to the optimal action, MCTS requires the entire tree to be expanded asymptotically. In this paper, we propose a new technique called Primal-Dual MCTS that utilizes sampled information relaxation upper bounds on potential actions, creating the possibility of "ignoring" parts of the tree that stem from highly suboptimal choices. This allows us to prove that despite converging to a partial decision tree in the limit, the recommended action from Primal-Dual MCTS is optimal. The new approach shows significant promise when used to optimize the behavior of a single driver navigating a graph while operating on a ride-sharing platform. Numerical experiments on a real dataset of 7,000 trips in New Jersey suggest that Primal-Dual MCTS improves upon standard MCTS by producing deeper decision trees and exhibits a reduced sensitivity to the size of the action space. △ Less

Submitted 19 April, 2017; originally announced April 2017.

Comments: 33 pages, 6 figures

Stochastic Optimization with Parametric Cost Function Approximations

Authors: Raymond T. Perkins III, Warren B. Powell

Abstract: A widely used heuristic for solving stochastic optimization problems is to use a deterministic rolling horizon procedure, which has been modified to handle uncertainty (e.g. buffer stocks, schedule slack). This approach has been criticized for its use of a deterministic approximation of a stochastic problem, which is the major motivation for stochastic programming. We recast this debate by identif… ▽ More A widely used heuristic for solving stochastic optimization problems is to use a deterministic rolling horizon procedure, which has been modified to handle uncertainty (e.g. buffer stocks, schedule slack). This approach has been criticized for its use of a deterministic approximation of a stochastic problem, which is the major motivation for stochastic programming. We recast this debate by identifying both deterministic and stochastic approaches as policies for solving a stochastic base model, which may be a simulator or the real world. Stochastic lookahead models (stochastic programming) require a range of approximations to keep the problem tractable. By contrast, so-called deterministic models are actually parametrically modified cost function approximations which use parametric adjustments to the objective function and/or the constraints. These parameters are then optimized in a stochastic base model which does not require making any of the types of simplifications required by stochastic programming. We formalize this strategy and describe a gradient-based stochastic search strategy to optimize the parameters. △ Less

Submitted 14 March, 2017; originally announced March 2017.

Optimal Learning for Stochastic Optimization with Nonlinear Parametric Belief Models

Authors: Xinyu He, Warren B. Powell

Abstract: We consider the problem of estimating the expected value of information (the knowledge gradient) for Bayesian learning problems where the belief model is nonlinear in the parameters. Our goal is to maximize some metric, while simultaneously learning the unknown parameters of the nonlinear belief model, by guiding a sequential experimentation process which is expensive. We overcome the problem of c… ▽ More We consider the problem of estimating the expected value of information (the knowledge gradient) for Bayesian learning problems where the belief model is nonlinear in the parameters. Our goal is to maximize some metric, while simultaneously learning the unknown parameters of the nonlinear belief model, by guiding a sequential experimentation process which is expensive. We overcome the problem of computing the expected value of an experiment, which is computationally intractable, by using a sampled approximation, which helps to guide experiments but does not provide an accurate estimate of the unknown parameters. We then introduce a resampling process which allows the sampled model to adapt to new information, exploiting past experiments. We show theoretically that the method converges asymptotically to the true parameters, while simultaneously maximizing our metric. We show empirically that the process exhibits rapid convergence, yielding good results with a very small number of experiments. △ Less

Submitted 22 November, 2016; originally announced November 2016.

An optimal learning method for develo** personalized treatment regimes

Authors: Yingfei Wang, Warren Powell

Abstract: A treatment regime is a function that maps individual patient information to a recommended treatment, hence explicitly incorporating the heterogeneity in need for treatment across individuals. Patient responses are dichotomous and can be predicted through an unknown relationship that depends on the patient information and the selected treatment. The goal is to find the treatments that lead to the… ▽ More A treatment regime is a function that maps individual patient information to a recommended treatment, hence explicitly incorporating the heterogeneity in need for treatment across individuals. Patient responses are dichotomous and can be predicted through an unknown relationship that depends on the patient information and the selected treatment. The goal is to find the treatments that lead to the best patient responses on average. Each experiment is expensive, forcing us to learn the most from each experiment. We adopt a Bayesian approach both to incorporate possible prior information and to update our treatment regime continuously as information accrues, with the potential to allow smaller yet more informative trials and for patients to receive better treatment. By formulating the problem as contextual bandits, we introduce a knowledge gradient policy to guide the treatment assignment by maximizing the expected value of information, for which an approximation method is used to overcome computational challenges. We provide a detailed study on how to make sequential medical decisions under uncertainty to reduce health care costs on a real world knee replacement dataset. We use clustering and LASSO to deal with the intrinsic sparsity in health datasets. We show experimentally that even though the problem is sparse, through careful selection of physicians (versus picking them at random), we can significantly improve the success rates. △ Less

Submitted 5 July, 2016; originally announced July 2016.

Finite-time Analysis for the Knowledge-Gradient Policy

Authors: Yingfei Wang, Warren Powell

Abstract: We consider sequential decision problems in which we adaptively choose one of finitely many alternatives and observe a stochastic reward. We offer a new perspective of interpreting Bayesian ranking and selection problems as adaptive stochastic multi-set maximization problems and derive the first finite-time bound of the knowledge-gradient policy for adaptive submodular objective functions. In addi… ▽ More We consider sequential decision problems in which we adaptively choose one of finitely many alternatives and observe a stochastic reward. We offer a new perspective of interpreting Bayesian ranking and selection problems as adaptive stochastic multi-set maximization problems and derive the first finite-time bound of the knowledge-gradient policy for adaptive submodular objective functions. In addition, we introduce the concept of prior-optimality and provide another insight into the performance of the knowledge gradient policy based on the submodular assumption on the value of information. We demonstrate submodularity for the two-alternative case and provide other conditions for more general problems, bringing out the issue and importance of submodularity in learning problems. Empirical experiments are conducted to further illustrate the finite time behavior of the knowledge gradient policy. △ Less

Submitted 14 June, 2016; originally announced June 2016.

The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response

Authors: Lina Al-Kanj, Warren B. Powell, Belgacem Bouzaiene-Ayari

Abstract: Utilities face the challenge of responding to power outages due to storms and ice damage, but most power grids are not equipped with sensors to pinpoint the precise location of the faults causing the outage. Instead, utilities have to depend primarily on phone calls (trouble calls) from customers who have lost power to guide the dispatching of utility trucks. In this paper, we develop a policy tha… ▽ More Utilities face the challenge of responding to power outages due to storms and ice damage, but most power grids are not equipped with sensors to pinpoint the precise location of the faults causing the outage. Instead, utilities have to depend primarily on phone calls (trouble calls) from customers who have lost power to guide the dispatching of utility trucks. In this paper, we develop a policy that routes a utility truck to restore outages in the power grid as quickly as possible, using phone calls to create beliefs about outages, but also using utility trucks as a mechanism for collecting additional information. This means that routing decisions change not only the physical state of the truck (as it moves from one location to another) and the grid (as the truck performs repairs), but also our belief about the network, creating the first stochastic vehicle routing problem that explicitly models information collection and belief modeling. We address the problem of managing a single utility truck, which we start by formulating as a sequential stochastic optimization model which captures our belief about the state of the grid. We propose a stochastic lookahead policy, and use Monte Carlo tree search (MCTS) to produce a practical policy that is asymptotically optimal. Simulation results show that the developed policy restores the power grid much faster compared to standard industry heuristics. △ Less

Submitted 18 May, 2016; originally announced May 2016.

Room-temperature exciton-polaritons with two-dimensional WS2

Authors: Lucas C. Flatten, Zhengyu He, David M. Coles, Aurelien A. P. Trichet, Alex W. Powell, Robert A. Taylor, Jamie H. Warner, Jason M. Smith

Abstract: Two-dimensional transition metal dichalcogenides exhibit strong optical transitions with significant potential for optoelectronic devices. In particular they are suited for cavity quantum electrodynamics in which strong coupling leads to polariton formation as a root to realisation of inversionless lasing, polariton condensationand superfluidity. Demonstrations of such strongly correlated phenomen… ▽ More Two-dimensional transition metal dichalcogenides exhibit strong optical transitions with significant potential for optoelectronic devices. In particular they are suited for cavity quantum electrodynamics in which strong coupling leads to polariton formation as a root to realisation of inversionless lasing, polariton condensationand superfluidity. Demonstrations of such strongly correlated phenomena to date have often relied on cryogenic temperatures, high excitation densities and were frequently impaired by strong material disorder. At room-temperature, experiments approaching the strong coupling regime with transition metal dichalcogenides have been reported, but well resolved exciton-polaritons have yet to be achieved. Here we report a study of monolayer WS$_2$ coupled to an open Fabry-Perot cavity at room-temperature, in which polariton eigenstates are unambiguously displayed. In-situ tunability of the cavity length results in a maximal Rabi splitting of $\hbar Ω_{\rm{Rabi}} = 70$ meV, exceeding the exciton linewidth. Our data are well described by a transfer matrix model appropriate for the large linewidth regime. This work provides a platform towards observing strongly correlated polariton phenomena in compact photonic devices for ambient temperature applications. △ Less

Submitted 30 August, 2016; v1 submitted 16 May, 2016; originally announced May 2016.

Comments: 12 pages, 6 figures

Practicality of Nested Risk Measures for Dynamic Electric Vehicle Charging

Authors: Daniel R. Jiang, Warren B. Powell

Abstract: We consider the sequential decision problem faced by the manager of an electric vehicle (EV) charging station, who aims to satisfy the charging demand of the customer while minimizing cost. Since the total time needed to charge the EV up to capacity is often less than the amount of time that the customer is away, there are opportunities to exploit electricity spot price variations within some rese… ▽ More We consider the sequential decision problem faced by the manager of an electric vehicle (EV) charging station, who aims to satisfy the charging demand of the customer while minimizing cost. Since the total time needed to charge the EV up to capacity is often less than the amount of time that the customer is away, there are opportunities to exploit electricity spot price variations within some reservation window. We formulate the problem as a finite horizon Markov decision process (MDP) and consider a risk-averse objective function by optimizing under a dynamic risk measure constructed using a convex combination of expected value and conditional value at risk (CVaR). It has been recognized that the objective function of a risk-averse MDP lacks a practical interpretation. Therefore, in both academic and industry practice, the dynamic risk measure objective is often not of primary interest; instead, the risk-averse MDP is used as a computational tool for solving problems with predefined "practical" risk and reward objectives (termed the base model). In this paper, we study the extent to which the two sides of this framework are compatible with each other for the EV setting -- roughly speaking, does a "more risk-averse" MDP provide lower risk in the practical sense as well? In order to answer such a question, the effect of the degree of dynamic risk-aversion on the optimal MDP policy is analyzed. Based on these results, we also propose a principled approximation approach to finding an instance of the risk-averse MDP whose optimal policy behaves well under the practical objectives of the base model. Our numerical experiments suggest that EV charging stations can be operated at a significantly higher level of profitability if dynamic charging is adopted and a small amount of risk is tolerated. △ Less

Submitted 3 October, 2017; v1 submitted 10 May, 2016; originally announced May 2016.

Comments: 45 pages, 15 figures

SDDP vs. ADP: The Effect of Dimensionality in Multistage Stochastic Optimization for Grid Level Energy Storage

Authors: Tsvetan Asamov, Daniel F. Salas, Warren B. Powell

Abstract: There has been widespread interest in the use of grid-level storage to handle the variability from increasing penetrations of wind and solar energy. This problem setting requires optimizing energy storage and release decisions for anywhere from a half-dozen, to potentially hundreds of storage devices spread around the grid as new technologies evolve. We approach this problem using two competing al… ▽ More There has been widespread interest in the use of grid-level storage to handle the variability from increasing penetrations of wind and solar energy. This problem setting requires optimizing energy storage and release decisions for anywhere from a half-dozen, to potentially hundreds of storage devices spread around the grid as new technologies evolve. We approach this problem using two competing algorithmic strategies. The first, developed within the stochastic programming literature, is stochastic dual dynamic programming (SDDP) which uses Benders decomposition to create a multidimensional value function approximations, which have been widely used to manage hydro reservoirs. The second approach, which has evolved using the language of approximate dynamic programming, uses separable, piecewise linear value function approximations, a method which has been successfully applied to high-dimensional fleet management problems. This paper brings these two approaches together using a common notational system, and contrasts the algorithmic strategies (which are both a form of approximate dynamic programming) used by each approach. The methods are then subjected to rigorous testing using the context of optimizing grid level storage. △ Less

Submitted 5 May, 2016; originally announced May 2016.

The Knowledge Gradient with Logistic Belief Models for Binary Classification

Authors: Yingfei Wang, Chu Wang, Warren Powell

Abstract: We consider sequential decision making problems for binary classification scenario in which the learner takes an active role in repeatedly selecting samples from the action pool and receives the binary label of the selected alternatives. Our problem is motivated by applications where observations are time consuming and/or expensive, resulting in small samples. The goal is to identify the best alte… ▽ More We consider sequential decision making problems for binary classification scenario in which the learner takes an active role in repeatedly selecting samples from the action pool and receives the binary label of the selected alternatives. Our problem is motivated by applications where observations are time consuming and/or expensive, resulting in small samples. The goal is to identify the best alternative with the highest response. We use Bayesian logistic regression to predict the response of each alternative. By formulating the problem as a Markov decision process, we develop a knowledge-gradient type policy to guide the experiment by maximizing the expected value of information of labeling each alternative and provide a finite-time analysis on the estimated error. Experiments on benchmark UCI datasets demonstrate the effectiveness of the proposed method. △ Less

Submitted 8 October, 2015; originally announced October 2015.

Observations and Analysis of Three Field RR Lyrae Stars Selected Using Single epoch SDSS Data

Authors: W. Lee Powell Jr., Stephanie Jameson, Nathan De Lee, Ronald J. Wilhelm

Abstract: We present the results of our Johnson B and V observations of three RR Lyrae candidate stars that we identified as likely variable stars using SDSS data. The stars were selected based upon a single epoch of photometry and spectroscopy. The stars were observed at McDonald Observatory to obtain full light curves. We present full light curves, measured periods, and amplitudes, as well as the results… ▽ More We present the results of our Johnson B and V observations of three RR Lyrae candidate stars that we identified as likely variable stars using SDSS data. The stars were selected based upon a single epoch of photometry and spectroscopy. The stars were observed at McDonald Observatory to obtain full light curves. We present full light curves, measured periods, and amplitudes, as well as the results of our Fourier analysis of the light curves. △ Less

Submitted 22 September, 2015; originally announced September 2015.

Comments: 6 pages. Accepted to the Journal of the American Association of Variable Star Observers (JAAVSO) August 28, 2015

Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures

Authors: Daniel R. Jiang, Warren B. Powell

Abstract: In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs; we call the overall objective a dynamic quantile-based risk measure (DQBRM). In particular, we consider optimizing dynamic risk measures where the one-step risk measures are QBRMs, a class of risk measur… ▽ More In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs; we call the overall objective a dynamic quantile-based risk measure (DQBRM). In particular, we consider optimizing dynamic risk measures where the one-step risk measures are QBRMs, a class of risk measures that includes the popular value at risk (VaR) and the conditional value at risk (CVaR). Although there is considerable theoretical development of risk-averse MDPs in the literature, the computational challenges have not been explored as thoroughly. We propose data-driven and simulation-based approximate dynamic programming (ADP) algorithms to solve the risk-averse sequential decision problem. We address the issue of inefficient sampling for risk applications in simulated settings and present a procedure, based on importance sampling, to direct samples toward the "risky region" as the ADP algorithm progresses. Finally, we show numerical results of our algorithms in the context of an application involving risk-averse bidding for energy storage. △ Less

Submitted 8 May, 2017; v1 submitted 7 September, 2015; originally announced September 2015.

Comments: 39 pages, 7 figures

A Knowledge Gradient Policy for Sequencing Experiments to Identify the Structure of RNA Molecules Using a Sparse Additive Belief Model

Authors: Yan Li, Kristofer G. Reyes, Jorge Vazquez-Anderson, Yingfei Wang, Lydia M. Contreras, Warren B. Powell

Abstract: We present a sparse knowledge gradient (SpKG) algorithm for adaptively selecting the targeted regions within a large RNA molecule to identify which regions are most amenable to interactions with other molecules. Experimentally, such regions can be inferred from fluorescence measurements obtained by binding a complementary probe with fluorescence markers to the targeted regions. We use a biophysica… ▽ More We present a sparse knowledge gradient (SpKG) algorithm for adaptively selecting the targeted regions within a large RNA molecule to identify which regions are most amenable to interactions with other molecules. Experimentally, such regions can be inferred from fluorescence measurements obtained by binding a complementary probe with fluorescence markers to the targeted regions. We use a biophysical model which shows that the fluorescence ratio under the log scale has a sparse linear relationship with the coefficients describing the accessibility of each nucleotide, since not all sites are accessible (due to the folding of the molecule). The SpKG algorithm uniquely combines the Bayesian ranking and selection problem with the frequentist $\ell_1$ regularized regression approach Lasso. We use this algorithm to identify the sparsity pattern of the linear model as well as sequentially decide the best regions to test before experimental budget is exhausted. Besides, we also develop two other new algorithms: batch SpKG algorithm, which generates more suggestions sequentially to run parallel experiments; and batch SpKG with a procedure which we call length mutagenesis. It dynamically adds in new alternatives, in the form of types of probes, are created by inserting, deleting or mutating nucleotides within existing probes. In simulation, we demonstrate these algorithms on the Group I intron (a mid-size RNA molecule), showing that they efficiently learn the correct sparsity pattern, identify the most accessible region, and outperform several other policies. △ Less

Submitted 6 August, 2015; originally announced August 2015.

Regularized Decomposition of High-Dimensional Multistage Stochastic Programs with Markov Uncertainty

Authors: Tsvetan Asamov, Warren B. Powell

Abstract: We develop a quadratic regularization approach for the solution of high-dimensional multistage stochastic optimization problems characterized by a potentially large number of time periods/stages (e.g. hundreds), a high-dimensional resource state variable, and a Markov information process. The resulting algorithms are shown to converge to an optimal policy after a finite number of iterations under… ▽ More We develop a quadratic regularization approach for the solution of high-dimensional multistage stochastic optimization problems characterized by a potentially large number of time periods/stages (e.g. hundreds), a high-dimensional resource state variable, and a Markov information process. The resulting algorithms are shown to converge to an optimal policy after a finite number of iterations under mild technical assumptions. Computational experiments are conducted using the setting of optimizing energy storage over a large transmission grid, which motivates both the spatial and temporal dimensions of our problem. Our numerical results indicate that the proposed methods exhibit significantly faster convergence than their classical counterparts, with greater gains observed for higher-dimensional problems. △ Less

Submitted 26 February, 2017; v1 submitted 8 May, 2015; originally announced May 2015.