Showing 1–50 of 104 results for author: Spanos, C

Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation

Authors: Shangding Gu, Laixi Shi, Yuhao Ding, Alois Knoll, Costas Spanos, Adam Wierman, Ming **

Abstract: Safe reinforcement learning (RL) is crucial for deploying RL agents in real-world applications, as it aims to maximize long-term rewards while satisfying safety constraints. However, safe RL often suffers from sample inefficiency, requiring extensive interactions with the environment to learn a safe policy. We propose Efficient Safe Policy Optimization (ESPO), a novel approach that enhances the ef… ▽ More Safe reinforcement learning (RL) is crucial for deploying RL agents in real-world applications, as it aims to maximize long-term rewards while satisfying safety constraints. However, safe RL often suffers from sample inefficiency, requiring extensive interactions with the environment to learn a safe policy. We propose Efficient Safe Policy Optimization (ESPO), a novel approach that enhances the efficiency of safe RL through sample manipulation. ESPO employs an optimization framework with three modes: maximizing rewards, minimizing costs, and balancing the trade-off between the two. By dynamically adjusting the sampling process based on the observed conflict between reward and safety gradients, ESPO theoretically guarantees convergence, optimization stability, and improved sample complexity bounds. Experiments on the Safety-MuJoCo and Omnisafe benchmarks demonstrate that ESPO significantly outperforms existing primal-based and primal-dual-based baselines in terms of reward maximization and constraint satisfaction. Moreover, ESPO achieves substantial gains in sample efficiency, requiring 25--29% fewer samples than baselines, and reduces training time by 21--38%. △ Less

Submitted 31 May, 2024; originally announced May 2024.

Active Reinforcement Learning for Robust Building Control

Authors: Doseok Jang, Larry Yan, Lucas Spangher, Costas Spanos

Abstract: Reinforcement learning (RL) is a powerful tool for optimal control that has found great success in Atari games, the game of Go, robotic control, and building optimization. RL is also very brittle; agents often overfit to their training environment and fail to generalize to new settings. Unsupervised environment design (UED) has been proposed as a solution to this problem, in which the agent trains… ▽ More Reinforcement learning (RL) is a powerful tool for optimal control that has found great success in Atari games, the game of Go, robotic control, and building optimization. RL is also very brittle; agents often overfit to their training environment and fail to generalize to new settings. Unsupervised environment design (UED) has been proposed as a solution to this problem, in which the agent trains in environments that have been specially selected to help it learn. Previous UED algorithms focus on trying to train an RL agent that generalizes across a large distribution of environments. This is not necessarily desirable when we wish to prioritize performance in one environment over others. In this work, we will be examining the setting of robust RL building control, where we wish to train an RL agent that prioritizes performing well in normal weather while still being robust to extreme weather conditions. We demonstrate a novel UED algorithm, ActivePLR, that uses uncertainty-aware neural network architectures to generate new training environments at the limit of the RL agent's ability while being able to prioritize performance in a desired base environment. We show that ActivePLR is able to outperform state-of-the-art UED algorithms in minimizing energy usage while maximizing occupant comfort in the setting of building control. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Toward Platform-based Building Design

Authors: Yu-Wen Lin, Tsz Ling Elaine Tang, Stefano Schiavon, Costas J. Spanos

Abstract: The electronic design industry has undergone a significant transformation, transitioning from traditional hand-drawn designs to modern automated design processes. While Computer-Aided Design (CAD) tools emerged alongside the electronic industry, the current building design process has little to no automation. There is a need for a unified platform to address the complexity of building design and p… ▽ More The electronic design industry has undergone a significant transformation, transitioning from traditional hand-drawn designs to modern automated design processes. While Computer-Aided Design (CAD) tools emerged alongside the electronic industry, the current building design process has little to no automation. There is a need for a unified platform to address the complexity of building design and provide a more systematic approach. Platform-based design (PBD), originally developed in the electronic industry, enables efficient design processes by promoting the reuse of hardware and software systems. It also facilitates design space exploration while optimizing performance. This paper proposes a modular approach that divides the building into various disciplines and introduces a design flow using the PBD framework to streamline the design process. We also present a case study that demonstrates the use of the PBD framework in the Heating, Ventilation, and Air Conditioning (HVAC) systems design. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Comments: 17 pages, 3 figures

From Electronic Design Automation to Building Design Automation: Challenges and Opportunities

Authors: Yu-Wen Lin, Tsz Ling Elaine Tang, Alberto L. Sangiovanni-Vincentelli, Stefano Schiavon, Costas J. Spanos

Abstract: Design automation, which involves the use of software tools and technologies to streamline the design process, has been widely adopted in the electronics industry, resulting in significant advancements in product development and manufacturing. However, building design, which involves the creation of complex structures and systems, has traditionally lagged behind in leveraging design automation tec… ▽ More Design automation, which involves the use of software tools and technologies to streamline the design process, has been widely adopted in the electronics industry, resulting in significant advancements in product development and manufacturing. However, building design, which involves the creation of complex structures and systems, has traditionally lagged behind in leveraging design automation technologies. Despite extensive research on design automation in the building industry, its application in the current design of buildings is limited. This paper aims to (1) compare the design processes between electronics and building design, (2) highlight similarities and differences in their approaches, and (3) examine challenges and opportunities associated with bringing the concept of design automation from electronics to building design. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Freeze-in baryogenesis and early matter domination

Authors: Ioannis Dalianis, Andreas Goudelis, Dimitrios Karamitros, Pantelis Papachristou, Vassilis C. Spanos

Abstract: The freeze-in mechanism has been shown to allow the simultaneous generation of cosmic dark matter and a viable matter-antimatter asymmetry in the universe. When the underlying interactions are described by higher-dimensional, non-renormalizable operators, the relevant freeze-in processes take place close to the highest considered cosmic temperatures. In this paper we study how the presence of a fl… ▽ More The freeze-in mechanism has been shown to allow the simultaneous generation of cosmic dark matter and a viable matter-antimatter asymmetry in the universe. When the underlying interactions are described by higher-dimensional, non-renormalizable operators, the relevant freeze-in processes take place close to the highest considered cosmic temperatures. In this paper we study how the presence of a fluid that temporarily dominates the energy content of the early universe affects the predictions of this ``Ultraviolet Freeze-In Baryogenesis'' scenario. We find that this additional cosmic component has a significant impact on the predictions of concrete microscopic models, allowing for reheating temperatures which are much lower than those required in the simplest cosmological scenario. Moreover, we show that inflationary observables can constrain the parameter space of such models, once the latter are examined in conjunction with concrete models of inflation. △ Less

Submitted 14 April, 2023; originally announced April 2023.

Comments: 25 pages, 6 figures

Machine Learning for Smart and Energy-Efficient Buildings

Authors: Hari Prasanna Das, Yu-Wen Lin, Utkarsha Agwan, Lucas Spangher, Alex Devonport, Yu Yang, Jan Drgona, Adrian Chong, Stefano Schiavon, Costas J. Spanos

Abstract: Energy consumption in buildings, both residential and commercial, accounts for approximately 40% of all energy usage in the U.S., and similar numbers are being reported from countries around the world. This significant amount of energy is used to maintain a comfortable, secure, and productive environment for the occupants. So, it is crucial that the energy consumption in buildings must be optimize… ▽ More Energy consumption in buildings, both residential and commercial, accounts for approximately 40% of all energy usage in the U.S., and similar numbers are being reported from countries around the world. This significant amount of energy is used to maintain a comfortable, secure, and productive environment for the occupants. So, it is crucial that the energy consumption in buildings must be optimized, all the while maintaining satisfactory levels of occupant comfort, health, and safety. Recently, Machine Learning has been proven to be an invaluable tool in deriving important insights from data and optimizing various systems. In this work, we review the ways in which machine learning has been leveraged to make buildings smart and energy-efficient. For the convenience of readers, we provide a brief introduction of several machine learning paradigms and the components and functioning of each smart building system we cover. Finally, we discuss challenges faced while implementing machine learning algorithms in smart buildings and provide future avenues for research at the intersection of smart buildings and machine learning. △ Less

Submitted 27 November, 2022; originally announced November 2022.

The CMSSM Survives Planck, the LHC, LUX-ZEPLIN, Fermi-LAT, H.E.S.S. and IceCube

Authors: John Ellis, Keith A. Olive, Vassilis C. Spanos, Ioanna D. Stamou

Abstract: We revisit the viability of the CMSSM, searching for regions of parameter space that yield a neutralino dark matter density compatible with Planck measurements, as well as LHC constraints including sparticle searches and the mass of the Higgs boson, recent direct limits on spin-independent and -dependent dark matter scattering from the LUX-ZEPLIN (LZ) experiment, the indirect constraints from Ferm… ▽ More We revisit the viability of the CMSSM, searching for regions of parameter space that yield a neutralino dark matter density compatible with Planck measurements, as well as LHC constraints including sparticle searches and the mass of the Higgs boson, recent direct limits on spin-independent and -dependent dark matter scattering from the LUX-ZEPLIN (LZ) experiment, the indirect constraints from Fermi-LAT and H.E.S.S. on dark matter annihilations to photons in dwarf spheroidal galaxies and the Galactic Centre, and the IceCube limits on muons from annihilations to neutrinos in the Sun. For representative values of $\tan β$ and $A_0$ we map in detail the Planck-compatible strips in CMSSM parameter planes, which exhibit multiple distinctive features for large $\tan β$, $A_0 = 0$ and $μ> 0$, and identify portions of the strips that survive all the phenomenological constraints. We find that the most powerful constraint is that from $m_h$, followed by the LZ limit on spin-independent scattering, whereas sparticle searches at the LHC and indirect dark matter searches are less restrictive. Most of the surviving CMSSM parameter space features a Higgsino-like dark matter particle with a mass $\sim 1000-1100$ GeV, which could best be probed with future direct searches for dark matter scattering. △ Less

Submitted 25 March, 2023; v1 submitted 28 October, 2022; originally announced October 2022.

Comments: 44 pages, 20 figures, journal version to appear in EPJ-C

Report number: KCL-PH-TH/2022-52, CERN-TH-2022-172, UMN-TH-4204/22, FTPI-MINN-22/29

Personalized Federated Hypernetworks for Privacy Preservation in Multi-Task Reinforcement Learning

Authors: Doseok Jang, Larry Yan, Lucas Spangher, Costas J. Spanos

Abstract: Multi-Agent Reinforcement Learning currently focuses on implementations where all data and training can be centralized to one machine. But what if local agents are split across multiple tasks, and need to keep data private between each? We develop the first application of Personalized Federated Hypernetworks (PFH) to Reinforcement Learning (RL). We then present a novel application of PFH to few-sh… ▽ More Multi-Agent Reinforcement Learning currently focuses on implementations where all data and training can be centralized to one machine. But what if local agents are split across multiple tasks, and need to keep data private between each? We develop the first application of Personalized Federated Hypernetworks (PFH) to Reinforcement Learning (RL). We then present a novel application of PFH to few-shot transfer, and demonstrate significant initial increases in learning. PFH has never been demonstrated beyond supervised learning benchmarks, so we apply PFH to an important domain: RL price-setting for energy demand response. We consider a general case across where agents are split across multiple microgrids, wherein energy consumption data must be kept private within each microgrid. Together, our work explores how the fields of personalized federated learning and RL can come together to make learning efficient across multiple tasks while kee** data secure. △ Less

Submitted 19 October, 2022; v1 submitted 13 October, 2022; originally announced October 2022.

Asset Participation and Aggregation in Incentive-Based Demand Response Programs

Authors: Utkarsha Agwan, Costas J. Spanos, Kameshwar Poolla

Abstract: In order to manage peak-grid events, utilities run incentive-based demand response (DR) programs in which they offer an incentive to assets who promise to curtail power consumption, and impose penalties if they fail to do so. We develop a probabilistic model for the curtailment capability of these assets, and use it to derive analytic expressions for the optimal participation (i.e., promised curta… ▽ More In order to manage peak-grid events, utilities run incentive-based demand response (DR) programs in which they offer an incentive to assets who promise to curtail power consumption, and impose penalties if they fail to do so. We develop a probabilistic model for the curtailment capability of these assets, and use it to derive analytic expressions for the optimal participation (i.e., promised curtailment) and profitability from the DR asset perspective. We also investigate the effects of risk-aversion and curtailment uncertainty on both promised curtailment and profit. We use the probabilistic model to evaluate the benefits of forming asset aggregations for participation in DR programs, and develop a numerical test to estimate asset complementarity. We illustrate our results using load data from commercial office buildings. △ Less

Submitted 23 August, 2022; originally announced August 2022.

Comments: IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (IEEE SmartGridComm 2021)

A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features

Authors: Yu Yang, Xiaohong Guan, Qing-Shan Jia, Liang Yu, Bolun Xu, Costas J. Spanos

Abstract: By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known alternating direction method of multipliers (ADMM) has turned out to be one of the most popular tools for distributed optimization due to many advantages, such as mod… ▽ More By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known alternating direction method of multipliers (ADMM) has turned out to be one of the most popular tools for distributed optimization due to many advantages, such as modular structure, superior convergence, easy implementation and high flexibility. In the past decade, ADMM has experienced widespread developments. The developments manifest in both handling more general problems and enabling more effective implementation. Specifically, the method has been generalized to broad classes of problems (i.e.,multi-block, coupled objective, nonconvex, etc.). Besides, it has been extensively reinforced for more effective implementation, such as improved convergence rate, easier subproblems, higher computation efficiency, flexible communication, compatible with inaccurate information, robust to communication delays, etc. These developments lead to a plentiful of ADMM variants to be celebrated by broad areas ranging from smart grids, smart buildings, wireless communications, machine learning and beyond. However, there lacks a survey to document those developments and discern the results. To achieve such a goal, this paper provides a comprehensive survey on ADMM variants. Particularly, we discern the five major classes of problems that have been mostly concerned and discuss the related ADMM variants in terms of main ideas, main assumptions, convergence behaviors and main features. In addition, we figure out several important future research directions to be addressed. This survey is expected to work as a tutorial for both develo** distributed optimization in broad areas and identifying existing theoretical research gaps. △ Less

Submitted 23 August, 2022; v1 submitted 7 August, 2022; originally announced August 2022.

Comments: 35 pages, 7 figures

Primordial Black Holes and Gravitational Waves in Multi-Axion-Chern-Simons Inflation

Authors: Nick E. Mavromatos, Vassilis C. Spanos, Ioanna D. Stamou

Abstract: We study aspects of inflation and the possibility of enhanced production of primordial black holes (PBHs) and gravitational waves (GWs) in a string-inspired model of two axion fields coupled to Chern-Simons gravity, which results in a running-vacuum-model inflation. Fluctuations of the scale invariant spectrum, consistent with the cosmological data, are provided in this model by world-sheet (non-p… ▽ More We study aspects of inflation and the possibility of enhanced production of primordial black holes (PBHs) and gravitational waves (GWs) in a string-inspired model of two axion fields coupled to Chern-Simons gravity, which results in a running-vacuum-model inflation. Fluctuations of the scale invariant spectrum, consistent with the cosmological data, are provided in this model by world-sheet (non-perturbative) instanton terms of the axion field arising from string compactification. As a result of such modulations, there is an enhanced production of PBHs and GWs in such cosmologies, which may lead to observable in principle patterns in the profile of GWs during the radiation era. Moreover, we demonstrate that the PBHs may provide a significant amount of Dark Matter in this Universe. For comparison, we also discuss a two-stage inflation cosmological model of conventional string-inspired axion monodromy, involving again two axion fields. The resulting modifications imprinted on the GWs spectra between these two classes of models are distinct, and can, in principle, be distinguished by future interferometers. We consider models with more or less instantaneous reheating. We also make some remarks on the effects of a prolonged reheating period in leading to further enhancement of the power spectrum and thus fractions of PBHs that play the role of Dark matter. △ Less

Submitted 17 September, 2022; v1 submitted 16 June, 2022; originally announced June 2022.

Comments: 30 pages, 14 figures. Version to appear in PRD. Discussion on reheating issues has been added in section V and in Appendix B

Report number: KCL-PH-TH/2022-36

Gravitational Waves From No-Scale Supergravity

Authors: Vassilis C. Spanos, Ioanna D. Stamou

Abstract: In this paper we study four concrete models, based on no-scale supergravity with SU(2,1)/SU(2)$\times$ U(1) symmetry. We modify either the Kähler potential or the superpotential, which are related to the no-scale theory with this symmetry. In this scenario, the induced Gravitational Waves, are calculated to be detectable by the future space-based observations such as LISA, BBO and DECIGO. The mode… ▽ More In this paper we study four concrete models, based on no-scale supergravity with SU(2,1)/SU(2)$\times$ U(1) symmetry. We modify either the Kähler potential or the superpotential, which are related to the no-scale theory with this symmetry. In this scenario, the induced Gravitational Waves, are calculated to be detectable by the future space-based observations such as LISA, BBO and DECIGO. The models under study are interrelated, as they all yield the Starobinsky effective-like scalar potential in the unmodified case. We evaluate numerically the scalar power spectrum and the stochastic background of the Gravitational Waves, satisfying the observational Planck cosmological constraints for inflation. △ Less

Submitted 19 December, 2022; v1 submitted 11 May, 2022; originally announced May 2022.

Comments: 33 pages, 8 figures, Version to appear in European Physical Journal C

Proximal ADMM for Nonconvex and Nonsmooth Optimization

Authors: Yu Yang, Qing-Shan Jia, Zhanbo Xu, Xiaohong Guan, Costas J. Spanos

Abstract: By enabling the nodes or agents to solve small-sized subproblems to achieve coordination, distributed algorithms are favored by many networked systems for efficient and scalable computation. While for convex problems, substantial distributed algorithms are available, the results for the more broad nonconvex counterparts are extremely lacking. This paper develops a distributed algorithm for a class… ▽ More By enabling the nodes or agents to solve small-sized subproblems to achieve coordination, distributed algorithms are favored by many networked systems for efficient and scalable computation. While for convex problems, substantial distributed algorithms are available, the results for the more broad nonconvex counterparts are extremely lacking. This paper develops a distributed algorithm for a class of nonconvex and nonsmooth problems featured by i) a nonconvex objective formed by both separate and composite objective components regarding the decision components of interconnected agents, ii) local bounded convex constraints, and iii) coupled linear constraints. This problem is directly originated from smart buildings and is also broad in other domains. To provide a distributed algorithm with convergence guarantee, we revise the powerful tool of alternating direction method of multiplier (ADMM) and proposed a proximal ADMM. Specifically, noting that the main difficulty to establish the convergence for the nonconvex and nonsmooth optimization within the ADMM framework is to assume the boundness of dual updates, we propose to update the dual variables in a discounted manner. This leads to the establishment of a so-called sufficiently decreasing and lower bounded Lyapunov function, which is critical to establish the convergence. We prove that the method converges to some approximate stationary points. We besides showcase the efficacy and performance of the method by a numerical example and the concrete application to multi-zone heating, ventilation, and air-conditioning (HVAC) control in smart buildings. △ Less

Submitted 23 August, 2022; v1 submitted 4 May, 2022; originally announced May 2022.

Comments: 15 pges, 3 figures

Optimal Network Charge for Peer-to-Peer Energy Trading: A Grid Perspective

Authors: Yu Yang, Yue Chen, Guoqiang Hu, Costas J. Spanos

Abstract: Peer-to-peer (P2P) energy trading is a promising market scheme to accommodate the increasing distributed energy resources (DERs). However, how P2P to be integrated into the existing power systems remains to be investigated. In this paper, we apply network charge as a means for the grid operator to attribute transmission loss and ensure network constraints for empowering P2P transaction. The intera… ▽ More Peer-to-peer (P2P) energy trading is a promising market scheme to accommodate the increasing distributed energy resources (DERs). However, how P2P to be integrated into the existing power systems remains to be investigated. In this paper, we apply network charge as a means for the grid operator to attribute transmission loss and ensure network constraints for empowering P2P transaction. The interaction between the grid operator and the prosumers is modeled as a Stackelberg game, which yields a bi-level optimization problem. We prove that the Stackelberg game admits an equilibrium network charge price. Besides, we propose a method to obtain the network charge price by converting the bi-level optimization into a single-level mixed-integer quadratic programming (MIQP), which can handle a reasonable scale of prosumers efficiently. Simulations on the IEEE bus systems show that the proposed optimal network charge is favorable as it can benefit both the grid operator and the prosumers for empowering the P2P market, and achieves near-optimal social welfare. Moreover, the results show that the presence of energy storage will make the prosumers more sensitive to the network charge price changes. △ Less

Submitted 23 August, 2022; v1 submitted 4 May, 2022; originally announced May 2022.

Comments: 12 pages, 16 figures

Ultraviolet freeze-in baryogenesis

Authors: Andreas Goudelis, Dimitrios Karamitros, Pantelis Papachristou, Vassilis C. Spanos

Abstract: We study a mechanism through which the cosmic dark matter density can be explained simultaneously with the observed baryon asymmetry of the Universe. At the core of our proposal lie the out-of-equilibrium scattering processes of bath particles which are responsible for the production of feebly-interacting dark matter. The same processes violate $CP$, which further leads to an asymmetry between mat… ▽ More We study a mechanism through which the cosmic dark matter density can be explained simultaneously with the observed baryon asymmetry of the Universe. At the core of our proposal lie the out-of-equilibrium scattering processes of bath particles which are responsible for the production of feebly-interacting dark matter. The same processes violate $CP$, which further leads to an asymmetry between matter and antimatter being generated in the visible sector. We focus on the possibility that these interactions are described through non-renormalizable operators, which leads to both dark matter and the baryon asymmetry being produced at high temperatures. The mechanism is exemplified by studying two concrete scenarios, one involving scalar and one involving fermion dark matter. We find that in both cases it is, indeed, possible to achieve a common explanation for the dark matter content and the matter-antimatter asymmetry of the Universe, provided that dark matter is in the keV mass range. △ Less

Submitted 14 September, 2022; v1 submitted 28 April, 2022; originally announced April 2022.

Comments: 26 pages, 8 figures, v2: Matches the published version

Conditional Synthetic Data Generation for Personal Thermal Comfort Models

Authors: Hari Prasanna Das, Costas J. Spanos

Abstract: Personal thermal comfort models aim to predict an individual's thermal comfort response, instead of the average response of a large group. Recently, machine learning algorithms have proven to be having enormous potential as a candidate for personal thermal comfort models. But, often within the normal settings of a building, personal thermal comfort data obtained via experiments are heavily class-i… ▽ More Personal thermal comfort models aim to predict an individual's thermal comfort response, instead of the average response of a large group. Recently, machine learning algorithms have proven to be having enormous potential as a candidate for personal thermal comfort models. But, often within the normal settings of a building, personal thermal comfort data obtained via experiments are heavily class-imbalanced. There are a disproportionately high number of data samples for the "Prefer No Change" class, as compared with the "Prefer Warmer" and "Prefer Cooler" classes. Machine learning algorithms trained on such class-imbalanced data perform sub-optimally when deployed in the real world. To develop robust machine learning-based applications using the above class-imbalanced data, as well as for privacy-preserving data sharing, we propose to implement a state-of-the-art conditional synthetic data generator to generate synthetic data corresponding to the low-frequency classes. Via experiments, we show that the synthetic data generated has a distribution that mimics the real data distribution. The proposed method can be extended for use by other smart building datasets/use-cases. △ Less

Submitted 20 November, 2022; v1 submitted 10 March, 2022; originally announced March 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2109.06486

Adapting Surprise Minimizing Reinforcement Learning Techniques for Transactive Control

Authors: William Arnold, Tarang Srivastava, Lucas Spangher, Utkarsha Agwan, Costas Spanos

Abstract: Optimizing prices for energy demand response requires a flexible controller with ability to navigate complex environments. We propose a reinforcement learning controller with surprise minimizing modifications in its architecture. We suggest that surprise minimization can be used to improve learning speed, taking advantage of predictability in peoples' energy usage. Our architecture performs well i… ▽ More Optimizing prices for energy demand response requires a flexible controller with ability to navigate complex environments. We propose a reinforcement learning controller with surprise minimizing modifications in its architecture. We suggest that surprise minimization can be used to improve learning speed, taking advantage of predictability in peoples' energy usage. Our architecture performs well in a simulation of energy demand response. We propose this modification to improve functionality and save in a large scale experiment. △ Less

Submitted 10 November, 2021; originally announced November 2021.

Mechanism for baryogenesis via feebly interacting massive particles

Authors: Andreas Goudelis, Pantelis Papachristou, Vassilis C. Spanos

Abstract: We present a simple mechanism which allows the simultaneous generation of the baryon asymmetry of the Universe along with its dark matter content. To this goal, we employ the out-of-equilibrium decays of heavy bath states into a feebly coupled dark matter particle and Standard Model charged fermions. These decays lead to dark matter production via the freeze-in mechanism and, assuming that they fu… ▽ More We present a simple mechanism which allows the simultaneous generation of the baryon asymmetry of the Universe along with its dark matter content. To this goal, we employ the out-of-equilibrium decays of heavy bath states into a feebly coupled dark matter particle and Standard Model charged fermions. These decays lead to dark matter production via the freeze-in mechanism and, assuming that they further violate $CP$, can generate a viable matter-antimatter asymmetry in the resonant regime. We illustrate this mechanism by studying a particular realization of this general scenario, where the role of the heavy bath particles is played by $SU(3)_{\text{c}}\times SU(2)_{\text{L}}$-singlet vectorlike fermions with a non-zero hypercharge and dark matter is identified with a gauge-singlet real scalar field. We show that in the context of this simple model the cosmological constraints for the dark matter abundance and the baryon asymmetry are satisfied for masses of heavy vectorlike fermion states of a few TeV, potentially within reach of the High-Luminosity Run of the Large Hadron Collider. Dark matter, in turn, is predicted to be rather light, with a mass of a few keV. △ Less

Submitted 16 February, 2022; v1 submitted 10 November, 2021; originally announced November 2021.

Comments: 33 pages, 5 figures. v2: References and minor comments added, typos fixed, conclusions unchanged, matches published version

Conditional Synthetic Data Generation for Robust Machine Learning Applications with Limited Pandemic Data

Authors: Hari Prasanna Das, Ryan Tran, Japjot Singh, Xiangyu Yue, Geoff Tison, Alberto Sangiovanni-Vincentelli, Costas J. Spanos

Abstract: $\textbf{Background:}… ▽ More $\textbf{Background:}$ At the onset of a pandemic, such as COVID-19, data with proper labeling/attributes corresponding to the new disease might be unavailable or sparse. Machine Learning (ML) models trained with the available data, which is limited in quantity and poor in diversity, will often be biased and inaccurate. At the same time, ML algorithms designed to fight pandemics must have good performance and be developed in a time-sensitive manner. To tackle the challenges of limited data, and label scarcity in the available data, we propose generating conditional synthetic data, to be used alongside real data for develo** robust ML models. $\textbf{Methods:}$ We present a hybrid model consisting of a conditional generative flow and a classifier for conditional synthetic data generation. The classifier decouples the feature representation for the condition, which is fed to the flow to extract the local noise. We generate synthetic data by manipulating the local noise with fixed conditional feature representation. We also propose a semi-supervised approach to generate synthetic samples in the absence of labels for a majority of the available data. $\textbf{Results:}$ We performed conditional synthetic generation for chest computed tomography (CT) scans corresponding to normal, COVID-19, and pneumonia afflicted patients. We show that our method significantly outperforms existing models both on qualitative and quantitative performance, and our semi-supervised approach can efficiently synthesize conditional samples under label scarcity. As an example of downstream use of synthetic data, we show improvement in COVID-19 detection from CT scans with conditional synthetic data augmentation. △ Less

Submitted 14 September, 2021; originally announced September 2021.

CDCGen: Cross-Domain Conditional Generation via Normalizing Flows and Adversarial Training

Authors: Hari Prasanna Das, Ryan Tran, Japjot Singh, Yu-Wen Lin, Costas J. Spanos

Abstract: How to generate conditional synthetic data for a domain without utilizing information about its labels/attributes? Our work presents a solution to the above question. We propose a transfer learning-based framework utilizing normalizing flows, coupled with both maximum-likelihood and adversarial training. We model a source domain (labels available) and a target domain (labels unavailable) with indi… ▽ More How to generate conditional synthetic data for a domain without utilizing information about its labels/attributes? Our work presents a solution to the above question. We propose a transfer learning-based framework utilizing normalizing flows, coupled with both maximum-likelihood and adversarial training. We model a source domain (labels available) and a target domain (labels unavailable) with individual normalizing flows, and perform domain alignment to a common latent space using adversarial discriminators. Due to the invertible property of flow models, the map** has exact cycle consistency. We also learn the joint distribution of the data samples and attributes in the source domain by employing an encoder to map attributes to the latent space via adversarial training. During the synthesis phase, given any combination of attributes, our method can generate synthetic samples conditioned on them in the target domain. Empirical studies confirm the effectiveness of our method on benchmarked datasets. We envision our method to be particularly useful for synthetic data generation in label-scarce systems by generating non-trivial augmentations via attribute transformations. These synthetic samples will introduce more entropy into the label-scarce domain than their geometric and photometric transformation counterparts, helpful for robust downstream tasks. △ Less

Submitted 25 August, 2021; originally announced August 2021.

Comments: Workshop on Machine Learning for Data: Automated Creation,Privacy, Bias, In 38th International Conference on Machine Learning (ICML) 2021

Offline-Online Reinforcement Learning for Energy Pricing in Office Demand Response: Lowering Energy and Data Costs

Authors: Doseok Jang, Lucas Spangher, Manan Khattar, Utkarsha Agwan, Selvaprabuh Nadarajah, Costas Spanos

Abstract: Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we examine how offline training can be leveraged to minimize data costs (accelerate convergence) and program impl… ▽ More Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we examine how offline training can be leveraged to minimize data costs (accelerate convergence) and program implementation costs. We present two approaches to doing so: pretraining our model to warm start the experiment with simulated tasks, and using a planning model trained to simulate the real world's rewards to the agent. We present results that demonstrate the utility of offline reinforcement learning to efficient price-setting in the energy demand response problem. △ Less

Submitted 14 August, 2021; originally announced August 2021.

Comments: arXiv admin note: text overlap with arXiv:2104.14670

Gravitational Waves and Primordial Black Holes from Supersymmetric Hybrid Inflation

Authors: Vassilis C. Spanos, Ioanna D. Stamou

Abstract: We study the effect of supergravity corrections due to a linear and a bilinear term in the Kähler potential, in the context of a supersymmetric hybrid inflation model. By appropriate choice of the parameters associated to these terms, we are able to satisfy the main cosmological constraints for the spectral index $n_s$ and the tensor-to-scalar ratio $r$. In addition, this model predicts primordial… ▽ More We study the effect of supergravity corrections due to a linear and a bilinear term in the Kähler potential, in the context of a supersymmetric hybrid inflation model. By appropriate choice of the parameters associated to these terms, we are able to satisfy the main cosmological constraints for the spectral index $n_s$ and the tensor-to-scalar ratio $r$. In addition, this model predicts primordial black hole abundance enough to account for the whole dark matter of the Universe and gravitational wave spectra within the reach of future detection experiments. The predictions of the model can be made compatible to the NANOGrav reported signal, at the cost of significantly lower primordial black hole abundance. △ Less

Submitted 13 November, 2021; v1 submitted 12 August, 2021; originally announced August 2021.

Comments: Latex file, 30 pages, 6 figures, analytical discussion updated, figures updated, references added

Gravitino thermal production

Authors: Helmut Eberl, Ioannis D. Gialamas, Vassilis C. Spanos

Abstract: In this talk we present a new calculation of the gravitino production rate, using its full one-loop corrected thermal self-energy, beyond the hard thermal loop approximation. Gravitino production $2 \to 2$ processes, that are not related to its self-energy have been taken properly into account. Our result, compared to the latest estimation, differs by almost 10%. In addition, we present a handy pa… ▽ More In this talk we present a new calculation of the gravitino production rate, using its full one-loop corrected thermal self-energy, beyond the hard thermal loop approximation. Gravitino production $2 \to 2$ processes, that are not related to its self-energy have been taken properly into account. Our result, compared to the latest estimation, differs by almost 10%. In addition, we present a handy parametrization of our finding, that can be used to calculate the gravitino thermal abundance, as a function of the reheating temperature. △ Less

Submitted 20 July, 2021; originally announced July 2021.

Comments: 9 pages, 6 figures, Contribution to Proceedings of BSM 2021, Egypt, Based on: arXiv:2010.14621

Scale-invariance, dynamically induced Planck scale and inflation in the Palatini formulation

Authors: Ioannis D. Gialamas, Alexandros Karam, Thomas D. Pappas, Antonio Racioppi, Vassilis C. Spanos

Abstract: We present two scale invariant models of inflation in which the addition of quadratic in curvature terms in the usual Einstein-Hilbert action, in the context of Palatini formulation of gravity, manages to reduce the value of the tensor-to-scalar ratio. In both models the Planck scale is dynamically generated via the vacuum expectation value of the scalar fields. We present two scale invariant models of inflation in which the addition of quadratic in curvature terms in the usual Einstein-Hilbert action, in the context of Palatini formulation of gravity, manages to reduce the value of the tensor-to-scalar ratio. In both models the Planck scale is dynamically generated via the vacuum expectation value of the scalar fields. △ Less

Submitted 8 July, 2021; originally announced July 2021.

Comments: 10 pages, 5 figures, Contribution to Proceedings of HEP 2021, Thessaloniki, Greece, Based on: 2006.09124 and 2104.04550. arXiv admin note: text overlap with arXiv:2104.04550

Using Meta Reinforcement Learning to Bridge the Gap between Simulation and Experiment in Energy Demand Response

Authors: Doseok Jang, Lucas Spangher, Manan Khattar, Utkarsha Agwan, Costas Spanos

Abstract: Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we apply a meta-learning architecture to warm start the experiment with simulated tasks, to increase sample effic… ▽ More Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we apply a meta-learning architecture to warm start the experiment with simulated tasks, to increase sample efficiency. We present results that demonstrate a similar a step up in complexity still corresponds with better learning. △ Less

Submitted 17 May, 2021; v1 submitted 29 April, 2021; originally announced April 2021.

Scale-Invariant Quadratic Gravity and Inflation in the Palatini Formalism

Authors: Ioannis D. Gialamas, Alexandros Karam, Thomas D. Pappas, Vassilis C. Spanos

Abstract: In the framework of classical scale invariance, we consider quadratic gravity in the Palatini formalism and investigate the inflationary predictions of the theory. Our model corresponds to a two-field scalar-tensor theory, that involves the Higgs field and an extra scalar field stemming from a gauge $U(1)_X$ extension of the Standard Model, which contains an extra gauge boson and three right-hande… ▽ More In the framework of classical scale invariance, we consider quadratic gravity in the Palatini formalism and investigate the inflationary predictions of the theory. Our model corresponds to a two-field scalar-tensor theory, that involves the Higgs field and an extra scalar field stemming from a gauge $U(1)_X$ extension of the Standard Model, which contains an extra gauge boson and three right-handed neutrinos. Both scalar fields couple nonminimally to gravity and induce the Planck scale dynamically, once they develop vacuum expectation values. By means of the Gildener-Weinberg approach, we describe the inflationary dynamics in terms of a single scalar degree of freedom along the flat direction of the tree-level potential. The one-loop effective potential in the Einstein frame exhibits plateaus on both sides of the minimum and thus the model can accommodate both small and large field inflation. The inflationary predictions of the model are found to comply with the latest bounds set by the Planck collaboration for a wide range of parameters and the effect of the quadratic in curvature terms is to reduce the value of the tensor-to-scalar ratio. △ Less

Submitted 8 July, 2021; v1 submitted 9 April, 2021; originally announced April 2021.

Comments: matches version to be published in PRD

Journal ref: Phys. Rev. D 104, 023521 (2021)

Selling Renewable Utilization Service to Consumers via Cloud Energy Storage

Authors: Yu Yang, Utkarsha Agwan, Guoqiang Hu, Costas J. Spanos

Abstract: This paper proposes a cloud energy storage (CES) model for enabling local renewable integration of building consumers (BCs). Different from most existing third-party based ES sharing models that the energy storage operator (ESO) gains profit by leasing energy or power capacity, our CES model allows the ESO to sell renewable utilization service (RUS) to its consumers, i.e., the total amount of loca… ▽ More This paper proposes a cloud energy storage (CES) model for enabling local renewable integration of building consumers (BCs). Different from most existing third-party based ES sharing models that the energy storage operator (ESO) gains profit by leasing energy or power capacity, our CES model allows the ESO to sell renewable utilization service (RUS) to its consumers, i.e., the total amount of local renewable generation shifted to supply their demand. Notably, we propose a quadratic price model for the ESO charging its consumers by their requested RUS and formulate their interactions as a Stackelberg game, which admits an equilibrium. We prove the CES model outperforms individual ES (IES) model in social welfare. Besides, we study the performance of the CES model compared with the IES model and an existing ES sharing model (referring to VES model) via case studies. We demonstrate the CES model can provide 2-4 times profit to the ESO than the VES model. Meanwhile, higher cost reduction for the BCs are secured by the CES model. Moreover, we show the CES model can achieve near social optima and high ES efficiency (i.e., utilization) which are not provided by the other ES models. △ Less

Submitted 24 January, 2021; v1 submitted 29 December, 2020; originally announced December 2020.

Comments: 11 pages, 8 figures

Optimal Sharing and Fair Cost Allocation of Community Energy Storage

Authors: Yu Yang, Guoqiang Hu, Costas J. Spanos

Abstract: This paper studies an energy storage (ES) sharing model which is cooperatively invested by multiple buildings for harnessing on-site renewable utilization and grid price arbitrage. To maximize the economic benefits, we jointly consider the ES sizing, operation, and cost allocation via a coalition game formulation. Particularly, we study a fair ex-post cost allocation based on nucleolus which addre… ▽ More This paper studies an energy storage (ES) sharing model which is cooperatively invested by multiple buildings for harnessing on-site renewable utilization and grid price arbitrage. To maximize the economic benefits, we jointly consider the ES sizing, operation, and cost allocation via a coalition game formulation. Particularly, we study a fair ex-post cost allocation based on nucleolus which addresses fairness by minimizing the minimal dissatisfaction of all the players. To overcome the exponential computation burden caused by the implicit characteristic function, we employ a constraint generation technique to gradually approach the unique nucleolus by leveraging the sparse problem structure. We demonstrate both the fairness and computational efficiency of the method through case studies, which are not provided by the existing Shapley approach or proportional method. Particularly, only a small fraction of characteristic function (less than 1% for 20 buildings) is required to achieve the cost allocation versus the exponential information required by Shapley approach. Though there exists a minor increase of computation over the proportional method, the proposed method can ensure fairness while the latter fails in some cases. Further, we demonstrate both the building-wise and community-wise economic benefits are enhanced with the ES sharing model over the individual ES (IES) model. Accordingly, the overall value of ES is considerably improved (about 1.83 times). △ Less

Submitted 5 April, 2021; v1 submitted 29 October, 2020; originally announced October 2020.

Comments: 11 pages, 3 figures

Gravitino thermal production revisited

Authors: Helmut Eberl, Ioannis D. Gialamas, Vassilis C. Spanos

Abstract: We calculate the gravitino production rate, computing its one-loop thermal self-energy. Gravitino production processes that do not result through thermal cuts of its self-energy, have been identified and taken into account. Correcting analytical errors and numerical approximations in the previous calculations, we present our result. This deviates from the latest estimation by almost 10%. More impo… ▽ More We calculate the gravitino production rate, computing its one-loop thermal self-energy. Gravitino production processes that do not result through thermal cuts of its self-energy, have been identified and taken into account. Correcting analytical errors and numerical approximations in the previous calculations, we present our result. This deviates from the latest estimation by almost 10%. More importantly, we provide a convenient formula, for calculating the gravitino production rate and its thermal abundance, as a function of the reheating temperature of the Universe. △ Less

Submitted 24 April, 2021; v1 submitted 27 October, 2020; originally announced October 2020.

Comments: 6 pages, 3 figures, references added, matches published version

Journal ref: Phys. Rev. D 103, 075025 (2021)

Primordial Black Holes from No-Scale Supergravity

Authors: Dimitri V. Nanopoulos, Vassilis C. Spanos, Ioanna D. Stamou

Abstract: We calculate the primordial black hole abundance in the context of a Wess-Zumino type no-scale supergravity model. We modify the Kähler potential, by adding an extra exponential term. Using just one parameter in the context of this model, we are able to satisfy the Planck cosmological constraints for the spectral index $n_s$, the tensor-to-scalar ratio $r$, and to produce up to $\sim 20\%$ of the… ▽ More We calculate the primordial black hole abundance in the context of a Wess-Zumino type no-scale supergravity model. We modify the Kähler potential, by adding an extra exponential term. Using just one parameter in the context of this model, we are able to satisfy the Planck cosmological constraints for the spectral index $n_s$, the tensor-to-scalar ratio $r$, and to produce up to $\sim 20\%$ of the dark matter of the Universe in the form of primordial black holes. △ Less

Submitted 29 October, 2020; v1 submitted 4 August, 2020; originally announced August 2020.

Comments: 23 pages, latex, 8 figures, published version in PRD

Report number: ACT-04-20, MI-TH-2015

Journal ref: Phys. Rev. D 102, 083536 (2020)

Distributed Control of Multi-zone HVAC Systems Considering Indoor Air Quality

Authors: Yu Yang, Seshadhri Srinivasan, Guoqiang Hu, Costas J. Spanos

Abstract: This paper studies a scalable control method for multi-zone heating, ventilation and air-conditioning (HVAC) systems to optimize the energy cost for maintaining thermal comfort and indoor air quality (IAQ) (represented by CO2) simultaneously. This problem is computationally challenging due to the complex system dynamics, various spatial and temporal couplings as well as multiple control variables… ▽ More This paper studies a scalable control method for multi-zone heating, ventilation and air-conditioning (HVAC) systems to optimize the energy cost for maintaining thermal comfort and indoor air quality (IAQ) (represented by CO2) simultaneously. This problem is computationally challenging due to the complex system dynamics, various spatial and temporal couplings as well as multiple control variables to be coordinated. To address the challenges, we propose a two-level distributed method (TLDM) with a upper level and lower level control integrated. The upper level computes zone mass flow rates for maintaining zone thermal comfort with minimal energy cost, and then the lower level strategically regulates zone mass flow rates and the ventilation rate to achieve IAQ while preserving the near energy saving performance of upper level. As both the upper and lower level computation are deployed in a distributed manner, the proposed method is scalable and computationally efficient. The near-optimal performance of the method in energy cost saving is demonstrated through comparison with the centralized method. In addition, the comparisons with the existing distributed method show that our method can provide IAQ with only little increase of energy cost while the latter fails. Moreover, we demonstrate our method outperforms the demand controlled ventilation strategies (DCVs) for IAQ management with about 8-10% energy cost reduction. △ Less

Submitted 4 January, 2021; v1 submitted 17 March, 2020; originally announced March 2020.

Comments: 12 pages, 12 figures

smartSDH: An Experimental Study of Mechanism Based Building Control

Authors: Ioannis C. Konstantakopoulos, Kristy A. Hamilton, Yashaswini Murthy, Tanya Veeravalli, Costas Spanos, Roy Dong

Abstract: As Internet of Things (IoT) technologies are increasingly being deployed, situations frequently arise where multiple stakeholders must reconcile preferences to control a shared resource. We perform a 5-month long experiment dubbed 'smartSDH' (carried out in 27 employees' office space) where users report their preferences for the brightness of overhead lighting. smartSDH implements a modified Vickr… ▽ More As Internet of Things (IoT) technologies are increasingly being deployed, situations frequently arise where multiple stakeholders must reconcile preferences to control a shared resource. We perform a 5-month long experiment dubbed 'smartSDH' (carried out in 27 employees' office space) where users report their preferences for the brightness of overhead lighting. smartSDH implements a modified Vickrey-Clarke-Groves (VCG) mechanism; assuming users are rational, it incentivizes truthful reporting, implements the socially desirable outcome, and compensates participants to ensure higher payoffs under smartSDH when compared with the default outside option(i.e., the option chosen in the absence of such a mechanism). smartSDH assesses the feasibility of the VCG mechanism in the context of smart building control and evaluated smartSDH's effect using metrics such as light level satisfaction, incentive satisfaction, and energy consumption. Although previous studies on the theoretical aspects of the mechanism indicate user satisfaction, our experiments indicate quite the contrary. We found that the participants were significantly less satisfied with light brightness and incentives determined by the VCG mechanism over time. These data suggest the need for more realistic behavioral models to design IoT technologies and highlights difficulties in estimating preferences from observable external factors such as atmospheric conditions. △ Less

Submitted 26 June, 2021; v1 submitted 8 January, 2020; originally announced January 2020.

Comments: 11 pages, 4 figures

A Proximal Linearization-based Decentralized Method for Nonconvex Problems with Nonlinear Constraints

Authors: Yu Yang, Guoqiang Hu, Costas J. Spanos

Abstract: Decentralized optimization for non-convex problems are now demanding by many emerging applications (e.g., smart grids, smart building, etc.). Though dramatic progress has been achieved in convex problems, the results for non-convex cases, especially with non-linear constraints, are still largely unexplored. This is mainly due to the challenges imposed by the non-linearity and non-convexity, which… ▽ More Decentralized optimization for non-convex problems are now demanding by many emerging applications (e.g., smart grids, smart building, etc.). Though dramatic progress has been achieved in convex problems, the results for non-convex cases, especially with non-linear constraints, are still largely unexplored. This is mainly due to the challenges imposed by the non-linearity and non-convexity, which makes establishing the convergence conditions bewildered. This paper investigates decentralized optimization for a class of structured non-convex problems characterized by: (i) nonconvex global objective function (possibly nonsmooth) and (ii) coupled nonlinear constraints and local bounded convex constraints w.r.t. the agents. For such problems, a decentralized approach called Proximal Linearizationbased Decentralized Method (PLDM) is proposed. Different from the traditional (augmented) Lagrangian-based methods which usually require the exact (local) optima at each iteration, the proposed method leverages a proximal linearization-based technique to update the decision variables iteratively, which makes it computationally efficient and viable for the non-linear cases. Under some standard conditions, the PLDM global convergence and local convergence rate to the epsilon-critical points are studied based on the Kurdyka-Lojasiewicz property which holds for most analytical functions. Finally, the performance and efficacy of the method are illustrated through a numerical example and an application to multi-zone heating, ventilation and air-conditioning (HVAC) control. △ Less

Submitted 27 August, 2022; v1 submitted 3 January, 2020; originally announced January 2020.

Comments: 17 pages, 5 figures

Stochastic Optimal Control of HVAC system for Energy-efficient Buildings

Authors: Yu Yang, Guoqiang Hu, Costas J. Spanos

Abstract: The heating, ventilation and air-conditioning (HVAC) system accounts for substantial energy use in buildings, whereas a large group of occupants are still not actually feeling comfortable staying inside. This poses the issue of develo** energy-efficient HVAC control, i.e., reduce energy use (cost) while simultaneously enhancing human comfort. This paper pursues the objective and studies the stoc… ▽ More The heating, ventilation and air-conditioning (HVAC) system accounts for substantial energy use in buildings, whereas a large group of occupants are still not actually feeling comfortable staying inside. This poses the issue of develo** energy-efficient HVAC control, i.e., reduce energy use (cost) while simultaneously enhancing human comfort. This paper pursues the objective and studies the stochastic optimal HVAC control subject to uncertain thermal demand (i.e., the weather and occupancy etc). Particularly, we involve the elaborate predicted mean vote (PMV) thermal comfort model in the optimization. The problem is computationally challenging due to the non-linear and non-analytical constraints imposed by the system dynamics and PMV model. We make the following contributions to address it. First, we formulate the problem as a Markov decision process (MDP) which is a desirable modeling technique capable of handling the complexities. Second, we propose a gradient-based learning (GB-L) method for progressively learning a stochastic control policy off-line and store it for on-line execution. Third, we prove the learning method converge to the optimal policies theoretically, and its performance (i.e., energy cost, thermal comfort and on-line computation) for HVAC control via simulations. The comparisons with the existing model predictive control based relaxation (MPC-R) method which is assumed with accurate future information and supposed to provide the near-optimal bounds, show that though there exists some performance loss in energy cost reduction (i.e., 6.5%), the proposed method can enable efficient on-line implementation (less than 1 second) and provide high probability of thermal comfort under uncertainties. △ Less

Submitted 4 February, 2021; v1 submitted 3 November, 2019; originally announced November 2019.

Comments: 11 pages, 10 figures

Design, Benchmarking and Explainability Analysis of a Game-Theoretic Framework towards Energy Efficiency in Smart Infrastructure

Authors: Ioannis C. Konstantakopoulos, Hari Prasanna Das, Andrew R. Barkan, Shiying He, Tanya Veeravalli, Huihan Liu, Aummul Baneen Manasawala, Yu-Wen Lin, Costas J. Spanos

Abstract: In this paper, we propose a gamification approach as a novel framework for smart building infrastructure with the goal of motivating human occupants to reconsider personal energy usage and to have positive effects on their environment. Human interaction in the context of cyber-physical systems is a core component and consideration in the implementation of any smart building technology. Research ha… ▽ More In this paper, we propose a gamification approach as a novel framework for smart building infrastructure with the goal of motivating human occupants to reconsider personal energy usage and to have positive effects on their environment. Human interaction in the context of cyber-physical systems is a core component and consideration in the implementation of any smart building technology. Research has shown that the adoption of human-centric building services and amenities leads to improvements in the operational efficiency of these cyber-physical systems directed towards controlling building energy usage. We introduce a strategy in form of a game-theoretic framework that incorporates humans-in-the-loop modeling by creating an interface to allow building managers to interact with occupants and potentially incentivize energy efficient behavior. Prior works on game theoretic analysis typically rely on the assumption that the utility function of each individual agent is known a priori. Instead, we propose novel utility learning framework for benchmarking that employs robust estimations of occupant actions towards energy efficiency. To improve forecasting performance, we extend the utility learning scheme by leveraging deep bi-directional recurrent neural networks. Using the proposed methods on data gathered from occupant actions for resources such as room lighting, we forecast patterns of energy resource usage to demonstrate the prediction performance of the methods. The results of our study show that we can achieve a highly accurate representation of the ground truth for occupant energy resource usage. We also demonstrate the explainable nature on human decision making towards energy usage inherent in the dataset using graphical lasso and granger causality algorithms. Finally, we open source the de-identified, high-dimensional data pertaining to the energy game-theoretic framework. △ Less

Submitted 16 October, 2019; originally announced October 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1809.05142, arXiv:1810.10533

A Novel Graphical Lasso based approach towards Segmentation Analysis in Energy Game-Theoretic Frameworks

Authors: Hari Prasanna Das, Ioannis C. Konstantakopoulos, Aummul Baneen Manasawala, Tanya Veeravalli, Huihan Liu, Costas J. Spanos

Abstract: Energy game-theoretic frameworks have emerged to be a successful strategy to encourage energy efficient behavior in large scale by leveraging human-in-the-loop strategy. A number of such frameworks have been introduced over the years which formulate the energy saving process as a competitive game with appropriate incentives for energy efficient players. However, prior works involve an incentive de… ▽ More Energy game-theoretic frameworks have emerged to be a successful strategy to encourage energy efficient behavior in large scale by leveraging human-in-the-loop strategy. A number of such frameworks have been introduced over the years which formulate the energy saving process as a competitive game with appropriate incentives for energy efficient players. However, prior works involve an incentive design mechanism which is dependent on knowledge of utility functions for all the players in the game, which is hard to compute especially when the number of players is high, common in energy game-theoretic frameworks. Our research proposes that the utilities of players in such a framework can be grouped together to a relatively small number of clusters, and the clusters can then be targeted with tailored incentives. The key to above segmentation analysis is to learn the features leading to human decision making towards energy usage in competitive environments. We propose a novel graphical lasso based approach to perform such segmentation, by studying the feature correlations in a real-world energy social game dataset. To further improve the explainability of the model, we perform causality study using grangers causality. Proposed segmentation analysis results in characteristic clusters demonstrating different energy usage behaviors. We also present avenues to implement intelligent incentive design using proposed segmentation method. △ Less

Submitted 5 October, 2019; originally announced October 2019.

Comments: Proceedings of the Special Session on Machine Learning in Energy Application, International Conference on Machine Learning and Applications (ICMLA) 2019. arXiv admin note: text overlap with arXiv:1810.10533

Journal ref: 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA)

Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms

Authors: Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J. Spanos, Dawn Song

Abstract: Given a data set $\mathcal{D}$ containing millions of data points and a data consumer who is willing to pay for \$$X$ to train a machine learning (ML) model over $\mathcal{D}$, how should we distribute this \$$X$ to each data point to reflect its "value"? In this paper, we define the "relative value of data" via the Shapley value, as it uniquely possesses properties with appealing real-world inter… ▽ More Given a data set $\mathcal{D}$ containing millions of data points and a data consumer who is willing to pay for \$$X$ to train a machine learning (ML) model over $\mathcal{D}$, how should we distribute this \$$X$ to each data point to reflect its "value"? In this paper, we define the "relative value of data" via the Shapley value, as it uniquely possesses properties with appealing real-world interpretations, such as fairness, rationality and decentralizability. For general, bounded utility functions, the Shapley value is known to be challenging to compute: to get Shapley values for all $N$ data points, it requires $O(2^N)$ model evaluations for exact computation and $O(N\log N)$ for $(ε, δ)$-approximation. In this paper, we focus on one popular family of ML models relying on $K$-nearest neighbors ($K$NN). The most surprising result is that for unweighted $K$NN classifiers and regressors, the Shapley value of all $N$ data points can be computed, exactly, in $O(N\log N)$ time -- an exponential improvement on computational complexity! Moreover, for $(ε, δ)$-approximation, we are able to develop an algorithm based on Locality Sensitive Hashing (LSH) with only sublinear complexity $O(N^{h(ε,K)}\log N)$ when $ε$ is not too small and $K$ is not too large. We empirically evaluate our algorithms on up to $10$ million data points and even our exact algorithm is up to three orders of magnitude faster than the baseline approximation algorithm. The LSH-based approximation algorithm can accelerate the value calculation process even further. We then extend our algorithms to other scenarios such as (1) weighed $K$NN classifiers, (2) different data points are clustered by different data curators, and (3) there are data analysts providing computation who also requires proper valuation. △ Less

Submitted 29 March, 2020; v1 submitted 22 August, 2019; originally announced August 2019.

Journal ref: PVLDB, 12(11): 1610-1623, 2019

Likelihood Contribution based Multi-scale Architecture for Generative Flows

Authors: Hari Prasanna Das, Pieter Abbeel, Costas J. Spanos

Abstract: Deep generative modeling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process. However, flow models suffer from the challenge of having high dimensional latent space, the same in dimension as the input space. An effective solution to the above challenge as proposed by Dinh et al. (2016) is a multi-scale architecture,… ▽ More Deep generative modeling using flows has gained popularity owing to the tractable exact log-likelihood estimation with efficient training and synthesis process. However, flow models suffer from the challenge of having high dimensional latent space, the same in dimension as the input space. An effective solution to the above challenge as proposed by Dinh et al. (2016) is a multi-scale architecture, which is based on iterative early factorization of a part of the total dimensions at regular intervals. Prior works on generative flow models involving a multi-scale architecture perform the dimension factorization based on static masking. We propose a novel multi-scale architecture that performs data-dependent factorization to decide which dimensions should pass through more flow layers. To facilitate the same, we introduce a heuristic based on the contribution of each dimension to the total log-likelihood which encodes the importance of the dimensions. Our proposed heuristic is readily obtained as part of the flow training process, enabling the versatile implementation of our likelihood contribution based multi-scale architecture for generic flow models. We present such implementations for several state-of-the-art flow models and demonstrate improvements in log-likelihood score and sampling quality on standard image benchmarks. We also conduct ablation studies to compare the proposed method with other options for dimension factorization. △ Less

Submitted 27 January, 2022; v1 submitted 5 August, 2019; originally announced August 2019.

HVAC Energy Cost Optimization for a Multi-zone Building via a Decentralized Approach

Authors: Yu Yang, Guoqiang Hu, Costas J. Spanos

Abstract: It has been well acknowledged that buildings account for a large proportion of the world's energy consumption. However, the energy use of buildings, especially the heating, ventilation and air-conditioning (HVAC), is far from being efficient. There still exists a dramatic potential to save energy through improving building energy efficiency. Therefore, this paper studies the control of HVAC system… ▽ More It has been well acknowledged that buildings account for a large proportion of the world's energy consumption. However, the energy use of buildings, especially the heating, ventilation and air-conditioning (HVAC), is far from being efficient. There still exists a dramatic potential to save energy through improving building energy efficiency. Therefore, this paper studies the control of HVAC system for multi-zone buildings with the objective to reduce energy consumption cost while satisfying thermal comfort. In particular, the thermal couplings due to the heat transfer between the adjacent zones are incorporated in the optimization. Considering that a centralized method is generally computationally prohibitive for large buildings, an efficient decentralized approach is developed, based on the Accelerated Distributed Augmented Lagrangian (ADAL) method [1]. To evaluate the performance of the proposed method, we first compare it with a centralized method, in which the optimal solution of a small-scale problem can be obtained. We find that this decentralized approach can almost approach the optimal solution of the problem. Further, this decentralized approach is compared with the Distributed Token-Based Scheduling Strategy (DTBSS) [2]. The numeric results reveal that when the number of zones is relatively small (less than 20), the two decentralized methods can achieve a comparable performance regarding the cost of the HVAC system. However, with an increase of the number of zones in the building, the proposed decentralized approach demonstrates better performance with a considerable reduction of the total cost. Moreover, the decentralized approach proposed in this paper demonstrate better scalability with less average computation required. △ Less

Submitted 21 May, 2019; originally announced May 2019.

Comments: 13 pages, 8 figures

Towards Efficient Data Valuation Based on the Shapley Value

Authors: Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gurel, Bo Li, Ce Zhang, Dawn Song, Costas Spanos

Abstract: "How much is my data worth?" is an increasingly common question posed by organizations and individuals alike. An answer to this question could allow, for instance, fairly distributing profits among multiple data contributors and determining prospective compensation when data breaches happen. In this paper, we study the problem of data valuation by utilizing the Shapley value, a popular notion of v… ▽ More "How much is my data worth?" is an increasingly common question posed by organizations and individuals alike. An answer to this question could allow, for instance, fairly distributing profits among multiple data contributors and determining prospective compensation when data breaches happen. In this paper, we study the problem of data valuation by utilizing the Shapley value, a popular notion of value which originated in coopoerative game theory. The Shapley value defines a unique payoff scheme that satisfies many desiderata for the notion of data value. However, the Shapley value often requires exponential time to compute. To meet this challenge, we propose a repertoire of efficient algorithms for approximating the Shapley value. We also demonstrate the value of each training instance for various benchmark datasets. △ Less

Submitted 16 August, 2020; v1 submitted 26 February, 2019; originally announced February 2019.

Segmentation Analysis in Human Centric Cyber-Physical Systems using Graphical Lasso

Authors: Hari Prasanna Das, Ioannis C. Konstantakopoulos, Aummul Baneen Manasawala, Tanya Veeravalli, Huihan Liu, Costas J. Spanos

Abstract: A generalized gamification framework is introduced as a form of smart infrastructure with potential to improve sustainability and energy efficiency by leveraging humans-in-the-loop strategy. The proposed framework enables a Human-Centric Cyber-Physical System using an interface to allow building managers to interact with occupants. The interface is designed for occupant engagement-integration supp… ▽ More A generalized gamification framework is introduced as a form of smart infrastructure with potential to improve sustainability and energy efficiency by leveraging humans-in-the-loop strategy. The proposed framework enables a Human-Centric Cyber-Physical System using an interface to allow building managers to interact with occupants. The interface is designed for occupant engagement-integration supporting learning of their preferences over resources in addition to understanding how preferences change as a function of external stimuli such as physical control, time or incentives. Towards intelligent and autonomous incentive design, a noble statistical learning algorithm performing occupants energy usage behavior segmentation is proposed. We apply the proposed algorithm, Graphical Lasso, on energy resource usage data by the occupants to obtain feature correlations--dependencies. Segmentation analysis results in characteristic clusters demonstrating different energy usage behaviors. The features--factors characterizing human decision-making are made explainable. △ Less

Submitted 16 January, 2019; v1 submitted 24 October, 2018; originally announced October 2018.

Comments: arXiv admin note: substantial text overlap with arXiv:1809.05142

One Bit Matters: Understanding Adversarial Examples as the Abuse of Redundancy

Authors: **gkang Wang, Ruoxi Jia, Gerald Friedland, Bo Li, Costas Spanos

Abstract: Despite the great success achieved in machine learning (ML), adversarial examples have caused concerns with regards to its trustworthiness: A small perturbation of an input results in an arbitrary failure of an otherwise seemingly well-trained ML model. While studies are being conducted to discover the intrinsic properties of adversarial examples, such as their transferability and universality, th… ▽ More Despite the great success achieved in machine learning (ML), adversarial examples have caused concerns with regards to its trustworthiness: A small perturbation of an input results in an arbitrary failure of an otherwise seemingly well-trained ML model. While studies are being conducted to discover the intrinsic properties of adversarial examples, such as their transferability and universality, there is insufficient theoretic analysis to help understand the phenomenon in a way that can influence the design process of ML experiments. In this paper, we deduce an information-theoretic model which explains adversarial attacks as the abuse of feature redundancies in ML algorithms. We prove that feature redundancy is a necessary condition for the existence of adversarial examples. Our model helps to explain some major questions raised in many anecdotal studies on adversarial examples. Our theory is backed up by empirical measurements of the information content of benign and adversarial examples on both image and text datasets. Our measurements show that typical adversarial examples introduce just enough redundancy to overflow the decision making of an ML model trained on corresponding benign examples. We conclude with actionable recommendations to improve the robustness of machine learners against adversarial examples. △ Less

Submitted 23 October, 2018; originally announced October 2018.

A Deep Learning and Gamification Approach to Energy Conservation at Nanyang Technological University

Authors: Ioannis C. Konstantakopoulos, Andrew R. Barkan, Shiying He, Tanya Veeravalli, Huihan Liu, Costas Spanos

Abstract: The implementation of smart building technology in the form of smart infrastructure applications has great potential to improve sustainability and energy efficiency by leveraging humans-in-the-loop strategy. However, human preference in regard to living conditions is usually unknown and heterogeneous in its manifestation as control inputs to a building. Furthermore, the occupants of a building typ… ▽ More The implementation of smart building technology in the form of smart infrastructure applications has great potential to improve sustainability and energy efficiency by leveraging humans-in-the-loop strategy. However, human preference in regard to living conditions is usually unknown and heterogeneous in its manifestation as control inputs to a building. Furthermore, the occupants of a building typically lack the independent motivation necessary to contribute to and play a key role in the control of smart building infrastructure. Moreover, true human actions and their integration with sensing/actuation platforms remains unknown to the decision maker tasked with improving operational efficiency. By modeling user interaction as a sequential discrete game between non-cooperative players, we introduce a gamification approach for supporting user engagement and integration in a human-centric cyber-physical system. We propose the design and implementation of a large-scale network game with the goal of improving the energy efficiency of a building through the utilization of cutting-edge Internet of Things (IoT) sensors and cyber-physical systems sensing/actuation platforms. A benchmark utility learning framework that employs robust estimations for classical discrete choice models provided for the derived high dimensional imbalanced data. To improve forecasting performance, we extend the benchmark utility learning scheme by leveraging Deep Learning end-to-end training with Deep bi-directional Recurrent Neural Networks. We apply the proposed methods to high dimensional data from a social game experiment designed to encourage energy efficient behavior among smart building occupants in Nanyang Technological University (NTU) residential housing. Using occupant-retrieved actions for resources such as lighting and A/C, we simulate the game defined by the estimated utility functions. △ Less

Submitted 25 September, 2018; v1 submitted 13 September, 2018; originally announced September 2018.

Comments: 16 double pages, shorter version submitted to Applied Energy Journal

Likelihood Analysis of the pMSSM11 in Light of LHC 13-TeV Data

Authors: E. Bagnaschi, K. Sakurai, M. Borsato, O. Buchmueller, M. Citron, J. C. Costa, A. De Roeck, M. J. Dolan, J. R. Ellis, H. Flächer, S. Heinemeyer, M. Lucio, D. Martínez Santos, K. A. Olive, A. Richards, V. C. Spanos, I. Suárez Fernández, G. Weiglein

Abstract: We use MasterCode to perform a frequentist analysis of the constraints on a phenomenological MSSM model with 11 parameters, the pMSSM11, including constraints from ~ 36/fb of LHC data at 13 TeV and PICO, XENON1T and PandaX-II searches for dark matter scattering, as well as previous accelerator and astrophysical measurements, presenting fits both with and without the $(g-2)_μ$ constraint. The pMSSM… ▽ More We use MasterCode to perform a frequentist analysis of the constraints on a phenomenological MSSM model with 11 parameters, the pMSSM11, including constraints from ~ 36/fb of LHC data at 13 TeV and PICO, XENON1T and PandaX-II searches for dark matter scattering, as well as previous accelerator and astrophysical measurements, presenting fits both with and without the $(g-2)_μ$ constraint. The pMSSM11 is specified by the following parameters: 3 gaugino masses $M_{1,2,3}$, a common mass for the first-and second-generation squarks $m_{\tilde{q}}$ and a distinct third-generation squark mass $m_{\tilde{q}_3}$, a common mass for the first-and second-generation sleptons $m_{\tilde l}$ and a distinct third-generation slepton mass $m_{\tilde τ}$, a common trilinear mixing parameter $A$, the Higgs mixing parameter $μ$, the pseudoscalar Higgs mass $M_A$ and $\tanβ$. In the fit including $(g-2)_μ$, a Bino-like $\tildeχ^0_1$ is preferred, whereas a Higgsino-like $\tilde χ^0_1$ is favoured when the $(g-2)_μ$ constraint is dropped. We identify the mechanisms that operate in different regions of the pMSSM11 parameter space to bring the relic density of the lightest neutralino, $\tildeχ^0_1$, into the range indicated by cosmological data. In the fit including $(g-2)_μ$, coannihilations with $\tilde χ^0_2$ and the Wino-like $\tildeχ^{\pm}_1$ or with nearly-degenerate first- and second-generation sleptons are favoured, whereas coannihilations with the $\tilde χ^0_2$ and the Higgsino-like $\tildeχ^{\pm}_1$ or with first- and second-generation squarks may be important when the $(g-2)_μ$ constraint is dropped. Prospects remain for discovering strongly-interacting sparticles at the LHC as well as for discovering electroweakly-interacting sparticles at a future linear $e^+ e^-$ collider such as the ILC or CLIC. △ Less

Submitted 1 May, 2018; v1 submitted 30 October, 2017; originally announced October 2017.

Comments: 48 pages, 24 figures, 65 plots, 6 tables; version published on EPJC

Report number: KCL-PH-TH/2017-22, CERN-PH-TH/2017-087, DESY 17-059, IFT-UAM/CSIC-17-035, FTPI-MINN-17/17, UMN-TH-3701/17

Journal ref: Eur.Phys.J. C78 (2018) no.3, 256

A Robust Utility Learning Framework via Inverse Optimization

Authors: Ioannis C. Konstantakopoulos, Lillian J. Ratliff, Ming **, S. Shankar Sastry, Costas Spanos

Abstract: In many smart infrastructure applications flexibility in achieving sustainability goals can be gained by engaging end-users. However, these users often have heterogeneous preferences that are unknown to the decision-maker tasked with improving operational efficiency. Modeling user interaction as a continuous game between non-cooperative players, we propose a robust parametric utility learning fram… ▽ More In many smart infrastructure applications flexibility in achieving sustainability goals can be gained by engaging end-users. However, these users often have heterogeneous preferences that are unknown to the decision-maker tasked with improving operational efficiency. Modeling user interaction as a continuous game between non-cooperative players, we propose a robust parametric utility learning framework that employs constrained feasible generalized least squares estimation with heteroskedastic inference. To improve forecasting performance, we extend the robust utility learning scheme by employing bootstrap** with bagging, bum**, and gradient boosting ensemble methods. Moreover, we estimate the noise covariance which provides approximated correlations between players which we leverage to develop a novel correlated utility learning framework. We apply the proposed methods both to a toy example arising from Bertrand-Nash competition between two firms as well as to data from a social game experiment designed to encourage energy efficient behavior amongst smart building occupants. Using occupant voting data for shared resources such as lighting, we simulate the game defined by the estimated utility functions to demonstrate the performance of the proposed methods. △ Less

Submitted 25 April, 2017; originally announced April 2017.

Comments: accepted for publication in IEEE Transactions on Control Systems Technology

Effective Theory for Electroweak Doublet Dark Matter

Authors: Athanasios Dedes, Dimitrios Karamitros, Vassilis C. Spanos

Abstract: We perform a detailed study of an effective field theory which includes the Standard Model particle content extended by a pair of Weyl fermionic SU(2)-doublets with opposite hypercharges. A discrete symmetry guarantees that a linear combination of the doublet components is stable and can act as a candidate particle for Dark Matter. The dark sector fermions interact with the Higgs and gauge bosons… ▽ More We perform a detailed study of an effective field theory which includes the Standard Model particle content extended by a pair of Weyl fermionic SU(2)-doublets with opposite hypercharges. A discrete symmetry guarantees that a linear combination of the doublet components is stable and can act as a candidate particle for Dark Matter. The dark sector fermions interact with the Higgs and gauge bosons through renormalizable $d=4$ operators, and non-renormalizable $d=5$ operators that appear after integrating out extra degrees of freedom above the TeV scale. We study collider, cosmological and astrophysical probes for this effective theory of Dark Matter. We find that a WIMP with a mass nearby to the electroweak scale, and thus observable at LHC, is consistent with collider and astrophysical data only when fairly large magnetic dipole moment transition operators with the gauge bosons exist, together with moderate Yukawa interactions. △ Less

Submitted 27 November, 2016; v1 submitted 18 July, 2016; originally announced July 2016.

Comments: 46 pages, v2: references and related comments added, v3: references added, discussion on LHC searches expanded, a discussion on the Sommerfeld enhancement added, matches published version

Journal ref: Phys. Rev. D 94, 095008 (2016)

Privacy-Enhanced Architecture for Occupancy-based HVAC Control

Authors: Ruoxi Jia, Roy Dong, S. Shankar Sastry, Costas J. Spanos

Abstract: Large-scale sensing and actuation infrastructures have allowed buildings to achieve significant energy savings; at the same time, these technologies introduce significant privacy risks that must be addressed. In this paper, we present a framework for modeling the trade-off between improved control performance and increased privacy risks due to occupancy sensing. More specifically, we consider occu… ▽ More Large-scale sensing and actuation infrastructures have allowed buildings to achieve significant energy savings; at the same time, these technologies introduce significant privacy risks that must be addressed. In this paper, we present a framework for modeling the trade-off between improved control performance and increased privacy risks due to occupancy sensing. More specifically, we consider occupancy-based HVAC control as the control objective and the location traces of individual occupants as the private variables. Previous studies have shown that individual location information can be inferred from occupancy measurements. To ensure privacy, we design an architecture that distorts the occupancy data in order to hide individual occupant location information while maintaining HVAC performance. Using mutual information between the individual's location trace and the reported occupancy measurement as a privacy metric, we are able to optimally design a scheme to minimize privacy risk subject to a control performance guarantee. We evaluate our framework using real-world occupancy data: first, we verify that our privacy metric accurately assesses the adversary's ability to infer private variables from the distorted sensor measurements; then, we show that control performance is maintained through simulations of building operations using these distorted occupancy readings. △ Less

Submitted 11 July, 2016; originally announced July 2016.

Inverse Reinforcement Learning via Deep Gaussian Process

Authors: Ming **, Andreas Damianou, Pieter Abbeel, Costas Spanos

Abstract: We propose a new approach to inverse reinforcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward structures with few demonstrations. Our model stacks multiple latent GP layers to learn abstract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. Incorpo… ▽ More We propose a new approach to inverse reinforcement learning (IRL) based on the deep Gaussian process (deep GP) model, which is capable of learning complicated reward structures with few demonstrations. Our model stacks multiple latent GP layers to learn abstract representations of the state feature space, which is linked to the demonstrations through the Maximum Entropy learning framework. Incorporating the IRL engine into the nonlinear latent structure renders existing deep GP inference approaches intractable. To tackle this, we develop a non-standard variational approximation framework which extends previous inference schemes. This allows for approximate Bayesian treatment of the feature space and guards against overfitting. Carrying out representation and inverse reinforcement learning simultaneously within our model outperforms state-of-the-art approaches, as we demonstrate with experiments on standard benchmarks ("object world","highway driving") and a new benchmark ("binary world"). △ Less

Submitted 4 May, 2017; v1 submitted 25 December, 2015; originally announced December 2015.

GravitinoPack and late decays involving gravitinos

Authors: Helmut Eberl, Vassilis C. Spanos

Abstract: In this talk, we present the package GravitinoPack that calculates decays of unstable supersymmetric particles, involving gravitinos in the final or initial state. If the gravitino is the dark matter particle and therefore stable, the package calculates the decays of the lightest neutralino, and the lighter stau or stop NLSP into the gravitino LSP and one or two Standard Model particles. On the ot… ▽ More In this talk, we present the package GravitinoPack that calculates decays of unstable supersymmetric particles, involving gravitinos in the final or initial state. If the gravitino is the dark matter particle and therefore stable, the package calculates the decays of the lightest neutralino, and the lighter stau or stop NLSP into the gravitino LSP and one or two Standard Model particles. On the other hand, assuming that the gravitino is unstable, GravitinoPack calculates all its two-body and the three-body decay widths to the neutralino LSP and Standard Model particles. Since all these decays, involving the gravitino, are of gravitational nature, the lifetime of the decaying particle can be of the order of seconds are more, hence called "late decays". The precise knowledge of all these partial decay widths enables the user to apply accurately the relevant cosmological constraints. △ Less

Submitted 12 October, 2015; originally announced October 2015.

Comments: 9 pages, 5 figures, Proceedings of the 18th International Conference From the Planck Scale to the Electroweak Scale, PLANCK-2015, 25-29 May 2015, Ioannina, Greece

Report number: HEPHY-PUB 960/15

Journal ref: PoS (PLANCK 2015) 124

GravitinoPack and decays of supersymmetric metastable particles

Authors: Helmut Eberl, Vassilis C. Spanos

Abstract: We present the package GravitinoPack that calculates the two- and three-body decays of unstable supersymmetric particles involving the gravitino in the final or initial state. In a previous paper, we already showed results for the gravitino decays into two and three particles. In this paper, we incorporate the processes where an unstable neutralino, stau or stop decays into a gravitino and Standar… ▽ More We present the package GravitinoPack that calculates the two- and three-body decays of unstable supersymmetric particles involving the gravitino in the final or initial state. In a previous paper, we already showed results for the gravitino decays into two and three particles. In this paper, we incorporate the processes where an unstable neutralino, stau or stop decays into a gravitino and Standard Model particles. This is the case in gravitino dark matter supersymmetric models, where the gravitino is the lightest SUSY particle. We give instructions for the installation and the use of the package. In the numerical analysis, we discuss various MSSM scenarios. We show that the calculation of all the decay channels and the three-body decay branching ratios is essential for the accurate application of cosmological bounds on these models. △ Less

Submitted 30 September, 2015; originally announced September 2015.

Comments: 32 pages, 8 figures, 7 tables. The package can be downloaded from http://www.hephy.at/susytools/gravitinopack

Report number: HEPHY-PUB 959/15