Showing 1–13 of 13 results for author: Berges, M

State-of-the-Art Review: The Use of Digital Twins to Support Artificial Intelligence-Guided Predictive Maintenance

Authors: Sizhe Ma, Katherine A. Flanigan, Mario Bergés

Abstract: In recent years, predictive maintenance (PMx) has gained prominence for its potential to enhance efficiency, automation, accuracy, and cost-effectiveness while reducing human involvement. Importantly, PMx has evolved in tandem with digital advancements, such as Big Data and the Internet of Things (IOT). These technological strides have enabled Artificial Intelligence (AI) to revolutionize PMx proc… ▽ More In recent years, predictive maintenance (PMx) has gained prominence for its potential to enhance efficiency, automation, accuracy, and cost-effectiveness while reducing human involvement. Importantly, PMx has evolved in tandem with digital advancements, such as Big Data and the Internet of Things (IOT). These technological strides have enabled Artificial Intelligence (AI) to revolutionize PMx processes, with increasing capacities for real-time automation of monitoring, analysis, and prediction tasks. However, PMx still faces challenges such as poor explainability and sample inefficiency in data-driven methods and high complexity in physics-based models, hindering broader adoption. This paper posits that Digital Twins (DTs) can be integrated into PMx to overcome these challenges, paving the way for more automated PMx applications across various stakeholders. Despite their potential, current DTs have not fully matured to bridge existing gaps. Our paper provides a comprehensive roadmap for DT evolution, addressing current limitations to foster large-scale automated PMx progression. We structure our approach in three stages: First, we reference prior work where we identified and defined the Information Requirements (IRs) and Functional Requirements (FRs) for PMx, forming the blueprint for a unified framework. Second, we conduct a literature review to assess current DT applications integrating these IRs and FRs, revealing standardized DT models and tools that support automated PMx. Lastly, we highlight gaps in current DT implementations, particularly those IRs and FRs not fully supported, and outline the necessary components for a comprehensive, automated PMx system. Our paper concludes with research directions aimed at seamlessly integrating DTs into the PMx paradigm to achieve this ambitious vision. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: This work has been submitted to Springer for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Unmasking the Role of Remote Sensors in Comfort, Energy and Demand Response

Authors: Ozan Baris Mulayim, Edson Severnini, Mario Bergés

Abstract: In single-zone multi-room houses (SZMRHs), temperature controls rely on a single probe near the thermostat, resulting in temperature discrepancies that cause thermal discomfort and energy waste. Augmenting smart thermostats (STs) with per-room sensors has gained acceptance by major ST manufacturers. This paper leverages additional sensory information to empirically characterize the services provid… ▽ More In single-zone multi-room houses (SZMRHs), temperature controls rely on a single probe near the thermostat, resulting in temperature discrepancies that cause thermal discomfort and energy waste. Augmenting smart thermostats (STs) with per-room sensors has gained acceptance by major ST manufacturers. This paper leverages additional sensory information to empirically characterize the services provided by buildings, including thermal comfort, energy efficiency, and demand response (DR). Utilizing room-level time-series data from 1,000 houses, metadata from 110,000 houses across the United States, and data from two real-world testbeds, we examine the limitations of SZMRHs and explore the potential of remote sensors. We discovered that comfortable DR durations (CDRDs) for rooms are typically 70% longer or 40% shorter than for the room with the thermostat. When averaging, rooms at the control temperature's bounds are typically deviated around -3°F to 2.5°F from the average. Moreover, in 95\% of houses, we identified rooms experiencing notably higher solar gains compared to the rest of the rooms, while 85% and 70% of houses demonstrated lower heat input and poor insulation, respectively. Lastly, it became evident that the consumption of cooling energy escalates with the increase in the number of sensors, whereas heating usage experiences fluctuations ranging from -19% to +25% This study serves as a benchmark for assessing the thermal comfort and DR services in the existing housing stock, while also highlighting the energy efficiency impacts of sensing technologies. Our approach sets the stage for more granular, precise control strategies of SZMRHs. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Comments: 13 Figures, 8 Tables, 25 Pages. Submitted to Data-Centric Engineering Journal and it is under review

State-of-the-Art Review and Synthesis: A Requirement-based Roadmap for Standardized Predictive Maintenance Automation Using Digital Twin Technologies

Authors: Sizhe Ma, Katherine A. Flanigan, Mario Bergés

Abstract: Recent digital advances have popularized predictive maintenance (PMx), offering enhanced efficiency, automation, accuracy, cost savings, and independence in maintenance. Yet, it continues to face numerous limitations such as poor explainability, sample inefficiency of data-driven methods, complexity of physics-based methods, and limited generalizability and scalability of knowledge-based methods.… ▽ More Recent digital advances have popularized predictive maintenance (PMx), offering enhanced efficiency, automation, accuracy, cost savings, and independence in maintenance. Yet, it continues to face numerous limitations such as poor explainability, sample inefficiency of data-driven methods, complexity of physics-based methods, and limited generalizability and scalability of knowledge-based methods. This paper proposes leveraging Digital Twins (DTs) to address these challenges and enable automated PMx adoption at larger scales. While we argue that DTs have this transformative potential, they have not yet reached the level of maturity needed to bridge these gaps in a standardized way. Without a standard definition for such evolution, this transformation lacks a solid foundation upon which to base its development. This paper provides a requirement-based roadmap supporting standardized PMx automation using DT technologies. A systematic approach comprising two primary stages is presented. First, we methodically identify the Informational Requirements (IRs) and Functional Requirements (FRs) for PMx, which serve as a foundation from which any unified framework must emerge. Our approach to defining and using IRs and FRs to form the backbone of any PMx DT is supported by the track record of IRs and FRs being successfully used as blueprints in other areas, such as for product development within the software industry. Second, we conduct a thorough literature review spanning fields to determine the ways in which these IRs and FRs are currently being used within DTs, enabling us to point to the specific areas where further research is warranted to support the progress and maturation of requirement-based PMx DTs. △ Less

Submitted 12 November, 2023; originally announced November 2023.

Comments: (1)This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

The Digital Twin Landscape at the Crossroads of Predictive Maintenance, Machine Learning and Physics Based Modeling

Authors: Brian Kunzer, Mario Berges, Artur Dubrawski

Abstract: The concept of a digital twin has exploded in popularity over the past decade, yet confusion around its plurality of definitions, its novelty as a new technology, and its practical applicability still exists, all despite numerous reviews, surveys, and press releases. The history of the term digital twin is explored, as well as its initial context in the fields of product life cycle management, ass… ▽ More The concept of a digital twin has exploded in popularity over the past decade, yet confusion around its plurality of definitions, its novelty as a new technology, and its practical applicability still exists, all despite numerous reviews, surveys, and press releases. The history of the term digital twin is explored, as well as its initial context in the fields of product life cycle management, asset maintenance, and equipment fleet management, operations, and planning. A definition for a minimally viable framework to utilize a digital twin is also provided based on seven essential elements. A brief tour through DT applications and industries where DT methods are employed is also outlined. The application of a digital twin framework is highlighted in the field of predictive maintenance, and its extensions utilizing machine learning and physics based modeling. Employing the combination of machine learning and physics based modeling to form hybrid digital twin frameworks, may synergistically alleviate the shortcomings of each method when used in isolation. Key challenges of implementing digital twin models in practice are additionally discussed. As digital twin technology experiences rapid growth and as it matures, its great promise to substantially enhance tools and solutions for intelligent upkeep of complex equipment, are expected to materialize. △ Less

Submitted 23 June, 2022; v1 submitted 21 June, 2022; originally announced June 2022.

Comments: 21 pages, 5 figures

HierMUD: Hierarchical Multi-task Unsupervised Domain Adaptation between Bridges for Drive-by Damage Diagnosis

Authors: **gxiao Liu, Susu Xu, Mario Bergés, Hae Young Noh

Abstract: Monitoring bridge health using vibrations of drive-by vehicles has various benefits, such as no need for directly installing and maintaining sensors on the bridge. However, many of the existing drive-by monitoring approaches are based on supervised learning models that require labeled data from every bridge of interest, which is expensive and time-consuming, if not impossible, to obtain. To this e… ▽ More Monitoring bridge health using vibrations of drive-by vehicles has various benefits, such as no need for directly installing and maintaining sensors on the bridge. However, many of the existing drive-by monitoring approaches are based on supervised learning models that require labeled data from every bridge of interest, which is expensive and time-consuming, if not impossible, to obtain. To this end, we introduce a new framework that transfers the model learned from one bridge to diagnose damage in another bridge without any labels from the target bridge. Our framework trains a hierarchical neural network model in an adversarial way to extract task-shared and task-specific features that are informative to multiple diagnostic tasks and invariant across multiple bridges. We evaluate our framework on experimental data collected from 2 bridges and 3 vehicles. We achieve accuracies of 95% for damage detection, 93% for localization, and up to 72% for quantification, which are ~2 times improvements from baseline methods. △ Less

Submitted 23 July, 2021; originally announced July 2021.

Journal ref: Structural Health Monitoring 22(3):1941-1968, 2023

Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization

Authors: Bingqing Chen, Priya Donti, Kyri Baker, J. Zico Kolter, Mario Berges

Abstract: While reinforcement learning (RL) is gaining popularity in energy systems control, its real-world applications are limited due to the fact that the actions from learned policies may not satisfy functional requirements or be feasible for the underlying physical system. In this work, we propose PROjected Feasibility (PROF), a method to enforce convex operational constraints within neural policies. S… ▽ More While reinforcement learning (RL) is gaining popularity in energy systems control, its real-world applications are limited due to the fact that the actions from learned policies may not satisfy functional requirements or be feasible for the underlying physical system. In this work, we propose PROjected Feasibility (PROF), a method to enforce convex operational constraints within neural policies. Specifically, we incorporate a differentiable projection layer within a neural network-based policy to enforce that all learned actions are feasible. We then update the policy end-to-end by propagating gradients through this differentiable projection layer, making the policy cognizant of the operational constraints. We demonstrate our method on two applications: energy-efficient building operation and inverter control. In the building operation setting, we show that PROF maintains thermal comfort requirements while improving energy efficiency by 4% over state-of-the-art methods. In the inverter control setting, PROF perfectly satisfies voltage constraints on the IEEE 37-bus feeder system, as it learns to curtail as little renewable energy as possible within its safety set. △ Less

Submitted 18 May, 2021; originally announced May 2021.

Comments: Accepted at Twelfth ACM International Conference on Future Energy Systems (ACM e-Energy)

When Interactive Graphic Storytelling Fails

Authors: James Barela, Tiago Espinha Gasiba, Santiago Reinhard Suppan, Marc Berges, Kristian Beckers

Abstract: Many people are unaware of the digital dangers that lie around each cyber-corner. Teaching people how to recognize dangerous situations is crucial, especially for those who work on or with computers. We postulated that interactive graphic vignettes could be a great way to expose professionals to dangerous situations and demonstrate the effects of their choices in these situations. In that way, we… ▽ More Many people are unaware of the digital dangers that lie around each cyber-corner. Teaching people how to recognize dangerous situations is crucial, especially for those who work on or with computers. We postulated that interactive graphic vignettes could be a great way to expose professionals to dangerous situations and demonstrate the effects of their choices in these situations. In that way, we aimed to inoculate employees against cybersecurity threats. We used the Comic-BEE platform to create interactive security awareness vignettes and evaluated for how employees of a major industrial company perceived them. For analysing the potential of these comics, we ran an evaluation study as part of a capture-the-flag (CTF) event, an interactive exercise for hacking vulnerable software. We evaluated whether the comics fulfilled our requirements based on the responses of the participants. We showed the comics, on various cybersecurity concepts, to 20 volunteers. In the context of a CTF event, our requirements were not fulfilled. Most participants considered the images distracting, stating a preference for text-only material. △ Less

Submitted 6 January, 2021; originally announced January 2021.

Comments: Preprint accepted for publication at the IEEE 27th International Requirements Engineering Conference (RE), 2019

Learning to Solve AC Optimal Power Flow by Differentiating through Holomorphic Embeddings

Authors: Henning Lange, Bingqing Chen, Mario Berges, Soummya Kar

Abstract: Alternating current optimal power flow (AC-OPF) is one of the fundamental problems in power systems operation. AC-OPF is traditionally cast as a constrained optimization problem that seeks optimal generation set points whilst fulfilling a set of non-linear equality constraints -- the power flow equations. With increasing penetration of renewable generation, grid operators need to solve larger prob… ▽ More Alternating current optimal power flow (AC-OPF) is one of the fundamental problems in power systems operation. AC-OPF is traditionally cast as a constrained optimization problem that seeks optimal generation set points whilst fulfilling a set of non-linear equality constraints -- the power flow equations. With increasing penetration of renewable generation, grid operators need to solve larger problems at shorter intervals. This motivates the research interest in learning OPF solutions with neural networks, which have fast inference time and is potentially scalable to large networks. The main difficulty in solving the AC-OPF problem lies in dealing with this equality constraint that has spurious roots, i.e. there are assignments of voltages that fulfill the power flow equations that however are not physically realizable. This property renders any method relying on projected-gradients brittle because these non-physical roots can act as attractors. In this paper, we show efficient strategies that circumvent this problem by differentiating through the operations of a power flow solver that embeds the power flow equations into a holomorphic function. The resulting learning-based approach is validated experimentally on a 200-bus system and we show that, after training, the learned agent produces optimized power flow solutions reliably and fast. Specifically, we report a 12x increase in speed and a 40% increase in robustness compared to a traditional solver. To the best of our knowledge, this approach constitutes the first learning-based approach that successfully respects the full non-linear AC-OPF equations. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Comments: 10 pages

On the Feasibility of Exploiting Traffic Collision Avoidance System Vulnerabilities

Authors: Paul M. Berges, Basavesh Ammanaghatta Shivakumar, Timothy Graziano, Ryan Gerdes, Z. Berkay Celik

Abstract: Traffic Collision Avoidance Systems (TCAS) are safety-critical systems required on most commercial aircrafts in service today. However, TCAS was not designed to account for malicious actors. While in the past it may have been infeasible for an attacker to craft radio signals to mimic TCAS signals, attackers today have access to open-source digital signal processing software, like GNU Radio, and in… ▽ More Traffic Collision Avoidance Systems (TCAS) are safety-critical systems required on most commercial aircrafts in service today. However, TCAS was not designed to account for malicious actors. While in the past it may have been infeasible for an attacker to craft radio signals to mimic TCAS signals, attackers today have access to open-source digital signal processing software, like GNU Radio, and inexpensive software defined radios (SDR) that enable the transmission of spurious TCAS messages. In this paper, methods, both qualitative and quantitative, for analyzing TCAS from an adversarial perspective are presented. To demonstrate the feasibility of inducing near mid-air collisions between current day TCAS-equipped aircraft, an experimental Phantom Aircraft generator is developed using GNU Radio and an SDR against a realistic threat model. △ Less

Submitted 25 June, 2020; originally announced June 2020.

Data-driven Thermal Model Inference with ARMAX, in Smart Environments, based on Normalized Mutual Information

Authors: Zhanhong Jiang, Jonathan Francis, Anit Kumar Sahu, Sirajum Munir, Charles Shelton, Anthony Rowe, Mario Bergés

Abstract: Understanding the models that characterize the thermal dynamics in a smart building is important for the comfort of its occupants and for its energy optimization. A significant amount of research has attempted to utilize thermodynamics (physical) models for smart building control, but these approaches remain challenging due to the stochastic nature of the intermittent environmental disturbances. T… ▽ More Understanding the models that characterize the thermal dynamics in a smart building is important for the comfort of its occupants and for its energy optimization. A significant amount of research has attempted to utilize thermodynamics (physical) models for smart building control, but these approaches remain challenging due to the stochastic nature of the intermittent environmental disturbances. This paper presents a novel data-driven approach for indoor thermal model inference, which combines an Autoregressive Moving Average with eXogenous inputs model (ARMAX) with a Normalized Mutual Information scheme (NMI). Based on this information-theoretic method, NMI, causal dependencies between the indoor temperature and exogenous inputs are explicitly obtained as a guideline for the ARMAX model to find the dominating inputs. For validation, we use three datasets based on building energy systems-against which we compare our method to an autoregressive model with exogenous inputs (ARX), a regularized ARMAX model, and state-space models. △ Less

Submitted 10 June, 2020; originally announced June 2020.

Journal ref: American Control Conference (2018) 4634-4639

Knowledge transfer between bridges for drive-by monitoring using adversarial and multi-task learning

Authors: **gxiao Liu, Mario Bergés, Jacobo Bielak, Hae Young Noh

Abstract: Monitoring bridge health using the vibrations of drive-by vehicles has various benefits, such as low cost and no need for direct installation or on-site maintenance of equipment on the bridge. However, many such approaches require labeled data from every bridge, which is expensive and time-consuming, if not impossible, to obtain. This is further exacerbated by having multiple diagnostic tasks, suc… ▽ More Monitoring bridge health using the vibrations of drive-by vehicles has various benefits, such as low cost and no need for direct installation or on-site maintenance of equipment on the bridge. However, many such approaches require labeled data from every bridge, which is expensive and time-consuming, if not impossible, to obtain. This is further exacerbated by having multiple diagnostic tasks, such as damage quantification and localization. One way to address this issue is to directly apply the supervised model trained for one bridge to other bridges, although this may significantly reduce the accuracy because of distribution mismatch between different bridges'data. To alleviate these problems, we introduce a transfer learning framework using domain-adversarial training and multi-task learning to detect, localize and quantify damage. Specifically, we train a deep network in an adversarial way to learn features that are 1) sensitive to damage and 2) invariant to different bridges. In addition, to improve the error propagation from one task to the next, our framework learns shared features for all the tasks using multi-task learning. We evaluate our framework using lab-scale experiments with two different bridges. On average, our framework achieves 94%, 97% and 84% accuracy for damage detection, localization and quantification, respectively. within one damage severity level. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Incremental Real-Time Personalization in Human Activity Recognition Using Domain Adaptive Batch Normalization

Authors: Alan Mazankiewicz, Klemens Böhm, Mario Bergés

Abstract: Human Activity Recognition (HAR) from devices like smartphone accelerometers is a fundamental problem in ubiquitous computing. Machine learning based recognition models often perform poorly when applied to new users that were not part of the training data. Previous work has addressed this challenge by personalizing general recognition models to the unique motion pattern of a new user in a static b… ▽ More Human Activity Recognition (HAR) from devices like smartphone accelerometers is a fundamental problem in ubiquitous computing. Machine learning based recognition models often perform poorly when applied to new users that were not part of the training data. Previous work has addressed this challenge by personalizing general recognition models to the unique motion pattern of a new user in a static batch setting. They require target user data to be available upfront. The more challenging online setting has received less attention. No samples from the target user are available in advance, but they arrive sequentially. Additionally, the motion pattern of users may change over time. Thus, adapting to new and forgetting old information must be traded off. Finally, the target user should not have to do any work to use the recognition system by, say, labeling any activities. Our work addresses all of these challenges by proposing an unsupervised online domain adaptation algorithm. Both classification and personalization happen continuously and incrementally in real time. Our solution works by aligning the feature distributions of all subjects, be they sources or the target, in hidden neural network layers. To this end, we normalize the input of a layer with user-specific mean and variance statistics. During training, these statistics are computed over user-specific batches. In the online phase, they are estimated incrementally for any new target user. △ Less

Submitted 21 December, 2020; v1 submitted 25 May, 2020; originally announced May 2020.

Comments: Updated version of the preprint from 05/2020 after going through revision. The content (experiments, results, proposed method) has not changed. The explanations changed. Certain sentences have been added/removed/rephrased to be clearer. Removed Figure 3. Added Discussion section. Renamed "Description of Approach" Section. Added a reference to related work

Journal ref: Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4, Article 144 (December 2020), 20 pages

Damage-sensitive and domain-invariant feature extraction for vehicle-vibration-based bridge health monitoring

Authors: **gxiao Liu, Bingqing Chen, Siheng Chen, Mario Berges, Jacobo Bielak, HaeYoung Noh

Abstract: We introduce a physics-guided signal processing approach to extract a damage-sensitive and domain-invariant (DS & DI) feature from acceleration response data of a vehicle traveling over a bridge to assess bridge health. Motivated by indirect sensing methods' benefits, such as low-cost and low-maintenance, vehicle-vibration-based bridge health monitoring has been studied to efficiently monitor brid… ▽ More We introduce a physics-guided signal processing approach to extract a damage-sensitive and domain-invariant (DS & DI) feature from acceleration response data of a vehicle traveling over a bridge to assess bridge health. Motivated by indirect sensing methods' benefits, such as low-cost and low-maintenance, vehicle-vibration-based bridge health monitoring has been studied to efficiently monitor bridges in real-time. Yet applying this approach is challenging because 1) physics-based features extracted manually are generally not damage-sensitive, and 2) features from machine learning techniques are often not applicable to different bridges. Thus, we formulate a vehicle bridge interaction system model and find a physics-guided DS & DI feature, which can be extracted using the synchrosqueezed wavelet transform representing non-stationary signals as intrinsic-mode-type components. We validate the effectiveness of the proposed feature with simulated experiments. Compared to conventional time- and frequency-domain features, our feature provides the best damage quantification and localization results across different bridges in five of six experiments. △ Less

Submitted 6 February, 2020; originally announced February 2020.

Comments: To appear in Proc. ICASSP2020, May 04-08, 2020, Barcelona, Spain. IEEE

MSC Class: 68T10 (Primary); 37N20 (Secondary) ACM Class: I.5.4; J.2