Showing 1–26 of 26 results for author: Vu, L

Toward end-to-end interpretable convolutional neural networks for waveform signals

Authors: Linh Vu, Thu Tran, Wern-Han Lim, Raphael Phan

Abstract: This paper introduces a novel convolutional neural networks (CNN) framework tailored for end-to-end audio deep learning models, presenting advancements in efficiency and explainability. By benchmarking experiments on three standard speech emotion recognition datasets with five-fold cross-validation, our framework outperforms Mel spectrogram features by up to seven percent. It can potentially repla… ▽ More This paper introduces a novel convolutional neural networks (CNN) framework tailored for end-to-end audio deep learning models, presenting advancements in efficiency and explainability. By benchmarking experiments on three standard speech emotion recognition datasets with five-fold cross-validation, our framework outperforms Mel spectrogram features by up to seven percent. It can potentially replace the Mel-Frequency Cepstral Coefficients (MFCC) while remaining lightweight. Furthermore, we demonstrate the efficiency and interpretability of the front-end layer using the PhysioNet Heart Sound Database, illustrating its ability to handle and capture intricate long waveform patterns. Our contributions offer a portable solution for building efficient and interpretable models for raw waveform data. △ Less

Submitted 2 May, 2024; originally announced May 2024.

A novel framework for adaptive stress testing of autonomous vehicles in highways

Authors: Linh Trinh, Quang-Hung Luu, Thai M. Nguyen, Hai L. Vu

Abstract: Guaranteeing the safe operations of autonomous vehicles (AVs) is crucial for their widespread adoption and public acceptance. It is thus of a great significance to not only assess the AV against the standard safety tests, but also discover potential corner cases of the AV under test that could lead to unsafe behaviour or scenario. In this paper, we propose a novel framework to systematically explo… ▽ More Guaranteeing the safe operations of autonomous vehicles (AVs) is crucial for their widespread adoption and public acceptance. It is thus of a great significance to not only assess the AV against the standard safety tests, but also discover potential corner cases of the AV under test that could lead to unsafe behaviour or scenario. In this paper, we propose a novel framework to systematically explore corner cases that can result in safety concerns in a highway traffic scenario. The framework is based on an adaptive stress testing (AST) approach, an emerging validation method that leverages a Markov decision process to formulate the scenarios and deep reinforcement learning (DRL) to discover the desirable patterns representing corner cases. To this end, we develop a new reward function for DRL to guide the AST in identifying crash scenarios based on the collision probability estimate between the AV under test (i.e., the ego vehicle) and the trajectory of other vehicles on the highway. The proposed framework is further integrated with a new driving model enabling us to create more realistic traffic scenarios capturing both the longitudinal and lateral movements of vehicles on the highway. In our experiment, we calibrate our model using real-world crash statistics involving automated vehicles in California, and then we analyze the characteristics of the AV and the framework. Quantitative and qualitative analyses of our experimental results demonstrate that our framework outperforms other existing AST schemes. The study can help discover crash scenarios of AV that are unknown or absent in human driving, thereby enhancing the safety and trustworthiness of AV technology. △ Less

Submitted 18 February, 2024; originally announced February 2024.

Adaptive Primal-Dual Method for Safe Reinforcement Learning

Authors: Weiqin Chen, James Onyejizu, Long Vu, Lan Hoang, Dharmashankar Subramanian, Koushik Kar, Sandipan Mishra, Santiago Paternain

Abstract: Primal-dual methods have a natural application in Safe Reinforcement Learning (SRL), posed as a constrained policy optimization problem. In practice however, applying primal-dual methods to SRL is challenging, due to the inter-dependency of the learning rate (LR) and Lagrangian multipliers (dual variables) each time an embedded unconstrained RL problem is solved. In this paper, we propose, analyze… ▽ More Primal-dual methods have a natural application in Safe Reinforcement Learning (SRL), posed as a constrained policy optimization problem. In practice however, applying primal-dual methods to SRL is challenging, due to the inter-dependency of the learning rate (LR) and Lagrangian multipliers (dual variables) each time an embedded unconstrained RL problem is solved. In this paper, we propose, analyze and evaluate adaptive primal-dual (APD) methods for SRL, where two adaptive LRs are adjusted to the Lagrangian multipliers so as to optimize the policy in each iteration. We theoretically establish the convergence, optimality and feasibility of the APD algorithm. Finally, we conduct numerical evaluation of the practical APD algorithm with four well-known environments in Bullet-Safey-Gym employing two state-of-the-art SRL algorithms: PPO-Lagrangian and DDPG-Lagrangian. All experiments show that the practical APD algorithm outperforms (or achieves comparable performance) and attains more stable training than the constant LR cases. Additionally, we substantiate the robustness of selecting the two adaptive LRs by empirical evidence. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Digital Infrastructure for Connected and Automated Vehicles

Authors: Quang-Hung Luu, Thai M. Nguyen, Nan Zheng, Hai L. Vu

Abstract: Connected and automated vehicles (CAV) are expected to deliver a much safer, more efficient, and eco-friendlier mobility. Being an indispensable component of the future transportation, their key driving features of CAVs include not only the automated functionality but also the cooperative capability. Despite the CAVs themselves are emerging and active research areas, there is a lack of a comprehen… ▽ More Connected and automated vehicles (CAV) are expected to deliver a much safer, more efficient, and eco-friendlier mobility. Being an indispensable component of the future transportation, their key driving features of CAVs include not only the automated functionality but also the cooperative capability. Despite the CAVs themselves are emerging and active research areas, there is a lack of a comprehensive literature review on the digital infrastructure that enables them. In this paper, we review the requirements and benefits of digital infrastructures for the CAVs including the vehicle built-in, roadside-based, operational and planning infrastructures. We then highlight challenges and opportunities on digital infrastructure research for the CAVs. Our study sheds lights on seamless integration of digital infrastructure for safe operations of CAVs. △ Less

Submitted 30 November, 2023; originally announced January 2024.

Comments: 24 pages, 2 figures, 1 table

DP-NMT: Scalable Differentially-Private Machine Translation

Authors: Timour Igamberdiev, Doan Nam Long Vu, Felix Künnecke, Zhuo Yu, Jannik Holmer, Ivan Habernal

Abstract: Neural machine translation (NMT) is a widely popular text generation task, yet there is a considerable research gap in the development of privacy-preserving NMT models, despite significant data privacy concerns for NMT systems. Differentially private stochastic gradient descent (DP-SGD) is a popular method for training machine learning models with concrete privacy guarantees; however, the implemen… ▽ More Neural machine translation (NMT) is a widely popular text generation task, yet there is a considerable research gap in the development of privacy-preserving NMT models, despite significant data privacy concerns for NMT systems. Differentially private stochastic gradient descent (DP-SGD) is a popular method for training machine learning models with concrete privacy guarantees; however, the implementation specifics of training a model with DP-SGD are not always clarified in existing models, with differing software libraries used and code bases not always being public, leading to reproducibility issues. To tackle this, we introduce DP-NMT, an open-source framework for carrying out research on privacy-preserving NMT with DP-SGD, bringing together numerous models, datasets, and evaluation metrics in one systematic software package. Our goal is to provide a platform for researchers to advance the development of privacy-preserving NMT systems, kee** the specific details of the DP-SGD algorithm transparent and intuitive to implement. We run a set of experiments on datasets from both general and privacy-related domains to demonstrate our framework in use. We make our framework publicly available and welcome feedback from the community. △ Less

Submitted 24 April, 2024; v1 submitted 24 November, 2023; originally announced November 2023.

Comments: Accepted at EACL 2024

Economics of Spot Instance Service: A Two-stage Dynamic Game Apporach

Authors: Hyojung Lee, Lam Vu, Minsung Jang

Abstract: This paper presents the economic impacts of spot instance service on the cloud service providers (CSPs) and the customers when the CSPs offer it along with the on-demand instance service to the customers. We model the interaction between CSPs and customers as a non-cooperative two-stage dynamic game. Our equilibrium analysis reveals (i) the techno-economic interrelationship between the customers'… ▽ More This paper presents the economic impacts of spot instance service on the cloud service providers (CSPs) and the customers when the CSPs offer it along with the on-demand instance service to the customers. We model the interaction between CSPs and customers as a non-cooperative two-stage dynamic game. Our equilibrium analysis reveals (i) the techno-economic interrelationship between the customers' heterogeneity, resource availability, and CSPs' pricing policy, and (ii) the impacts of the customers' service selection (spot vs. on-demand) and the CSPs' pricing decision on the CSPs' market share and revenue, as well as the customers' utility. The key technical challenges lie in, first, how we capture the strategic interactions between CSPs and customers, and second, how we consider the various practical aspects of cloud services, such as heterogeneity of customers' willingness to pay for the quality of service (QoS) and the fluctuating resource availability. The main contribution of this paper is to provide CSPs and customers with a better understanding of the economic impact caused by a certain price policy for the spot service when the equilibrium price, which from our two-stage dynamic game analysis, is able to set as the baseline price for their spot service. △ Less

Submitted 1 June, 2023; v1 submitted 31 May, 2023; originally announced May 2023.

AutoDOViz: Human-Centered Automation for Decision Optimization

Authors: Daniel Karl I. Weidele, Shazia Afzal, Abel N. Valente, Cole Makuch, Owen Cornec, Long Vu, Dharmashankar Subramanian, Werner Geyer, Rahul Nair, Inge Vejsbjerg, Radu Marinescu, Paulito Palmes, Elizabeth M. Daly, Loraine Franke, Daniel Haehn

Abstract: We present AutoDOViz, an interactive user interface for automated decision optimization (AutoDO) using reinforcement learning (RL). Decision optimization (DO) has classically being practiced by dedicated DO researchers where experts need to spend long periods of time fine tuning a solution through trial-and-error. AutoML pipeline search has sought to make it easier for a data scientist to find the… ▽ More We present AutoDOViz, an interactive user interface for automated decision optimization (AutoDO) using reinforcement learning (RL). Decision optimization (DO) has classically being practiced by dedicated DO researchers where experts need to spend long periods of time fine tuning a solution through trial-and-error. AutoML pipeline search has sought to make it easier for a data scientist to find the best machine learning pipeline by leveraging automation to search and tune the solution. More recently, these advances have been applied to the domain of AutoDO, with a similar goal to find the best reinforcement learning pipeline through algorithm selection and parameter tuning. However, Decision Optimization requires significantly more complex problem specification when compared to an ML problem. AutoDOViz seeks to lower the barrier of entry for data scientists in problem specification for reinforcement learning problems, leverage the benefits of AutoDO algorithms for RL pipeline search and finally, create visualizations and policy insights in order to facilitate the typical interactive nature when communicating problem formulation and solution proposals between DO experts and domain experts. In this paper, we report our findings from semi-structured expert interviews with DO practitioners as well as business consultants, leading to design requirements for human-centered automation for DO with RL. We evaluate a system implementation with data scientists and find that they are significantly more open to engage in DO after using our proposed solution. AutoDOViz further increases trust in RL agent models and makes the automated training and evaluation process more comprehensible. As shown for other automation in ML tasks, we also conclude automation of RL for DO can benefit from user and vice-versa when the interface promotes human-in-the-loop. △ Less

Submitted 19 February, 2023; originally announced February 2023.

Superhuman Fairness

Authors: Omid Memarrast, Linh Vu, Brian Ziebart

Abstract: The fairness of machine learning-based decisions has become an increasingly important focus in the design of supervised machine learning methods. Most fairness approaches optimize a specified trade-off between performance measure(s) (e.g., accuracy, log loss, or AUC) and fairness metric(s) (e.g., demographic parity, equalized odds). This begs the question: are the right performance-fairness trade-… ▽ More The fairness of machine learning-based decisions has become an increasingly important focus in the design of supervised machine learning methods. Most fairness approaches optimize a specified trade-off between performance measure(s) (e.g., accuracy, log loss, or AUC) and fairness metric(s) (e.g., demographic parity, equalized odds). This begs the question: are the right performance-fairness trade-offs being specified? We instead re-cast fair machine learning as an imitation learning task by introducing superhuman fairness, which seeks to simultaneously outperform human decisions on multiple predictive performance and fairness measures. We demonstrate the benefits of this approach given suboptimal decisions. △ Less

Submitted 31 January, 2023; originally announced January 2023.

Layer or Representation Space: What makes BERT-based Evaluation Metrics Robust?

Authors: Doan Nam Long Vu, Nafise Sadat Moosavi, Steffen Eger

Abstract: The evaluation of recent embedding-based evaluation metrics for text generation is primarily based on measuring their correlation with human evaluations on standard benchmarks. However, these benchmarks are mostly from similar domains to those used for pretraining word embeddings. This raises concerns about the (lack of) generalization of embedding-based metrics to new and noisy domains that conta… ▽ More The evaluation of recent embedding-based evaluation metrics for text generation is primarily based on measuring their correlation with human evaluations on standard benchmarks. However, these benchmarks are mostly from similar domains to those used for pretraining word embeddings. This raises concerns about the (lack of) generalization of embedding-based metrics to new and noisy domains that contain a different vocabulary than the pretraining data. In this paper, we examine the robustness of BERTScore, one of the most popular embedding-based metrics for text generation. We show that (a) an embedding-based metric that has the highest correlation with human evaluations on a standard benchmark can have the lowest correlation if the amount of input noise or unknown tokens increases, (b) taking embeddings from the first layer of pretrained models improves the robustness of all metrics, and (c) the highest robustness is achieved when using character-level embeddings, instead of token-based embeddings, from the first layer of the pretrained model. △ Less

Submitted 7 September, 2022; v1 submitted 6 September, 2022; originally announced September 2022.

Comments: COLING 2022 camera-ready version

Closed-loop Position Control of a Pediatric Soft Robotic Wearable Device for Upper Extremity Assistance

Authors: Caio Mucchiani, Zhichao Liu, Ipsita Sahin, Jared Dube, Linh Vu, Elena Kokkoni, Konstantinos Karydis

Abstract: This work focuses on closed-loop control based on proprioceptive feedback for a pneumatically-actuated soft wearable device aimed at future support of infant reaching tasks. The device comprises two soft pneumatic actuators (one textile-based and one silicone-casted) actively controlling two degrees-of-freedom per arm (shoulder adduction/abduction and elbow flexion/extension, respectively). Inerti… ▽ More This work focuses on closed-loop control based on proprioceptive feedback for a pneumatically-actuated soft wearable device aimed at future support of infant reaching tasks. The device comprises two soft pneumatic actuators (one textile-based and one silicone-casted) actively controlling two degrees-of-freedom per arm (shoulder adduction/abduction and elbow flexion/extension, respectively). Inertial measurement units (IMUs) attached to the wearable device provide real-time joint angle feedback. Device kinematics analysis is informed by anthropometric data from infants (arm lengths) reported in the literature. Range of motion and muscle co-activation patterns in infant reaching are considered to derive desired trajectories for the device's end-effector. Then, a proportional-derivative controller is developed to regulate the pressure inside the actuators and in turn move the arm along desired setpoints within the reachable workspace. Experimental results on tracking desired arm trajectories using an engineered mannequin are presented, demonstrating that the proposed controller can help guide the mannequin's wrist to the desired setpoints. △ Less

Submitted 16 June, 2022; originally announced June 2022.

Comments: 6 pages

Journal ref: Roman 2022

Testing Ocean Software with Metamorphic Testing

Authors: Quang-Hung Luu, Huai Liu, Tsong Yueh Chen, Hai L. Vu

Abstract: Advancing ocean science has a significant impact to the development of the world, from operating a safe navigation for vessels to maintaining a healthy and diverse ocean ecosystem. Various ocean software systems have been extensively adopted for different purposes, for instance, predicting hourly sea level elevation across shorelines, simulating large-scale ocean circulations, as well as integrati… ▽ More Advancing ocean science has a significant impact to the development of the world, from operating a safe navigation for vessels to maintaining a healthy and diverse ocean ecosystem. Various ocean software systems have been extensively adopted for different purposes, for instance, predicting hourly sea level elevation across shorelines, simulating large-scale ocean circulations, as well as integrating into Earth system models for weather forecasts and climate projections. Regardless of their significance, guaranteeing the trustworthiness of ocean software and modelling systems is a long-standing challenge. The testing of ocean software suffers a lot from the so-called oracle problem, which refers to the absence of test oracles mainly due to the nonlinear interactions of multiple physical variables and the high complexity in computation. In the ocean, observed tidal signals are distorted by non-deterministic physical variables, hindering us from knowing the "true" astronomical tidal constituents existing in the timeseries. In this paper, we present how to test tidal analysis and prediction (TAP) software based on metamorphic testing (MT), a simple yet effective testing approach to the oracle problem. In particular, we construct metamorphic relations from the periodic property of astronomical tide, and then use them to successfully detect a real-life defect in an open-source TAP software. We also conduct a series of experiments to further demonstrate the applicability and effectiveness of MT in the testing of TAP software. Our study not only justifies the potential of MT in testing more complex ocean software and modelling systems, but also can be expanded to assess and improve the quality of a broader range of scientific simulation software systems. △ Less

Submitted 11 June, 2022; originally announced June 2022.

Comments: 7 pages, 3 tables

A Sequential Metamorphic Testing Framework for Understanding Automated Driving Systems

Authors: Quang-Hung Luu, Huai Liu, Tsong Yueh Chen, Hai L. Vu

Abstract: Automated driving systems (ADS) are expected to be reliable and robust against a wide range of driving scenarios. Their decisions, first and foremost, must be well understood. Understanding a decision made by ADS is a great challenge, because it is not straightforward to tell whether the decision is correct or not, and how to verify it systematically. In this paper, a Sequential MetAmoRphic Testin… ▽ More Automated driving systems (ADS) are expected to be reliable and robust against a wide range of driving scenarios. Their decisions, first and foremost, must be well understood. Understanding a decision made by ADS is a great challenge, because it is not straightforward to tell whether the decision is correct or not, and how to verify it systematically. In this paper, a Sequential MetAmoRphic Testing Smart framework is proposed based on metamorphic testing, a mainstream software testing approach. In metamorphic testing, metamorphic groups are constructed by selecting multiple inputs according to the so-called metamorphic relations, which are basically the system's necessary properties; the violation of certain relations by some corresponding metamorphic groups implies the detection of erroneous system behaviors. The proposed framework makes use of sequences of metamorphic groups to understand ADS behaviors, and is applicable without the need of ground-truth datasets. To demonstrate its effectiveness, the framework is applied to test three ADS models that steer an autonomous car in different scenarios with another car either leading in front or approaching in the opposite direction. The conducted experiments reveal a large number of undesirable behaviors in these top-ranked deep learning models in the scenarios. These counter-intuitive behaviors are associated with how the core models of ADS respond to different positions, directions and properties of the other car in its proximity. Further analysis of the results helps identify critical factors affecting ADS decisions and thus demonstrates that the framework can be used to provide a comprehensive understanding of ADS before their deployment △ Less

Submitted 7 June, 2022; originally announced June 2022.

Comments: 11 pages, 6 figures, 3 tables

Traffic4cast at NeurIPS 2021 -- Temporal and Spatial Few-Shot Transfer Learning in Gridded Geo-Spatial Processes

Authors: Christian Eichenberger, Moritz Neun, Henry Martin, Pedro Herruzo, Markus Spanring, Yichao Lu, Sungbin Choi, Vsevolod Konyakhin, Nina Lukashina, Aleksei Shpilman, Nina Wiedemann, Martin Raubal, Bo Wang, Hai L. Vu, Reza Mohajerpoor, Chen Cai, Inhi Kim, Luca Hermes, Andrew Melnik, Riza Velioglu, Markus Vieth, Malte Schilling, Alabi Bojesomo, Hasan Al Marzouqi, Panos Liatsis , et al. (12 additional authors not shown)

Abstract: The IARAI Traffic4cast competitions at NeurIPS 2019 and 2020 showed that neural networks can successfully predict future traffic conditions 1 hour into the future on simply aggregated GPS probe data in time and space bins. We thus reinterpreted the challenge of forecasting traffic conditions as a movie completion task. U-Nets proved to be the winning architecture, demonstrating an ability to extra… ▽ More The IARAI Traffic4cast competitions at NeurIPS 2019 and 2020 showed that neural networks can successfully predict future traffic conditions 1 hour into the future on simply aggregated GPS probe data in time and space bins. We thus reinterpreted the challenge of forecasting traffic conditions as a movie completion task. U-Nets proved to be the winning architecture, demonstrating an ability to extract relevant features in this complex real-world geo-spatial process. Building on the previous competitions, Traffic4cast 2021 now focuses on the question of model robustness and generalizability across time and space. Moving from one city to an entirely different city, or moving from pre-COVID times to times after COVID hit the world thus introduces a clear domain shift. We thus, for the first time, release data featuring such domain shifts. The competition now covers ten cities over 2 years, providing data compiled from over 10^12 GPS probe data. Winning solutions captured traffic dynamics sufficiently well to even cope with these complex domain shifts. Surprisingly, this seemed to require only the previous 1h traffic dynamic history and static road graph as input. △ Less

Submitted 1 April, 2022; v1 submitted 31 March, 2022; originally announced March 2022.

Comments: Pre-print under review, submitted to Proceedings of Machine Learning Research

Safe Reinforcement Learning for Grid Voltage Control

Authors: Thanh Long Vu, Sayak Mukherjee, Renke Huang, Qiuhua Huang

Abstract: Under voltage load shedding has been considered as a standard approach to recover the voltage stability of the electric power grid under emergency conditions, yet this scheme usually trips a massive amount of load inefficiently. Reinforcement learning (RL) has been adopted as a promising approach to circumvent the issues; however, RL approach usually cannot guarantee the safety of the systems unde… ▽ More Under voltage load shedding has been considered as a standard approach to recover the voltage stability of the electric power grid under emergency conditions, yet this scheme usually trips a massive amount of load inefficiently. Reinforcement learning (RL) has been adopted as a promising approach to circumvent the issues; however, RL approach usually cannot guarantee the safety of the systems under control. In this paper, we discuss a couple of novel safe RL approaches, namely constrained optimization approach and Barrier function-based approach, that can safely recover voltage under emergency events. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark are performed to demonstrate the effectiveness of the proposed safe RL emergency control. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: Workshop on Safe and Robust Control of Uncertain Systems at the 35th Conference on Neural Information Processing Systems (NeurIPS) 2021. arXiv admin note: substantial text overlap with arXiv:2103.14186, arXiv:2011.09664, arXiv:2006.12667

Journal ref: Non-archival document for workshop discussion, 2021

Traffic4cast -- Large-scale Traffic Prediction using 3DResNet and Sparse-UNet

Authors: Bo Wang, Reza Mohajerpoor, Chen Cai, Inhi Kim, Hai L. Vu

Abstract: The IARAI competition Traffic4cast 2021 aims to predict short-term city-wide high-resolution traffic states given the static and dynamic traffic information obtained previously. The aim is to build a machine learning model for predicting the normalized average traffic speed and flow of the subregions of multiple large-scale cities using historical data points. The model is supposed to be generic,… ▽ More The IARAI competition Traffic4cast 2021 aims to predict short-term city-wide high-resolution traffic states given the static and dynamic traffic information obtained previously. The aim is to build a machine learning model for predicting the normalized average traffic speed and flow of the subregions of multiple large-scale cities using historical data points. The model is supposed to be generic, in a way that it can be applied to new cities. By considering spatiotemporal feature learning and modeling efficiency, we explore 3DResNet and Sparse-UNet approaches for the tasks in this competition. The 3DResNet based models use 3D convolution to learn the spatiotemporal features and apply sequential convolutional layers to enhance the temporal relationship of the outputs. The Sparse-UNet model uses sparse convolutions as the backbone for spatiotemporal feature learning. Since the latter algorithm mainly focuses on non-zero data points of the inputs, it dramatically reduces the computation time, while maintaining a competitive accuracy. Our results show that both of the proposed models achieve much better performance than the baseline algorithms. The codes and pretrained models are available at https://github.com/resuly/Traffic4Cast-2021. △ Less

Submitted 10 November, 2021; originally announced November 2021.

A novel activity pattern generation incorporating deep learning for transport demand models

Authors: Danh T. Phan, Hai L. Vu

Abstract: Activity generation plays an important role in activity-based demand modelling systems. While machine learning, especially deep learning, has been increasingly used for mode choice and traffic flow prediction, much less research exploiting the advantage of deep learning for activity generation tasks. This paper proposes a novel activity pattern generation framework by incorporating deep learning w… ▽ More Activity generation plays an important role in activity-based demand modelling systems. While machine learning, especially deep learning, has been increasingly used for mode choice and traffic flow prediction, much less research exploiting the advantage of deep learning for activity generation tasks. This paper proposes a novel activity pattern generation framework by incorporating deep learning with travel domain knowledge. We model each activity schedule as one primary activity tour and several secondary activity tours. We then develop different deep neural networks with entity embedding and random forest models to classify activity type, as well as to predict activity times. The proposed framework can capture the activity patterns for individuals in both training and validation sets. Results show high accuracy for the start time and end time of work and school activities. The framework also replicates the start time patterns of stop-before and stop-after primary work activity well. This provides a promising direction to deploy advanced machine learning methods to generate more reliable activity-travel patterns for transport demand systems and their applications. △ Less

Submitted 6 April, 2021; originally announced April 2021.

Comments: 21 pages, 12 figures

Boosted Genetic Algorithm using Machine Learning for traffic control optimization

Authors: Tuo Mao, Adriana-Simona Mihaita, Fang Chen, Hai L. Vu

Abstract: Traffic control optimization is a challenging task for various traffic centers around the world and the majority of existing approaches focus only on develo** adaptive methods under normal (recurrent) traffic conditions. Optimizing the control plans when severe incidents occur still remains an open problem, especially when a high number of lanes or entire intersections are affected. This paper… ▽ More Traffic control optimization is a challenging task for various traffic centers around the world and the majority of existing approaches focus only on develo** adaptive methods under normal (recurrent) traffic conditions. Optimizing the control plans when severe incidents occur still remains an open problem, especially when a high number of lanes or entire intersections are affected. This paper aims at tackling this problem and presents a novel methodology for optimizing the traffic signal timings in signalized urban intersections, under non-recurrent traffic incidents. With the purpose of producing fast and reliable decisions, we combine the fast running Machine Learning (ML) algorithms and the reliable Genetic Algorithms (GA) into a single optimization framework. As a benchmark, we first start with deploying a typical GA algorithm by considering the phase duration as the decision variable and the objective function to minimize the total travel time in the network. We fine tune the GA for crossover, mutation, fitness calculation and obtain the optimal parameters. Secondly, we train various machine learning regression models to predict the total travel time of the studied traffic network, and select the best performing regressor which we further hyper-tune to find the optimal training parameters. Lastly, we propose a new algorithm BGA-ML combining the GA algorithm and the extreme-gradient decision-tree, which is the best performing regressor, together in a single optimization framework. Comparison and results show that the new BGA-ML is much faster than the original GA algorithm and can be successfully applied under non-recurrent incident conditions. △ Less

Submitted 10 March, 2021; originally announced March 2021.

Comments: 26 pages, 28 figures. arXiv admin note: text overlap with arXiv:1906.05356

AutoAI-TS: AutoAI for Time Series Forecasting

Authors: Syed Yousaf Shah, Dhaval Patel, Long Vu, Xuan-Hong Dang, Bei Chen, Peter Kirchner, Horst Samulowitz, David Wood, Gregory Bramble, Wesley M. Gifford, Giridhar Ganapavarapu, Roman Vaculin, Petros Zerfos

Abstract: A large number of time series forecasting models including traditional statistical models, machine learning models and more recently deep learning have been proposed in the literature. However, choosing the right model along with good parameter values that performs well on a given data is still challenging. Automatically providing a good set of models to users for a given dataset saves both time a… ▽ More A large number of time series forecasting models including traditional statistical models, machine learning models and more recently deep learning have been proposed in the literature. However, choosing the right model along with good parameter values that performs well on a given data is still challenging. Automatically providing a good set of models to users for a given dataset saves both time and effort from using trial-and-error approaches with a wide variety of available models along with parameter optimization. We present AutoAI for Time Series Forecasting (AutoAI-TS) that provides users with a zero configuration (zero-conf ) system to efficiently train, optimize and choose best forecasting model among various classes of models for the given dataset. With its flexible zero-conf design, AutoAI-TS automatically performs all the data preparation, model creation, parameter optimization, training and model selection for users and provides a trained model that is ready to use. For given data, AutoAI-TS utilizes a wide variety of models including classical statistical models, Machine Learning (ML) models, statistical-ML hybrid models and deep learning models along with various transformations to create forecasting pipelines. It then evaluates and ranks pipelines using the proposed T-Daub mechanism to choose the best pipeline. The paper describe in detail all the technical aspects of AutoAI-TS along with extensive benchmarking on a variety of real world data sets for various use-cases. Benchmark results show that AutoAI-TS, with no manual configuration from the user, automatically trains and selects pipelines that on average outperform existing state-of-the-art time series forecasting toolkits. △ Less

Submitted 8 March, 2021; v1 submitted 24 February, 2021; originally announced February 2021.

Comments: Accepted for publication at ACM SIGMOD 2021 Industry Track

Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning

Authors: Sayak Mukherjee, Renke Huang, Qiuhua Huang, Thanh Long Vu, Tianzhixi Yin

Abstract: This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. DRL agents are trained for fast, and adaptive selection of control actions such that the voltage recovery criterion can be met following disturbances. Existing voltage control techniques suffer from the issues of speed of operation, optimal coordination between different… ▽ More This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. DRL agents are trained for fast, and adaptive selection of control actions such that the voltage recovery criterion can be met following disturbances. Existing voltage control techniques suffer from the issues of speed of operation, optimal coordination between different locations, and scalability. We exploit the area-wise division structure of the power system to propose a hierarchical DRL design that can be scaled to the larger grid models. We employ an enhanced augmented random search algorithm that is tailored for the voltage control problem in a two-level architecture. We train area-wise decentralized RL agents to compute lower-level policies for the individual areas, and concurrently train a higher-level DRL agent that uses the updates of the lower-level policies to efficiently coordinate the control actions taken by the lower-level agents. Numerical experiments on the IEEE benchmark 39-bus model with 3 areas demonstrate the advantages and various intricacies of the proposed hierarchical approach. △ Less

Submitted 29 January, 2021; originally announced February 2021.

Comments: 8 pages, 13 figures

Imposing Robust Structured Control Constraint on Reinforcement Learning of Linear Quadratic Regulator

Authors: Sayak Mukherjee, Thanh Long Vu

Abstract: This paper discusses learning a structured feedback control to obtain sufficient robustness to exogenous inputs for linear dynamic systems with unknown state matrix. The structural constraint on the controller is necessary for many cyber-physical systems, and our approach presents a design for any generic structure, paving the way for distributed learning control. The ideas from reinforcement lear… ▽ More This paper discusses learning a structured feedback control to obtain sufficient robustness to exogenous inputs for linear dynamic systems with unknown state matrix. The structural constraint on the controller is necessary for many cyber-physical systems, and our approach presents a design for any generic structure, paving the way for distributed learning control. The ideas from reinforcement learning (RL) in conjunction with control-theoretic sufficient stability and performance guarantees are used to develop the methodology. First, a model-based framework is formulated using dynamic programming to embed the structural constraint in the linear quadratic regulator (LQR) setting along with sufficient robustness conditions. Thereafter, we translate these conditions to a data-driven learning-based framework - robust structured reinforcement learning (RSRL) that enjoys the control-theoretic guarantees on stability and convergence. We validate our theoretical results with a simulation on a multi-agent network with 6 agents. △ Less

Submitted 19 February, 2021; v1 submitted 11 November, 2020; originally announced November 2020.

Comments: 16 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:2011.01128

Reinforcement Learning of Structured Control for Linear Systems with Unknown State Matrix

Authors: Sayak Mukherjee, Thanh Long Vu

Abstract: This paper delves into designing stabilizing feedback control gains for continuous linear systems with unknown state matrix, in which the control is subject to a general structural constraint. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of st… ▽ More This paper delves into designing stabilizing feedback control gains for continuous linear systems with unknown state matrix, in which the control is subject to a general structural constraint. We bring forth the ideas from reinforcement learning (RL) in conjunction with sufficient stability and performance guarantees in order to design these structured gains using the trajectory measurements of states and controls. We first formulate a model-based framework using dynamic programming (DP) to embed the structural constraint to the Linear Quadratic Regulator (LQR) gain computation in the continuous-time setting. Subsequently, we transform this LQR formulation into a policy iteration RL algorithm that can alleviate the requirement of known state matrix in conjunction with maintaining the feedback gain structure. Theoretical guarantees are provided for stability and convergence of the structured RL (SRL) algorithm. The introduced RL framework is general and can be applied to any control structure. A special control structure enabled by this RL framework is distributed learning control which is necessary for many large-scale cyber-physical systems. As such, we validate our theoretical results with numerical simulations on a multi-agent networked linear time-invariant (LTI) dynamic system. △ Less

Submitted 2 November, 2020; originally announced November 2020.

Comments: 13 pages, 7 figures

The two-echelon routing problem with truck and drones

Authors: Minh Hoàng Hà, Lam Vu, Duy Manh Vu

Abstract: In this paper, we study novel variants of the well-known two-echelon vehicle routing problem in which a truck works on the first echelon to transport parcels and a fleet of drones to intermediate depots while in the second echelon, the drones are used to deliver parcels from intermediate depots to customers. The objective is to minimize the completion time instead of the transportation cost as in… ▽ More In this paper, we study novel variants of the well-known two-echelon vehicle routing problem in which a truck works on the first echelon to transport parcels and a fleet of drones to intermediate depots while in the second echelon, the drones are used to deliver parcels from intermediate depots to customers. The objective is to minimize the completion time instead of the transportation cost as in classical 2-echelon vehicle routing problems. Depending on the context, a drone can be launched from the truck at an intermediate depot once (single trip drone) or several times (multiple trip drone). Mixed Integer Linear Programming (MILP) models are first proposed to formulate mathematically the problems and solve to optimality small-size instances. To handle larger instances, a metaheuristic based on the idea of Greedy Randomized Adaptive Search Procedure (GRASP) is introduced. Experimental results obtained on instances of different contexts are reported and analyzed. △ Less

Submitted 5 April, 2020; originally announced April 2020.

Comments: 29 pages, 4 figures

Policy Learning for Malaria Control

Authors: Van Bach Nguyen, Belaid Mohamed Karim, Bao Long Vu, Jörg Schlötterer, Michael Granitzer

Abstract: Sequential decision making is a typical problem in reinforcement learning with plenty of algorithms to solve it. However, only a few of them can work effectively with a very small number of observations. In this report, we introduce the progress to learn the policy for Malaria Control as a Reinforcement Learning problem in the KDD Cup Challenge 2019 and propose diverse solutions to deal with the l… ▽ More Sequential decision making is a typical problem in reinforcement learning with plenty of algorithms to solve it. However, only a few of them can work effectively with a very small number of observations. In this report, we introduce the progress to learn the policy for Malaria Control as a Reinforcement Learning problem in the KDD Cup Challenge 2019 and propose diverse solutions to deal with the limited observations problem. We apply the Genetic Algorithm, Bayesian Optimization, Q-learning with sequence breaking to find the optimal policy for five years in a row with only 20 episodes/100 evaluations. We evaluate those algorithms and compare their performance with Random Search as a baseline. Among these algorithms, Q-Learning with sequence breaking has been submitted to the challenge and got ranked 7th in KDD Cup. △ Less

Submitted 20 October, 2019; originally announced October 2019.

Don't relax: early stop** for convex regularization

Authors: Simon Matet, Lorenzo Rosasco, Silvia Villa, Bang Long Vu

Abstract: We consider the problem of designing efficient regularization algorithms when regularization is encoded by a (strongly) convex functional. Unlike classical penalization methods based on a relaxation approach, we propose an iterative method where regularization is achieved via early stop**. Our results show that the proposed procedure achieves the same recovery accuracy as penalization methods, w… ▽ More We consider the problem of designing efficient regularization algorithms when regularization is encoded by a (strongly) convex functional. Unlike classical penalization methods based on a relaxation approach, we propose an iterative method where regularization is achieved via early stop**. Our results show that the proposed procedure achieves the same recovery accuracy as penalization methods, while naturally integrating computational considerations. An empirical analysis on a number of problems provides promising results with respect to the state of the art. △ Less

Submitted 17 July, 2017; originally announced July 2017.

MSC Class: 47H05; 49M29; 49M27; 90C25

A strategic timing of arrivals to a linear slowdown processor sharing system

Authors: Liron Ravner, Moshe Haviv, Hai L. Vu

Abstract: We consider a discrete population of users with homogeneous service demand who need to decide when to arrive to a system in which the service rate deteriorates linearly with the number of users in the system. The users have heterogeneous desired departure times from the system, and their goal is to minimize a weighted sum of the travel time and square deviation from the desired departure times. Us… ▽ More We consider a discrete population of users with homogeneous service demand who need to decide when to arrive to a system in which the service rate deteriorates linearly with the number of users in the system. The users have heterogeneous desired departure times from the system, and their goal is to minimize a weighted sum of the travel time and square deviation from the desired departure times. Users join the system sequentially, according to the order of their desired departure times. We model this scenario as a non-cooperative game in which each user selects his actual arrival time. We present explicit equilibria solutions for a two-user example, namely the subgame perfect and Nash equilibria and show that multiple equilibria may exist. We further explain why a general solution for any number of users is computationally challenging. The difficulty lies in the fact that the objective functions are piecewise convex, i.e., non-smooth and non-convex. As a result, the minimization of the costs relies on checking all arrival and departure order permutations, which is exponentially large with respect to the population size. Instead we propose an iterated best-response algorithm which can be efficiently studied numerically. Finally, we compare the equilibrium arrival profiles to a socially optimal solution and discuss the implications. △ Less

Submitted 17 May, 2016; v1 submitted 14 August, 2015; originally announced August 2015.

The age of information in gossip networks

Authors: Jori Selen, Yoni Nazarathy, Lachlan L. H. Andrew, Hai L. Vu

Abstract: We introduce models of gossip based communication networks in which each node is simultaneously a sensor, a relay and a user of information. We model the status of ages of information between nodes as a discrete time Markov chain. In this setting a gossip transmission policy is a decision made at each node regarding what type of information to relay at any given time (if any). When transmission po… ▽ More We introduce models of gossip based communication networks in which each node is simultaneously a sensor, a relay and a user of information. We model the status of ages of information between nodes as a discrete time Markov chain. In this setting a gossip transmission policy is a decision made at each node regarding what type of information to relay at any given time (if any). When transmission policies are based on random decisions, we are able to analyze the age of information in certain illustrative structured examples either by means of an explicit analysis, an algorithm or asymptotic approximations. Our key contribution is presenting this class of models. △ Less

Submitted 29 October, 2013; originally announced October 2013.

Comments: 15 pages, 8 figures

Journal ref: In: Analytical and Stochastic Modeling Techniques and Applications, pages 364-379, Springer Berlin Heidelberg, 2013