-
Koopman Invertible Autoencoder: Leveraging Forward and Backward Dynamics for Temporal Modeling
Authors:
Kshitij Tayal,
Arvind Renganathan,
Rahul Ghosh,
Xiaowei Jia,
Vipin Kumar
Abstract:
Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. However, building accurate long-term prediction models remains challenging due to the limitations of existing temporal models like recurrent neural networks (RNNs), as they capture only the statistical connections in the training data and may fail to learn the underlying dynamic…
▽ More
Accurate long-term predictions are the foundations for many machine learning applications and decision-making processes. However, building accurate long-term prediction models remains challenging due to the limitations of existing temporal models like recurrent neural networks (RNNs), as they capture only the statistical connections in the training data and may fail to learn the underlying dynamics of the target system. To tackle this challenge, we propose a novel machine learning model based on Koopman operator theory, which we call Koopman Invertible Autoencoders (KIA), that captures the inherent characteristic of the system by modeling both forward and backward dynamics in the infinite-dimensional Hilbert space. This enables us to efficiently learn low-dimensional representations, resulting in more accurate predictions of long-term system behavior. Moreover, our method's invertibility design guarantees reversibility and consistency in both forward and inverse operations. We illustrate the utility of KIA on pendulum and climate datasets, demonstrating 300% improvements in long-term prediction capability for pendulum while maintaining robustness against noise. Additionally, our method excels in long-term climate prediction, further validating our method's effectiveness.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
Using Monte Carlo Methods for Retirement Simulations
Authors:
Aditya Gupta,
Vijay K. Tayal
Abstract:
Retirement prediction helps individuals and institutions make informed financial, lifestyle, and workforce decisions based on estimated retirement portfolios. This paper attempts to predict retirement using Monte Carlo simulations, allowing one to probabilistically account for a range of possibilities. The authors propose a model to predict the values of the investment accounts IRA and 401(k) thro…
▽ More
Retirement prediction helps individuals and institutions make informed financial, lifestyle, and workforce decisions based on estimated retirement portfolios. This paper attempts to predict retirement using Monte Carlo simulations, allowing one to probabilistically account for a range of possibilities. The authors propose a model to predict the values of the investment accounts IRA and 401(k) through the simulation of inflation rates, interest rates, and other pertinent factors. They provide a user case study to discuss the implications of the proposed model.
△ Less
Submitted 20 November, 2023; v1 submitted 28 June, 2023;
originally announced June 2023.
-
Robust Inverse Framework using Knowledge-guided Self-Supervised Learning: An application to Hydrology
Authors:
Rahul Ghosh,
Arvind Renganathan,
Kshitij Tayal,
Xiang Li,
Ankush Khandelwal,
Xiaowei Jia,
Chris Duffy,
John Neiber,
Vipin Kumar
Abstract:
Machine Learning is beginning to provide state-of-the-art performance in a range of environmental applications such as streamflow prediction in a hydrologic basin. However, building accurate broad-scale models for streamflow remains challenging in practice due to the variability in the dominant hydrologic processes, which are best captured by sets of process-related basin characteristics. Existing…
▽ More
Machine Learning is beginning to provide state-of-the-art performance in a range of environmental applications such as streamflow prediction in a hydrologic basin. However, building accurate broad-scale models for streamflow remains challenging in practice due to the variability in the dominant hydrologic processes, which are best captured by sets of process-related basin characteristics. Existing basin characteristics suffer from noise and uncertainty, among many other things, which adversely impact model performance. To tackle the above challenges, in this paper, we propose a novel Knowledge-guided Self-Supervised Learning (KGSSL) inverse framework to extract system characteristics from driver and response data. This first-of-its-kind framework achieves robust performance even when characteristics are corrupted. We show that KGSSL achieves state-of-the-art results for streamflow modeling for CAMELS (Catchment Attributes and MEteorology for Large-sample Studies) which is a widely used hydrology benchmark dataset. Specifically, KGSSL outperforms other methods by up to 16 \% in reconstructing characteristics. Furthermore, we show that KGSSL is relatively more robust to distortion than baseline methods, and outperforms the baseline model by 35\% when plugging in KGSSL inferred characteristics.
△ Less
Submitted 8 June, 2022; v1 submitted 14 September, 2021;
originally announced September 2021.
-
Phase Retrieval using Single-Instance Deep Generative Prior
Authors:
Kshitij Tayal,
Raunak Manekar,
Zhong Zhuang,
David Yang,
Vipin Kumar,
Felix Hofmann,
Ju Sun
Abstract:
Several deep learning methods for phase retrieval exist, but most of them fail on realistic data without precise support information. We propose a novel method based on single-instance deep generative prior that works well on complex-valued crystal data.
Several deep learning methods for phase retrieval exist, but most of them fail on realistic data without precise support information. We propose a novel method based on single-instance deep generative prior that works well on complex-valued crystal data.
△ Less
Submitted 22 June, 2021; v1 submitted 9 June, 2021;
originally announced June 2021.
-
Noise Robust Named Entity Understanding for Voice Assistants
Authors:
Deepak Muralidharan,
Joel Ruben Antony Moniz,
Sida Gao,
Xiao Yang,
Justine Kao,
Stephen Pulman,
Atish Kothari,
Ray Shen,
Yinying Pan,
Vivek Kaul,
Mubarak Seyed Ibrahim,
Gang Xiang,
Nan Dun,
Yidan Zhou,
Andy O,
Yuan Zhang,
Pooja Chitkara,
Xuan Wang,
Alkesh Patel,
Kushal Tayal,
Roger Zheng,
Peter Grasch,
Jason D. Williams,
Lin Li
Abstract:
Named Entity Recognition (NER) and Entity Linking (EL) play an essential role in voice assistant interaction, but are challenging due to the special difficulties associated with spoken user queries. In this paper, we propose a novel architecture that jointly solves the NER and EL tasks by combining them in a joint reranking module. We show that our proposed framework improves NER accuracy by up to…
▽ More
Named Entity Recognition (NER) and Entity Linking (EL) play an essential role in voice assistant interaction, but are challenging due to the special difficulties associated with spoken user queries. In this paper, we propose a novel architecture that jointly solves the NER and EL tasks by combining them in a joint reranking module. We show that our proposed framework improves NER accuracy by up to 3.13% and EL accuracy by up to 3.6% in F1 score. The features used also lead to better accuracies in other natural language understanding tasks, such as domain classification and semantic parsing.
△ Less
Submitted 10 August, 2021; v1 submitted 29 May, 2020;
originally announced May 2020.
-
Inverse Problems, Deep Learning, and Symmetry Breaking
Authors:
Kshitij Tayal,
Chieh-Hsin Lai,
Vipin Kumar,
Ju Sun
Abstract:
In many physical systems, inputs related by intrinsic system symmetries are mapped to the same output. When inverting such systems, i.e., solving the associated inverse problems, there is no unique solution. This causes fundamental difficulties for deploying the emerging end-to-end deep learning approach. Using the generalized phase retrieval problem as an illustrative example, we show that carefu…
▽ More
In many physical systems, inputs related by intrinsic system symmetries are mapped to the same output. When inverting such systems, i.e., solving the associated inverse problems, there is no unique solution. This causes fundamental difficulties for deploying the emerging end-to-end deep learning approach. Using the generalized phase retrieval problem as an illustrative example, we show that careful symmetry breaking on the training data can help get rid of the difficulties and significantly improve the learning performance. We also extract and highlight the underlying mathematical principle of the proposed solution, which is directly applicable to other inverse problems.
△ Less
Submitted 19 March, 2020;
originally announced March 2020.
