Showing 1–50 of 297 results for author: Ying, L

A Randomized Method for Simulating Lindblad Equations and Thermal State Preparation

Authors: Hongrui Chen, Bowen Li, Jianfeng Lu, Lexing Ying

Abstract: We study a qDRIFT-type randomized method to simulate the Lindblad equations. For Lindblad dynamics generated by an ensemble of Lindbladians $\{\mathcal{L}_a\}_{a \in \mathcal{A}}$, our approach implements a single randomly sampled Lindbladian $\mathcal{L}_a$ at each time step. The only assumption is that each $\mathcal{L}_a$ involves only a single jump operator with an efficient implementation ava… ▽ More We study a qDRIFT-type randomized method to simulate the Lindblad equations. For Lindblad dynamics generated by an ensemble of Lindbladians $\{\mathcal{L}_a\}_{a \in \mathcal{A}}$, our approach implements a single randomly sampled Lindbladian $\mathcal{L}_a$ at each time step. The only assumption is that each $\mathcal{L}_a$ involves only a single jump operator with an efficient implementation available for the evolution $e^{t \mathcal{L}_a}$. A notable application of the randomized method is for quantum Gibbs sampling, where the Lindblad dynamics is utilized to prepare a specific Gibbs state. Unlike existing deterministic methods that require numerous jump operators to ensure ergodicity, our approach simplifies the implementation by using a single randomly sampled jump operator. As an example, we demonstrate that our method ensures fast thermalization of Hamiltonian systems characterized by random Pauli strings, where the spectral density closely adheres to the semi-circle law. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Comments: 23 pages

MOSP: A User-interface Package for Simulating Metal Nanoparticle Structure and Reactivity under Operando Conditions

Authors: Lei Ying, Beien Zhu, Yi Gao

Abstract: Structures of metal nanoparticles (NPs) significantly influence their catalytic reactivities. Recent in situ experimental observations of dramatic structural changes in NPs have underscored the need to establish a dynamic structure-property relationship that accounts for the reconstruction of NPs in reactive environments. Here, we present the MOSP, a free and open-source graphical user interface (… ▽ More Structures of metal nanoparticles (NPs) significantly influence their catalytic reactivities. Recent in situ experimental observations of dramatic structural changes in NPs have underscored the need to establish a dynamic structure-property relationship that accounts for the reconstruction of NPs in reactive environments. Here, we present the MOSP, a free and open-source graphical user interface (GUI) package designed to simulate the structure and reactivity of metal NPs under operando conditions. MOSP integrates two models: the multiscale structure reconstruction (MSR) model predicting equilibrium metal NP structures under specific reaction conditions and the kinetic Monte Carlo (KMC) model simulating the reaction dynamics. This combination allows for the exploration of the dynamic structure-property relationships of NPs. MOSP enhances user accessibility through its intuitive GUI, facilitating easy input, post-processing, and visualization of simulation data. This article is the release note of MOSP, focusing on its implementation and functionality. △ Less

Submitted 1 July, 2024; v1 submitted 28 June, 2024; originally announced June 2024.

Comments: 14 pages, 6 figures

A Bayesian Drift-Diffusion Model of Schachter-Singer's Two Factor Theory of Emotion

Authors: Lance Ying, Audrey Michal, Jun Zhang

Abstract: Bayesian inference has been used in the past to model visual perception (Kersten, Mamassian, & Yuille, 2004), accounting for the Helmholtz principle of perception as "unconscious inference" that is constrained by bottom-up sensory evidence (likelihood) while subject to top-down expectation, priming, or other contextual influences (prior bias); here "unconsciousness" merely relates to the "directne… ▽ More Bayesian inference has been used in the past to model visual perception (Kersten, Mamassian, & Yuille, 2004), accounting for the Helmholtz principle of perception as "unconscious inference" that is constrained by bottom-up sensory evidence (likelihood) while subject to top-down expectation, priming, or other contextual influences (prior bias); here "unconsciousness" merely relates to the "directness" of perception in the sense of Gibson. Here, we adopt the same Bayesian framework to model emotion process in accordance with Schachter-Singer's Two-Factor theory, which argues that emotion is the outcome of cognitive labeling or attribution of a diffuse pattern of autonomic arousal (Schachter & Singer, 1962). In analogous to visual perception, we conceptualize the emotion process as an instance of Bayesian inference, combining the contextual information with a person's physiological arousal patterns. Drift-diffusion models were constructed to simulate emotional processes, where the decision boundaries correspond to the emotional state experienced by the participants, and boundary-crossing constitutes "labeling" in Schachter-Singer's sense. Our model is tested against experimental data from the Schachter & Singer's study (1962) and the Ross et al. study (1969). Two model scenarios are investigated, in which arousal pattern as one factor is pitted against contextual interaction with an confederate (in Schachter-Singer case) or explicitly instructed mis-attribution (in Ross et al. case) as another factor, map** onto the Bayesian prior (initial position of the drift) and the likelihood function (evidence accumulation or drift rate). We find that the first scenario (arousal as the prior and context as the likelihood) has a better fit with Schachter & Singer (1962) whereas the second scenario (context as the prior and arousal as the likelihood) has a better fit with Ross et al. (1969). △ Less

Submitted 16 June, 2024; originally announced June 2024.

Comments: published at CogSci 2022

Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysis

Authors: Qining Zhang, Honghao Wei, Lei Ying

Abstract: In this paper, we study reinforcement learning from human feedback (RLHF) under an episodic Markov decision process with a general trajectory-wise reward model. We developed a model-free RLHF best policy identification algorithm, called $\mathsf{BSAD}$, without explicit reward model inference, which is a critical intermediate step in the contemporary RLHF paradigms for training large language mode… ▽ More In this paper, we study reinforcement learning from human feedback (RLHF) under an episodic Markov decision process with a general trajectory-wise reward model. We developed a model-free RLHF best policy identification algorithm, called $\mathsf{BSAD}$, without explicit reward model inference, which is a critical intermediate step in the contemporary RLHF paradigms for training large language models (LLM). The algorithm identifies the optimal policy directly from human preference information in a backward manner, employing a dueling bandit sub-routine that constantly duels actions to identify the superior one. $\mathsf{BSAD}$ adopts a reward-free exploration and best-arm-identification-like adaptive stop** criteria to equalize the visitation among all states in the same decision step while moving to the previous step as soon as the optimal action is identifiable, leading to a provable, instance-dependent sample complexity $\tilde{\mathcal{O}}(c_{\mathcal{M}}SA^3H^3M\log\frac{1}δ)$ which resembles the result in classic RL, where $c_{\mathcal{M}}$ is the instance-dependent constant and $M$ is the batch size. Moreover, $\mathsf{BSAD}$ can be transformed into an explore-then-commit algorithm with logarithmic regret and generalized to discounted MDPs using a frame-based approach. Our results show: (i) sample-complexity-wise, RLHF is not significantly harder than classic RL and (ii) end-to-end RLHF may deliver improved performance by avoiding pitfalls in reward inferring such as overfit and distribution shift. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Few-Body Quantum Chaos, Localization, and Multi-Photon Entanglement in Optical Synthetic Frequency Dimension

Authors: Junlin Wang, Luojia Wang, **lou Ma, Ang Yang, Luqi Yuan, Lei Ying

Abstract: Generation and control of entanglement are fundamental tasks in quantum information processing. In this paper, we propose a novel approach to generate controllable frequency-entangled photons by using the concept of synthetic frequency dimension in an optical system. Such a system consists of a ring resonator made by a tailored third-order nonlinear media to induce photon-photon interactions and a… ▽ More Generation and control of entanglement are fundamental tasks in quantum information processing. In this paper, we propose a novel approach to generate controllable frequency-entangled photons by using the concept of synthetic frequency dimension in an optical system. Such a system consists of a ring resonator made by a tailored third-order nonlinear media to induce photon-photon interactions and a periodic modulator to manipulate coupling between different frequency modes. We show this system provides a unique platform for the exploration of distinct few- or many-body quantum phases including chaos, localization, and integrability in a highly integrable photonics platform. In particular, we develop the potential experimental method to calculate the spectral form factor, which characterizes the degree of chaos in the system and differentiates between these phases based on observable measurements. Interestingly, the transition signatures of each phase can lead to an efficient generation of frequency-entangled multi photons. This work is the first to explore rich and controllable quantum phases beyond single particle in a synthetic dimension. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 15 pages, 8 figures

Tangent differential privacy

Authors: Lexing Ying

Abstract: Differential privacy is a framework for protecting the identity of individual data points in the decision-making process. In this note, we propose a new form of differential privacy called tangent differential privacy. Compared with the usual differential privacy that is defined uniformly across data distributions, tangent differential privacy is tailored towards a specific data distribution of in… ▽ More Differential privacy is a framework for protecting the identity of individual data points in the decision-making process. In this note, we propose a new form of differential privacy called tangent differential privacy. Compared with the usual differential privacy that is defined uniformly across data distributions, tangent differential privacy is tailored towards a specific data distribution of interest. It also allows for general distribution distances such as total variation distance and Wasserstein distance. In the case of risk minimization, we show that entropic regularization guarantees tangent differential privacy under rather general conditions on the risk function. △ Less

Submitted 6 June, 2024; originally announced June 2024.

PowerPeeler: A Precise and General Dynamic Deobfuscation Method for PowerShell Scripts

Authors: Ruijie Li, Chenyang Zhang, Huajun Chai, Lingyun Ying, Haixin Duan, Jun Tao

Abstract: PowerShell is a powerful and versatile task automation tool. Unfortunately, it is also widely abused by cyber attackers. To bypass malware detection and hinder threat analysis, attackers often employ diverse techniques to obfuscate malicious PowerShell scripts. Existing deobfuscation tools suffer from the limitation of static analysis, which fails to simulate the real deobfuscation process accurat… ▽ More PowerShell is a powerful and versatile task automation tool. Unfortunately, it is also widely abused by cyber attackers. To bypass malware detection and hinder threat analysis, attackers often employ diverse techniques to obfuscate malicious PowerShell scripts. Existing deobfuscation tools suffer from the limitation of static analysis, which fails to simulate the real deobfuscation process accurately. In this paper, we propose PowerPeeler. To the best of our knowledge, it is the first dynamic PowerShell script deobfuscation approach at the instruction level. It utilizes expression-related Abstract Syntax Tree (AST) nodes to identify potential obfuscated script pieces. Then, PowerPeeler correlates the AST nodes with their corresponding instructions and monitors the script's entire execution process. Subsequently, PowerPeeler dynamically tracks the execution of these instructions and records their execution results. Finally, PowerPeeler stringifies these results to replace the corresponding obfuscated script pieces and reconstruct the deobfuscated script. To evaluate the effectiveness of PowerPeeler, we collect 1,736,669 real-world malicious PowerShell samples with diversity obfuscation methods. We compare PowerPeeler with five state-of-the-art deobfuscation tools and GPT-4. The evaluation results demonstrate that PowerPeeler can effectively handle all well-known obfuscation methods. Additionally, the deobfuscation correctness rate of PowerPeeler reaches 95%, significantly surpassing that of other tools. PowerPeeler not only recovers the highest amount of sensitive data but also maintains a semantic consistency over 97%, which is also the best. Moreover, PowerPeeler effectively obtains the largest quantity of valid deobfuscated results within a limited time frame. Furthermore, PowerPeeler is extendable and can be used as a helpful tool for other cyber security solutions. △ Less

Submitted 19 June, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Comments: To appear in the ACM CCS 2024

Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity

Authors: Haoxuan Chen, Yinuo Ren, Lexing Ying, Grant M. Rotskoff

Abstract: Diffusion models have become a leading method for generative modeling of both image and scientific data. As these models are costly to train and evaluate, reducing the inference cost for diffusion models remains a major goal. Inspired by the recent empirical success in accelerating diffusion models via the parallel sampling technique~\cite{shih2024parallel}, we propose to divide the sampling proce… ▽ More Diffusion models have become a leading method for generative modeling of both image and scientific data. As these models are costly to train and evaluate, reducing the inference cost for diffusion models remains a major goal. Inspired by the recent empirical success in accelerating diffusion models via the parallel sampling technique~\cite{shih2024parallel}, we propose to divide the sampling process into $\mathcal{O}(1)$ blocks with parallelizable Picard iterations within each block. Rigorous theoretical analysis reveals that our algorithm achieves $\widetilde{\mathcal{O}}(\mathrm{poly} \log d)$ overall time complexity, marking the first implementation with provable sub-linear complexity w.r.t. the data dimension $d$. Our analysis is based on a generalized version of Girsanov's theorem and is compatible with both the SDE and probability flow ODE implementations. Our results shed light on the potential of fast and efficient sampling of high-dimensional data on fast-evolving modern large-memory GPU clusters. △ Less

Submitted 24 May, 2024; originally announced May 2024.

RCInvestigator: Towards Better Investigation of Anomaly Root Causes in Cloud Computing Systems

Authors: Shuhan Liu, Yunfan Zhou, Lu Ying, Yuan Tian, Jue Zhang, Shandan Zhou, Weiwei Cui, Qingwei Lin, Thomas Moscibroda, Haidong Zhang, Di Weng, Yingcai Wu

Abstract: Finding the root causes of anomalies in cloud computing systems quickly is crucial to ensure availability and efficiency since accurate root causes can guide engineers to take appropriate actions to address the anomalies and maintain customer satisfaction. However, it is difficult to investigate and identify the root causes based on large-scale and high-dimension monitoring data collected from com… ▽ More Finding the root causes of anomalies in cloud computing systems quickly is crucial to ensure availability and efficiency since accurate root causes can guide engineers to take appropriate actions to address the anomalies and maintain customer satisfaction. However, it is difficult to investigate and identify the root causes based on large-scale and high-dimension monitoring data collected from complex cloud computing environments. Due to the inherently dynamic characteristics of cloud computing systems, the existing approaches in practice largely rely on manual analyses for flexibility and reliability, but massive unpredictable factors and high data complexity make the process time-consuming. Despite recent advances in automated detection and investigation approaches, the speed and quality of root cause analyses remain limited by the lack of expert involvement in these approaches. The limitations found in the current solutions motivate us to propose a visual analytics approach that facilitates the interactive investigation of the anomaly root causes in cloud computing systems. We identified three challenges, namely, a) modeling databases for the root cause investigation, b) inferring root causes from large-scale time series, and c) building comprehensible investigation results. In collaboration with domain experts, we addressed these challenges with RCInvestigator, a novel visual analytics system that establishes a tight collaboration between human and machine and assists experts in investigating the root causes of cloud computing system anomalies. We evaluated the effectiveness of RCInvestigator through two use cases based on real-world data and received positive feedback from experts. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Policy Gradient Methods for Risk-Sensitive Distributional Reinforcement Learning with Provable Convergence

Authors: Minheng Xiao, Xian Yu, Lei Ying

Abstract: Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in many high-stakes applications. While most RL methods aim to learn a point estimate of the random cumulative cost, distributional RL (DRL) seeks to estimate the entire distribution of it. The distribution provides all necessary information about the cost and leads to a unified framework for handling variou… ▽ More Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in many high-stakes applications. While most RL methods aim to learn a point estimate of the random cumulative cost, distributional RL (DRL) seeks to estimate the entire distribution of it. The distribution provides all necessary information about the cost and leads to a unified framework for handling various risk measures in a risk-sensitive setting. However, develo** policy gradient methods for risk-sensitive DRL is inherently more complex as it pertains to finding the gradient of a probability measure. This paper introduces a policy gradient method for risk-sensitive DRL with general coherent risk measures, where we provide an analytical form of the probability measure's gradient. We further prove the local convergence of the proposed algorithm under mild smoothness assumptions. For practical use, we also design a categorical distributional policy gradient algorithm (CDPG) based on categorical distributional policy evaluation and trajectory-based gradient estimation. Through experiments on a stochastic cliff-walking environment, we illustrate the benefits of considering a risk-sensitive setting in DRL. △ Less

Submitted 23 May, 2024; originally announced May 2024.

A note on continuous-time online learning

Authors: Lexing Ying

Abstract: In online learning, the data is provided in a sequential order, and the goal of the learner is to make online decisions to minimize overall regrets. This note is concerned with continuous-time models and algorithms for several online learning problems: online linear optimization, adversarial bandit, and adversarial linear bandit. For each problem, we extend the discrete-time algorithm to the conti… ▽ More In online learning, the data is provided in a sequential order, and the goal of the learner is to make online decisions to minimize overall regrets. This note is concerned with continuous-time models and algorithms for several online learning problems: online linear optimization, adversarial bandit, and adversarial linear bandit. For each problem, we extend the discrete-time algorithm to the continuous-time setting and provide a concise proof of the optimal regret bound. △ Less

Submitted 16 May, 2024; originally announced May 2024.

Quantum wave packet transforms with compact frequency support

Authors: Hongkang Ni, Lexing Ying

Abstract: Different kinds of wave packet transforms are widely used for extracting multi-scale structures in signal processing tasks. This paper introduces the quantum circuit implementation of a broad class of wave packets, including Gabor atoms and wavelets, with compact frequency support. Our approach operates in the frequency space, involving reallocation and reshuffling of signals tailored for manipula… ▽ More Different kinds of wave packet transforms are widely used for extracting multi-scale structures in signal processing tasks. This paper introduces the quantum circuit implementation of a broad class of wave packets, including Gabor atoms and wavelets, with compact frequency support. Our approach operates in the frequency space, involving reallocation and reshuffling of signals tailored for manipulation on quantum computers. The resulting implementation is different from the existing quantum algorithms for spatially compactly supported wavelets and can be readily extended to quantum transforms of other wave packets with compact frequency support. △ Less

Submitted 3 May, 2024; v1 submitted 1 May, 2024; originally announced May 2024.

A perturbative analysis for noisy spectral estimation

Authors: Lexing Ying

Abstract: Spectral estimation is a fundamental task in signal processing. Recent algorithms in quantum phase estimation are concerned with the large noise, large frequency regime of the spectral estimation problem. The recent work in Ding-Epperly-Lin-Zhang shows that the ESPRIT algorithm exhibits superconvergence behavior for the spike locations in terms of the maximum frequency. This note provides a pertur… ▽ More Spectral estimation is a fundamental task in signal processing. Recent algorithms in quantum phase estimation are concerned with the large noise, large frequency regime of the spectral estimation problem. The recent work in Ding-Epperly-Lin-Zhang shows that the ESPRIT algorithm exhibits superconvergence behavior for the spike locations in terms of the maximum frequency. This note provides a perturbative analysis to explain this behavior. It also extends the discussion to the case where the noise grows with the sampling frequency. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Orthogonal Bootstrap: Efficient Simulation of Input Uncertainty

Authors: Kaizhao Liu, Jose Blanchet, Lexing Ying, Yi** Lu

Abstract: Bootstrap is a popular methodology for simulating input uncertainty. However, it can be computationally expensive when the number of samples is large. We propose a new approach called \textbf{Orthogonal Bootstrap} that reduces the number of required Monte Carlo replications. We decomposes the target being simulated into two parts: the \textit{non-orthogonal part} which has a closed-form result kno… ▽ More Bootstrap is a popular methodology for simulating input uncertainty. However, it can be computationally expensive when the number of samples is large. We propose a new approach called \textbf{Orthogonal Bootstrap} that reduces the number of required Monte Carlo replications. We decomposes the target being simulated into two parts: the \textit{non-orthogonal part} which has a closed-form result known as Infinitesimal Jackknife and the \textit{orthogonal part} which is easier to be simulated. We theoretically and numerically show that Orthogonal Bootstrap significantly reduces the computational cost of Bootstrap while improving empirical accuracy and maintaining the same width of the constructed interval. △ Less

Submitted 30 April, 2024; v1 submitted 29 April, 2024; originally announced April 2024.

Solving high-dimensional Kolmogorov backward equations with functional hierarchical tensor operators

Authors: Xun Tang, Leah Collis, Lexing Ying

Abstract: Solving high-dimensional partial differential equations necessitates methods free of exponential scaling in the dimension of the problem. This work introduces a tensor network approach for the Kolmogorov backward equation via approximating directly the Markov operator. We show that the high-dimensional Markov operator can be obtained under a functional hierarchical tensor (FHT) ansatz with a hiera… ▽ More Solving high-dimensional partial differential equations necessitates methods free of exponential scaling in the dimension of the problem. This work introduces a tensor network approach for the Kolmogorov backward equation via approximating directly the Markov operator. We show that the high-dimensional Markov operator can be obtained under a functional hierarchical tensor (FHT) ansatz with a hierarchical sketching algorithm. When the terminal condition admits an FHT ansatz, the proposed operator outputs an FHT ansatz for the PDE solution through an efficient functional tensor network contraction procedure. In addition, the proposed operator-based approach also provides an efficient way to solve the Kolmogorov forward equation when the initial distribution is in an FHT ansatz. We apply the proposed approach successfully to two challenging time-dependent Ginzburg-Landau models with hundreds of variables. △ Less

Submitted 22 April, 2024; v1 submitted 12 April, 2024; originally announced April 2024.

MSC Class: 65M99; 15A69; 65F99

Emergent Anomalous Hydrodynamics at Infinite Temperature in a Long-Range XXZ Model

Authors: Ang Yang, **lou Ma, Lei Ying

Abstract: The conventional wisdom suggests that transports of conserved quantities in non-integrable quantum many-body systems at high temperatures are diffusive. However, we discover a counterexample of this paradigm by uncovering anomalous hydrodynamics in a spin-1/2 XXZ chain with power-law couplings. This model, classified as non-integrable due to its Wigner-Dyson level-spacing statistics in the random… ▽ More The conventional wisdom suggests that transports of conserved quantities in non-integrable quantum many-body systems at high temperatures are diffusive. However, we discover a counterexample of this paradigm by uncovering anomalous hydrodynamics in a spin-1/2 XXZ chain with power-law couplings. This model, classified as non-integrable due to its Wigner-Dyson level-spacing statistics in the random matrix theory, exhibits a surprising superdiffusive-ballistic-superdiffusive transport transition by varying the power-law exponent of couplings for a fixed anisotropy. Our findings are verified by multiple observables, including the spin-spin autocorrelator, mean-square displacement, and spin conductivity. Interestingly, we further quantify the degree of quantum chaos using the Kullback-Leibler divergence between the entanglement entropy distributions of the model's eigenstates and a random state. Remarkably, an observed local maximum in the divergence near the transition boundary suggests a link between anomalous hydrodynamics and a suppression of quantum chaos. This work offers another deep understanding of emergent anomalous transport phenomena in a wider range of non-integrable quantum many-body systems △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: 12 pages, 10 figures

Measuring Spectral Form Factor in Many-Body Chaotic and Localized Phases of Quantum Processors

Authors: Hang Dong, Pengfei Zhang, Ceren B. Dag, Yu Gao, Ning Wang, **feng Deng, Xu Zhang, Jiachen Chen, Shibo Xu, Ke Wang, Yaozu Wu, Chuanyu Zhang, Feitong **, Xuhao Zhu, Aosai Zhang, Yiren Zou, Ziqi Tan, Zhengyi Cui, Zitian Zhu, Fanhao Shen, Tingting Li, Jiarun Zhong, Zehang Bao, Hekang Li, Zhen Wang , et al. (6 additional authors not shown)

Abstract: The spectral form factor (SFF) captures universal spectral fluctuations as signatures of quantum chaos, and has been instrumental in advancing multiple frontiers of physics including the studies of black holes and quantum many-body systems. However, the measurement of SFF in many-body systems is challenging due to the difficulty in resolving level spacings that become exponentially small with incr… ▽ More The spectral form factor (SFF) captures universal spectral fluctuations as signatures of quantum chaos, and has been instrumental in advancing multiple frontiers of physics including the studies of black holes and quantum many-body systems. However, the measurement of SFF in many-body systems is challenging due to the difficulty in resolving level spacings that become exponentially small with increasing system size. Here we experimentally measure the SFF to probe the presence or absence of chaos in quantum many-body systems using a superconducting quantum processor with a randomized measurement protocol. For a Floquet chaotic system, we observe signatures of spectral rigidity of random matrix theory in SFF given by the ramp-plateau behavior. For a Hamiltonian system, we utilize SFF to distinguish the quantum many-body chaotic phase and the prethermal many-body localization. We observe the dip-ramp-plateau behavior of random matrix theory in the chaotic phase, and contrast the scaling of the plateau time in system size between the many-body chaotic and localized phases. Furthermore, we probe the eigenstate statistics by measuring a generalization of the SFF, known as the partial SFF, and observe distinct behaviors in the purities of the reduced density matrix in the two phases. This work unveils a new way of extracting the universal signatures of many-body quantum chaos in quantum devices by probing the correlations in eigenenergies and eigenstates. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: 12 pages, 9 figures

Learning-Based Pricing and Matching for Two-Sided Queues

Authors: Zixian Yang, Lei Ying

Abstract: We consider a dynamic system with multiple types of customers and servers. Each type of waiting customer or server joins a separate queue, forming a bipartite graph with customer-side queues and server-side queues. The platform can match the servers and customers if their types are compatible. The matched pairs then leave the system. The platform will charge a customer a price according to their t… ▽ More We consider a dynamic system with multiple types of customers and servers. Each type of waiting customer or server joins a separate queue, forming a bipartite graph with customer-side queues and server-side queues. The platform can match the servers and customers if their types are compatible. The matched pairs then leave the system. The platform will charge a customer a price according to their type when they arrive and will pay a server a price according to their type. The arrival rate of each queue is determined by the price according to some unknown demand or supply functions. Our goal is to design pricing and matching algorithms to maximize the profit of the platform with unknown demand and supply functions, while kee** queue lengths of both customers and servers below a predetermined threshold. This system can be used to model two-sided markets such as ride-sharing markets with passengers and drivers. The difficulties of the problem include simultaneous learning and decision making, and the tradeoff between maximizing profit and minimizing queue length. We use a longest-queue-first matching algorithm and propose a learning-based pricing algorithm, which combines gradient-free stochastic projected gradient ascent with bisection search. We prove that our proposed algorithm yields a sublinear regret $\tilde{O}(T^{5/6})$ and queue-length bound $\tilde{O}(T^{2/3})$, where $T$ is the time horizon. We further establish a tradeoff between the regret bound and the queue-length bound: $\tilde{O}(T^{1-γ/4})$ versus $\tilde{O}(T^γ)$ for $γ\in (0, 2/3].$ △ Less

Submitted 17 March, 2024; originally announced March 2024.

Comments: 48 pages, 8 figures

GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment

Authors: Lance Ying, Kunal Jha, Shivam Aarya, Joshua B. Tenenbaum, Antonio Torralba, Tianmin Shu

Abstract: Verbal communication plays a crucial role in human cooperation, particularly when the partners only have incomplete information about the task, environment, and each other's mental state. In this paper, we propose a novel cooperative communication framework, Goal-Oriented Mental Alignment (GOMA). GOMA formulates verbal communication as a planning problem that minimizes the misalignment between the… ▽ More Verbal communication plays a crucial role in human cooperation, particularly when the partners only have incomplete information about the task, environment, and each other's mental state. In this paper, we propose a novel cooperative communication framework, Goal-Oriented Mental Alignment (GOMA). GOMA formulates verbal communication as a planning problem that minimizes the misalignment between the parts of agents' mental states that are relevant to the goals. This approach enables an embodied assistant to reason about when and how to proactively initialize communication with humans verbally using natural language to help achieve better cooperation. We evaluate our approach against strong baselines in two challenging environments, Overcooked (a multiplayer game) and VirtualHome (a household simulator). Our experimental results demonstrate that large language models struggle with generating meaningful communication that is grounded in the social and physical context. In contrast, our approach can successfully generate concise verbal communication for the embodied assistant to effectively boost the performance of the cooperation as well as human users' perception of the assistant. △ Less

Submitted 16 March, 2024; originally announced March 2024.

Comments: 8 pages, 5 figures

A Sinkhorn-type Algorithm for Constrained Optimal Transport

Authors: Xun Tang, Holakou Rahmanian, Michael Shavlovsky, Kiran Koshy Thekumparampil, Tesi Xiao, Lexing Ying

Abstract: Entropic optimal transport (OT) and the Sinkhorn algorithm have made it practical for machine learning practitioners to perform the fundamental task of calculating transport distance between statistical distributions. In this work, we focus on a general class of OT problems under a combination of equality and inequality constraints. We derive the corresponding entropy regularization formulation an… ▽ More Entropic optimal transport (OT) and the Sinkhorn algorithm have made it practical for machine learning practitioners to perform the fundamental task of calculating transport distance between statistical distributions. In this work, we focus on a general class of OT problems under a combination of equality and inequality constraints. We derive the corresponding entropy regularization formulation and introduce a Sinkhorn-type algorithm for such constrained OT problems supported by theoretical guarantees. We first bound the approximation error when solving the problem through entropic regularization, which reduces exponentially with the increase of the regularization parameter. Furthermore, we prove a sublinear first-order convergence rate of the proposed Sinkhorn-type algorithm in the dual space by characterizing the optimization procedure with a Lyapunov function. To achieve fast and higher-order convergence under weak entropy regularization, we augment the Sinkhorn-type algorithm with dynamic regularization scheduling and second-order acceleration. Overall, this work systematically combines recent theoretical and numerical advances in entropic optimal transport with the constrained case, allowing practitioners to derive approximate transport plans in complex scenarios. △ Less

Submitted 8 March, 2024; originally announced March 2024.

Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning

Authors: Tan Zhi-Xuan, Lance Ying, Vikash Mansinghka, Joshua B. Tenenbaum

Abstract: People often give instructions whose meaning is ambiguous without further context, expecting that their actions or goals will disambiguate their intentions. How can we build assistive agents that follow such instructions in a flexible, context-sensitive manner? This paper introduces cooperative language-guided inverse plan search (CLIPS), a Bayesian agent architecture for pragmatic instruction fol… ▽ More People often give instructions whose meaning is ambiguous without further context, expecting that their actions or goals will disambiguate their intentions. How can we build assistive agents that follow such instructions in a flexible, context-sensitive manner? This paper introduces cooperative language-guided inverse plan search (CLIPS), a Bayesian agent architecture for pragmatic instruction following and goal assistance. Our agent assists a human by modeling them as a cooperative planner who communicates joint plans to the assistant, then performs multimodal Bayesian inference over the human's goal from actions and language, using large language models (LLMs) to evaluate the likelihood of an instruction given a hypothesized plan. Given this posterior, our assistant acts to minimize expected goal achievement cost, enabling it to pragmatically follow ambiguous instructions and provide effective assistance even when uncertain about the goal. We evaluate these capabilities in two cooperative planning domains (Doors, Keys & Gems and VirtualHome), finding that CLIPS significantly outperforms GPT-4V, LLM-based literal instruction following and unimodal inverse planning in both accuracy and helpfulness, while closely matching the inferences and assistive judgments provided by human raters. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Comments: Accepted to AAMAS 2024. 8 pages (excl. references), 5 figures/tables. (Appendix: 8 pages, 8 figures/tables). Code available at: https://github.com/probcomp/CLIPS.jl

Multidimensional unstructured sparse recovery via eigenmatrix

Authors: Lexing Ying

Abstract: This note considers the multidimensional unstructured sparse recovery problems. Examples include Fourier inversion and sparse deconvolution. The eigenmatrix is a data-driven construction with desired approximate eigenvalues and eigenvectors proposed for the one-dimensional problems. This note extends the eigenmatrix approach to multidimensional problems. Numerical results are provided to demonstra… ▽ More This note considers the multidimensional unstructured sparse recovery problems. Examples include Fourier inversion and sparse deconvolution. The eigenmatrix is a data-driven construction with desired approximate eigenvalues and eigenvectors proposed for the one-dimensional problems. This note extends the eigenmatrix approach to multidimensional problems. Numerical results are provided to demonstrate the performance of the proposed method. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2311.16609

Cost Aware Best Arm Identification

Authors: Kellen Kanarios, Qining Zhang, Lei Ying

Abstract: In this paper, we study a best arm identification problem with dual objects. In addition to the classic reward, each arm is associated with a cost distribution and the goal is to identify the largest reward arm using the minimum expected cost. We call it \emph{Cost Aware Best Arm Identification} (CABAI), which captures the separation of testing and implementation phases in product development pipe… ▽ More In this paper, we study a best arm identification problem with dual objects. In addition to the classic reward, each arm is associated with a cost distribution and the goal is to identify the largest reward arm using the minimum expected cost. We call it \emph{Cost Aware Best Arm Identification} (CABAI), which captures the separation of testing and implementation phases in product development pipelines and models the objective shift between phases, i.e., cost for testing and reward for implementation. We first derive a theoretical lower bound for CABAI and propose an algorithm called $\mathsf{CTAS}$ to match it asymptotically. To reduce the computation of $\mathsf{CTAS}$, we further propose a simple algorithm called \emph{Chernoff Overlap} (CO), based on a square-root rule, which we prove is optimal in simplified two-armed models and generalizes well in numerical experiments. Our results show that (i) ignoring the heterogeneous action cost results in sub-optimality in practice, and (ii) simple algorithms can deliver near-optimal performance over a wide range of problems. △ Less

Submitted 30 June, 2024; v1 submitted 26 February, 2024; originally announced February 2024.

A sublinear-time randomized algorithm for column and row subset selection based on strong rank-revealing QR factorizations

Authors: Alice Cortinovis, Lexing Ying

Abstract: In this work, we analyze a sublinear-time algorithm for selecting a few rows and columns of a matrix for low-rank approximation purposes. The algorithm is based on an initial uniformly random selection of rows and columns, followed by a refinement of this choice using a strong rank-revealing QR factorization. We prove bounds on the error of the corresponding low-rank approximation (more precisely,… ▽ More In this work, we analyze a sublinear-time algorithm for selecting a few rows and columns of a matrix for low-rank approximation purposes. The algorithm is based on an initial uniformly random selection of rows and columns, followed by a refinement of this choice using a strong rank-revealing QR factorization. We prove bounds on the error of the corresponding low-rank approximation (more precisely, the CUR approximation error) when the matrix is a perturbation of a low-rank matrix that can be factorized into the product of matrices with suitable incoherence and/or sparsity assumptions. △ Less

Submitted 21 February, 2024; originally announced February 2024.

MSC Class: 65F30 (Primary) 15A23 (Secondary)

Grounding Language about Belief in a Bayesian Theory-of-Mind

Authors: Lance Ying, Tan Zhi-Xuan, Lionel Wong, Vikash Mansinghka, Joshua Tenenbaum

Abstract: Despite the fact that beliefs are mental states that cannot be directly observed, humans talk about each others' beliefs on a regular basis, often using rich compositional language to describe what others think and know. What explains this capacity to interpret the hidden epistemic content of other minds? In this paper, we take a step towards an answer by grounding the semantics of belief statemen… ▽ More Despite the fact that beliefs are mental states that cannot be directly observed, humans talk about each others' beliefs on a regular basis, often using rich compositional language to describe what others think and know. What explains this capacity to interpret the hidden epistemic content of other minds? In this paper, we take a step towards an answer by grounding the semantics of belief statements in a Bayesian theory-of-mind: By modeling how humans jointly infer coherent sets of goals, beliefs, and plans that explain an agent's actions, then evaluating statements about the agent's beliefs against these inferences via epistemic logic, our framework provides a conceptual role semantics for belief, explaining the gradedness and compositionality of human belief attributions, as well as their intimate connection with goals and plans. We evaluate this framework by studying how humans attribute goals and beliefs while watching an agent solve a doors-and-keys gridworld puzzle that requires instrumental reasoning about hidden objects. In contrast to pure logical deduction, non-mentalizing baselines, and mentalizing that ignores the role of instrumental plans, our model provides a much better fit to human goal and belief attributions, demonstrating the importance of theory-of-mind for a semantics of belief. △ Less

Submitted 8 July, 2024; v1 submitted 15 February, 2024; originally announced February 2024.

Comments: Published at CogSci 2024

Convergence Analysis of Discrete Diffusion Model: Exact Implementation through Uniformization

Authors: Hongrui Chen, Lexing Ying

Abstract: Diffusion models have achieved huge empirical success in data generation tasks. Recently, some efforts have been made to adapt the framework of diffusion models to discrete state space, providing a more natural approach for modeling intrinsically discrete data, such as language and graphs. This is achieved by formulating both the forward noising process and the corresponding reversed process as Co… ▽ More Diffusion models have achieved huge empirical success in data generation tasks. Recently, some efforts have been made to adapt the framework of diffusion models to discrete state space, providing a more natural approach for modeling intrinsically discrete data, such as language and graphs. This is achieved by formulating both the forward noising process and the corresponding reversed process as Continuous Time Markov Chains (CTMCs). In this paper, we investigate the theoretical properties of the discrete diffusion model. Specifically, we introduce an algorithm leveraging the uniformization of continuous Markov chains, implementing transitions on random time points. Under reasonable assumptions on the learning of the discrete score function, we derive Total Variation distance and KL divergence guarantees for sampling from any distribution on a hypercube. Our results align with state-of-the-art achievements for diffusion models in $\mathbb{R}^d$ and further underscore the advantages of discrete diffusion models in comparison to the $\mathbb{R}^d$ setting. △ Less

Submitted 14 February, 2024; v1 submitted 12 February, 2024; originally announced February 2024.

Comments: 19 pages

Knowledge-driven deep learning for fast MR imaging: undersampled MR image reconstruction from supervised to un-supervised learning

Authors: Shanshan Wang, Ruoyou Wu, Sen Jia, Alou Diakite, Cheng Li, Qiegen Liu, Leslie Ying

Abstract: Deep learning (DL) has emerged as a leading approach in accelerating MR imaging. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MR imaging involves physics-based imaging processes, unique data properties, and diverse imaging tasks.… ▽ More Deep learning (DL) has emerged as a leading approach in accelerating MR imaging. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MR imaging involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MR imaging along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems. △ Less

Submitted 4 February, 2024; originally announced February 2024.

Comments: 46 pages, 5figures, 1 table

Quantum Multiple Eigenvalue Gaussian filtered Search: an efficient and versatile quantum phase estimation method

Authors: Zhiyan Ding, Haoya Li, Lin Lin, HongKang Ni, Lexing Ying, Ruizhe Zhang

Abstract: Quantum phase estimation is one of the most powerful quantum primitives. This work proposes a new approach for the problem of multiple eigenvalue estimation: Quantum Multiple Eigenvalue Gaussian filtered Search (QMEGS). QMEGS leverages the Hadamard test circuit structure and only requires simple classical postprocessing. QMEGS is the first algorithm to simultaneously satisfy the following two prop… ▽ More Quantum phase estimation is one of the most powerful quantum primitives. This work proposes a new approach for the problem of multiple eigenvalue estimation: Quantum Multiple Eigenvalue Gaussian filtered Search (QMEGS). QMEGS leverages the Hadamard test circuit structure and only requires simple classical postprocessing. QMEGS is the first algorithm to simultaneously satisfy the following two properties: (1) It can achieve the Heisenberg-limited scaling without relying on any spectral gap assumption. (2) With a positive energy gap and additional assumptions on the initial state, QMEGS can estimate all dominant eigenvalues to $ε$ accuracy utilizing a significantly reduced circuit depth compared to the standard quantum phase estimation algorithm. In the most favorable scenario, the maximal runtime can be reduced to as low as $\log(1/ε)$. This implies that QMEGS serves as an efficient and versatile approach, achieving the best-known results for both gapped and gapless systems. Numerical results validate the efficiency of our proposed algorithm in various regimes. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Ensemble-Based Annealed Importance Sampling

Authors: Haoxuan Chen, Lexing Ying

Abstract: Sampling from a multimodal distribution is a fundamental and challenging problem in computational science and statistics. Among various approaches proposed for this task, one popular method is Annealed Importance Sampling (AIS). In this paper, we propose an ensemble-based version of AIS by combining it with population-based Monte Carlo methods to improve its efficiency. By kee** track of an ense… ▽ More Sampling from a multimodal distribution is a fundamental and challenging problem in computational science and statistics. Among various approaches proposed for this task, one popular method is Annealed Importance Sampling (AIS). In this paper, we propose an ensemble-based version of AIS by combining it with population-based Monte Carlo methods to improve its efficiency. By kee** track of an ensemble instead of a single particle along some continuation path between the starting distribution and the target distribution, we take advantage of the interaction within the ensemble to encourage the exploration of undiscovered modes. Specifically, our main idea is to utilize either the snooker algorithm or the genetic algorithm used in Evolutionary Monte Carlo. We discuss how the proposed algorithm can be implemented and derive a partial differential equation governing the evolution of the ensemble under the continuous time and mean-field limit. We also test the efficiency of the proposed algorithm on various continuous and discrete distributions. △ Less

Submitted 28 January, 2024; originally announced January 2024.

Comments: 33 pages, 13 figures

MSC Class: 65C05; 65C40; 65C60; 62P35

Accelerating Sinkhorn Algorithm with Sparse Newton Iterations

Authors: Xun Tang, Michael Shavlovsky, Holakou Rahmanian, Elisa Tardini, Kiran Koshy Thekumparampil, Tesi Xiao, Lexing Ying

Abstract: Computing the optimal transport distance between statistical distributions is a fundamental task in machine learning. One remarkable recent advancement is entropic regularization and the Sinkhorn algorithm, which utilizes only matrix scaling and guarantees an approximated solution with near-linear runtime. Despite the success of the Sinkhorn algorithm, its runtime may still be slow due to the pote… ▽ More Computing the optimal transport distance between statistical distributions is a fundamental task in machine learning. One remarkable recent advancement is entropic regularization and the Sinkhorn algorithm, which utilizes only matrix scaling and guarantees an approximated solution with near-linear runtime. Despite the success of the Sinkhorn algorithm, its runtime may still be slow due to the potentially large number of iterations needed for convergence. To achieve possibly super-exponential convergence, we present Sinkhorn-Newton-Sparse (SNS), an extension to the Sinkhorn algorithm, by introducing early stop** for the matrix scaling steps and a second stage featuring a Newton-type subroutine. Adopting the variational viewpoint that the Sinkhorn algorithm maximizes a concave Lyapunov potential, we offer the insight that the Hessian matrix of the potential function is approximately sparse. Sparsification of the Hessian results in a fast $O(n^2)$ per-iteration complexity, the same as the Sinkhorn algorithm. In terms of total iteration count, we observe that the SNS algorithm converges orders of magnitude faster across a wide range of practical cases, including optimal transportation between empirical distributions and calculating the Wasserstein $W_1, W_2$ distance of discretized densities. The empirical performance is corroborated by a rigorous bound on the approximate sparsity of the Hessian matrix. △ Less

Submitted 20 January, 2024; originally announced January 2024.

Comments: In ICLR 2024

Understanding the Generalization Benefits of Late Learning Rate Decay

Authors: Yinuo Ren, Chao Ma, Lexing Ying

Abstract: Why do neural networks trained with large learning rates for a longer time often lead to better generalization? In this paper, we delve into this question by examining the relation between training and testing loss in neural networks. Through visualization of these losses, we note that the training trajectory with a large learning rate navigates through the minima manifold of the training loss, fi… ▽ More Why do neural networks trained with large learning rates for a longer time often lead to better generalization? In this paper, we delve into this question by examining the relation between training and testing loss in neural networks. Through visualization of these losses, we note that the training trajectory with a large learning rate navigates through the minima manifold of the training loss, finally nearing the neighborhood of the testing loss minimum. Motivated by these findings, we introduce a nonlinear model whose loss landscapes mirror those observed for real neural networks. Upon investigating the training process using SGD on our model, we demonstrate that an extended phase with a large learning rate steers our model towards the minimum norm solution of the training loss, which may achieve near-optimal generalization, thereby affirming the empirically observed benefits of late learning rate decay. △ Less

Submitted 21 January, 2024; originally announced January 2024.

Comments: Accepted by AISTATS 2024

Multi-Modal Federated Learning for Cancer Staging over Non-IID Datasets with Unbalanced Modalities

Authors: Kasra Borazjani, Naji Khosravan, Leslie Ying, Seyyedali Hosseinalipour

Abstract: The use of machine learning (ML) for cancer staging through medical image analysis has gained substantial interest across medical disciplines. When accompanied by the innovative federated learning (FL) framework, ML techniques can further overcome privacy concerns related to patient data exposure. Given the frequent presence of diverse data modalities within patient records, leveraging FL in a mul… ▽ More The use of machine learning (ML) for cancer staging through medical image analysis has gained substantial interest across medical disciplines. When accompanied by the innovative federated learning (FL) framework, ML techniques can further overcome privacy concerns related to patient data exposure. Given the frequent presence of diverse data modalities within patient records, leveraging FL in a multi-modal learning framework holds considerable promise for cancer staging. However, existing works on multi-modal FL often presume that all data-collecting institutions have access to all data modalities. This oversimplified approach neglects institutions that have access to only a portion of data modalities within the system. In this work, we introduce a novel FL architecture designed to accommodate not only the heterogeneity of data samples, but also the inherent heterogeneity/non-uniformity of data modalities across institutions. We shed light on the challenges associated with varying convergence speeds observed across different data modalities within our FL system. Subsequently, we propose a solution to tackle these challenges by devising a distributed gradient blending and proximity-aware client weighting strategy tailored for multi-modal FL. To show the superiority of our method, we conduct experiments using The Cancer Genome Atlas program (TCGA) datalake considering different cancer types and three modalities of data: mRNA sequences, histopathological image data, and clinical information. Our results further unveil the impact and severity of class-based vs type-based heterogeneity across institutions on the model performance, which widens the perspective to the notion of data heterogeneity in multi-modal FL literature. △ Less

Submitted 11 July, 2024; v1 submitted 7 January, 2024; originally announced January 2024.

Comments: 17 pages, 5 figures, 18 tables

Quantum Hamiltonian Learning for the Fermi-Hubbard Model

Authors: Hongkang Ni, Haoya Li, Lexing Ying

Abstract: This work proposes a protocol for Fermionic Hamiltonian learning. For the Hubbard model defined on a bounded-degree graph, the Heisenberg-limited scaling is achieved while allowing for state preparation and measurement errors. To achieve $ε$-accurate estimation for all parameters, only $\tilde{\mathcal{O}}(ε^{-1})$ total evolution time is needed, and the constant factor is independent of the syste… ▽ More This work proposes a protocol for Fermionic Hamiltonian learning. For the Hubbard model defined on a bounded-degree graph, the Heisenberg-limited scaling is achieved while allowing for state preparation and measurement errors. To achieve $ε$-accurate estimation for all parameters, only $\tilde{\mathcal{O}}(ε^{-1})$ total evolution time is needed, and the constant factor is independent of the system size. Moreover, our method only involves simple one or two-site Fermionic manipulations, which is desirable for experiment implementation. △ Less

Submitted 1 May, 2024; v1 submitted 28 December, 2023; originally announced December 2023.

Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration

Authors: Honghao Wei, Xin Liu, Lei Ying

Abstract: This paper studies safe Reinforcement Learning (safe RL) with linear function approximation and under hard instantaneous constraints where unsafe actions must be avoided at each step. Existing studies have considered safe RL with hard instantaneous constraints, but their approaches rely on several key assumptions: $(i)$ the RL agent knows a safe action set for {\it every} state or knows a {\it saf… ▽ More This paper studies safe Reinforcement Learning (safe RL) with linear function approximation and under hard instantaneous constraints where unsafe actions must be avoided at each step. Existing studies have considered safe RL with hard instantaneous constraints, but their approaches rely on several key assumptions: $(i)$ the RL agent knows a safe action set for {\it every} state or knows a {\it safe graph} in which all the state-action-state triples are safe, and $(ii)$ the constraint/cost functions are {\it linear}. In this paper, we consider safe RL with instantaneous hard constraints without assumption $(i)$ and generalize $(ii)$ to Reproducing Kernel Hilbert Space (RKHS). Our proposed algorithm, LSVI-AE, achieves $\tilde{\cO}(\sqrt{d^3H^4K})$ regret and $\tilde{\cO}(H \sqrt{dK})$ hard constraint violation when the cost function is linear and $\cO(Hγ_K \sqrt{K})$ hard constraint violation when the cost function belongs to RKHS. Here $K$ is the learning horizon, $H$ is the length of each episode, and $γ_K$ is the information gain w.r.t the kernel used to approximate cost functions. Our results achieve the optimal dependency on the learning horizon $K$, matching the lower bound we provide in this paper and demonstrating the efficiency of LSVI-AE. Notably, the design of our approach encourages aggressive policy exploration, providing a unique perspective on safe RL with general cost functions and no prior knowledge of safe actions, which may be of independent interest. △ Less

Submitted 22 December, 2023; originally announced December 2023.

Observation of many-body dynamical localization

Authors: Yanliang Guo, Sudipta Dhar, Ang Yang, Zekai Chen, Hepeng Yao, Milena Horvath, Lei Ying, Manuele Landini, Hanns-Christoph Nägerl

Abstract: The quantum kicked rotor is a paradigmatic model system in quantum physics. As a driven quantum system, it is used to study the transition from the classical to the quantum world and to elucidate the emergence of chaos and diffusion. In contrast to its classical counterpart, it features dynamical localization, specifically Anderson localization in momentum space. The interacting many-body kicked r… ▽ More The quantum kicked rotor is a paradigmatic model system in quantum physics. As a driven quantum system, it is used to study the transition from the classical to the quantum world and to elucidate the emergence of chaos and diffusion. In contrast to its classical counterpart, it features dynamical localization, specifically Anderson localization in momentum space. The interacting many-body kicked rotor is believed to break localization, as recent experiments suggest. Here, we present evidence for many-body dynamical localization for the Lieb-Liniger version of the many-body quantum kicked rotor. After some initial evolution, the momentum distribution of interacting quantum-degenerate bosonic atoms in one-dimensional geometry, kicked hundreds of times by means of a pulsed sinusoidal potential, stops spreading. We quantify the arrested evolution by analysing the energy and the information entropy of the system as the interaction strength is tuned. In the limiting cases of vanishing and strong interactions, the first-order correlation function exhibits a very different decay behavior. Our results shed light on the boundary between the classical, chaotic world and the realm of quantum physics. △ Less

Submitted 21 December, 2023; originally announced December 2023.

Comments: 13 pages, 11 figures

Disorder-tunable entanglement at infinite temperature

Authors: Hang Dong, Jean-Yves Desaules, Yu Gao, Ning Wang, Zexian Guo, Jiachen Chen, Yiren Zou, Feitong **, Xuhao Zhu, Pengfei Zhang, Hekang Li, Zhen Wang, Qiujiang Guo, Junxiang Zhang, Lei Ying, Zlatko Papić

Abstract: Complex entanglement structures in many-body quantum systems offer potential benefits for quantum technology, yet their applicability tends to be severely limited by thermal noise and disorder. To bypass this roadblock, we utilize a custom-built superconducting qubit ladder to realize a new paradigm of non-thermalizing states with rich entanglement structures in the middle of the energy spectrum.… ▽ More Complex entanglement structures in many-body quantum systems offer potential benefits for quantum technology, yet their applicability tends to be severely limited by thermal noise and disorder. To bypass this roadblock, we utilize a custom-built superconducting qubit ladder to realize a new paradigm of non-thermalizing states with rich entanglement structures in the middle of the energy spectrum. Despite effectively forming an "infinite" temperature ensemble, these states robustly encode quantum information far from equilibrium, as we demonstrate by measuring the fidelity and entanglement entropy in the quench dynamics of the ladder. Our approach harnesses the recently proposed type of non-ergodic behavior known as "rainbow scar", which allows us to obtain analytically exact eigenfunctions whose ergodicity-breaking properties can be conveniently controlled by randomizing the couplings of the model, without affecting their energy. The on-demand tunability of entanglement structure via disorder allows for in situ control over ergodicity breaking and it provides a knob for designing exotic many-body states that defy thermalization. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: 7+17 pages

Solving high-dimensional Fokker-Planck equation with functional hierarchical tensor

Authors: Xun Tang, Lexing Ying

Abstract: This work is concerned with solving high-dimensional Fokker-Planck equations with the novel perspective that solving the PDE can be reduced to independent instances of density estimation tasks based on the trajectories sampled from its associated particle dynamics. With this approach, one sidesteps error accumulation occurring from integrating the PDE dynamics on a parameterized function class. Th… ▽ More This work is concerned with solving high-dimensional Fokker-Planck equations with the novel perspective that solving the PDE can be reduced to independent instances of density estimation tasks based on the trajectories sampled from its associated particle dynamics. With this approach, one sidesteps error accumulation occurring from integrating the PDE dynamics on a parameterized function class. This approach significantly simplifies deployment, as one is free of the challenges of implementing loss terms based on the differential equation. In particular, we introduce a novel class of high-dimensional functions called the functional hierarchical tensor (FHT). The FHT ansatz leverages a hierarchical low-rank structure, offering the advantage of linearly scalable runtime and memory complexity relative to the dimension count. We introduce a sketching-based technique that performs density estimation over particles simulated from the particle dynamics associated with the equation, thereby obtaining a representation of the Fokker-Planck solution in terms of our ansatz. We apply the proposed approach successfully to three challenging time-dependent Ginzburg-Landau models with hundreds of variables. △ Less

Submitted 12 December, 2023; originally announced December 2023.

Statistical Spatially Inhomogeneous Diffusion Inference

Authors: Yinuo Ren, Yi** Lu, Lexing Ying, Grant M. Rotskoff

Abstract: Inferring a diffusion equation from discretely-observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments. Assuming that the underlying dynamical process obeys a $d$-dimensional stochastic differential equation of the form… ▽ More Inferring a diffusion equation from discretely-observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments. Assuming that the underlying dynamical process obeys a $d$-dimensional stochastic differential equation of the form $$\mathrm{d}\boldsymbol{x}_t=\boldsymbol{b}(\boldsymbol{x}_t)\mathrm{d} t+Σ(\boldsymbol{x}_t)\mathrm{d}\boldsymbol{w}_t,$$ we propose neural network-based estimators of both the drift $\boldsymbol{b}$ and the spatially-inhomogeneous diffusion tensor $D = ΣΣ^{T}$ and provide statistical convergence guarantees when $\boldsymbol{b}$ and $D$ are $s$-Hölder continuous. Notably, our bound aligns with the minimax optimal rate $N^{-\frac{2s}{2s+d}}$ for nonparametric function estimation even in the presence of correlation within observational data, which necessitates careful handling when establishing fast-rate generalization bounds. Our theoretical results are bolstered by numerical experiments demonstrating accurate inference of spatially-inhomogeneous diffusion tensors. △ Less

Submitted 10 December, 2023; originally announced December 2023.

Comments: Accepted by AAAI 2024

Eigenmatrix for unstructured sparse recovery

Authors: Lexing Ying

Abstract: This note considers the unstructured sparse recovery problems in a general form. Examples include rational approximation, spectral function estimation, Fourier inversion, Laplace inversion, and sparse deconvolution. The main challenges are the noise in the sample values and the unstructured nature of the sample locations. This note proposes the eigenmatrix, a data-driven construction with desired… ▽ More This note considers the unstructured sparse recovery problems in a general form. Examples include rational approximation, spectral function estimation, Fourier inversion, Laplace inversion, and sparse deconvolution. The main challenges are the noise in the sample values and the unstructured nature of the sample locations. This note proposes the eigenmatrix, a data-driven construction with desired approximate eigenvalues and eigenvectors. The eigenmatrix offers a new way for these sparse recovery problems. Numerical results are provided to demonstrate the efficiency of the proposed method. △ Less

Submitted 7 March, 2024; v1 submitted 28 November, 2023; originally announced November 2023.

Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image

Authors: Zijian Kuang, Lihang Ying, Shi **, Li Cheng

Abstract: Pixel-aligned Implicit Function (PIFu) effectively captures subtle variations in body shape within a low-dimensional space through extensive training with human 3D scans, its application to live animals presents formidable challenges due to the difficulty of obtaining animal cooperation for 3D scanning. To address this challenge, we propose the combination of two-stage supervised and self-supervis… ▽ More Pixel-aligned Implicit Function (PIFu) effectively captures subtle variations in body shape within a low-dimensional space through extensive training with human 3D scans, its application to live animals presents formidable challenges due to the difficulty of obtaining animal cooperation for 3D scanning. To address this challenge, we propose the combination of two-stage supervised and self-supervised training to address the challenge of obtaining animal cooperation for 3D scanning. In the first stage, we leverage synthetic animal models for supervised learning. This allows the model to learn from a diverse set of virtual animal instances. In the second stage, we use 2D multi-view consistency as a self-supervised training method. This further enhances the model's ability to reconstruct accurate and realistic 3D shape and texture from largely available single-view images of real animals. The results of our study demonstrate that our approach outperforms state-of-the-art methods in both quantitative and qualitative aspects of bird 3D digitization. The source code is available at https://github.com/kuangzijian/drifu-for-animals. △ Less

Submitted 19 February, 2024; v1 submitted 22 November, 2023; originally announced November 2023.

Multi-Objective Optimization via Wasserstein-Fisher-Rao Gradient Flow

Authors: Yinuo Ren, Tesi Xiao, Tanmay Gangwani, Anshuka Rangi, Holakou Rahmanian, Lexing Ying, Subhajit Sanyal

Abstract: Multi-objective optimization (MOO) aims to optimize multiple, possibly conflicting objectives with widespread applications. We introduce a novel interacting particle method for MOO inspired by molecular dynamics simulations. Our approach combines overdamped Langevin and birth-death dynamics, incorporating a "dominance potential" to steer particles toward global Pareto optimality. In contrast to pr… ▽ More Multi-objective optimization (MOO) aims to optimize multiple, possibly conflicting objectives with widespread applications. We introduce a novel interacting particle method for MOO inspired by molecular dynamics simulations. Our approach combines overdamped Langevin and birth-death dynamics, incorporating a "dominance potential" to steer particles toward global Pareto optimality. In contrast to previous methods, our method is able to relocate dominated particles, making it particularly adept at managing Pareto fronts of complicated geometries. Our method is also theoretically grounded as a Wasserstein-Fisher-Rao gradient flow with convergence guarantees. Extensive experiments confirm that our approach outperforms state-of-the-art methods on challenging synthetic and real-world datasets. △ Less

Submitted 21 November, 2023; originally announced November 2023.

Multimodal Sampling via Approximate Symmetries

Authors: Lexing Ying

Abstract: Sampling from multimodal distributions is a challenging task in scientific computing. When a distribution has an exact symmetry between the modes, direct jumps among them can accelerate the samplings significantly. However, the distributions from most applications do not have exact symmetries. This paper considers the distributions with approximate symmetries. We first construct an exactly symmetr… ▽ More Sampling from multimodal distributions is a challenging task in scientific computing. When a distribution has an exact symmetry between the modes, direct jumps among them can accelerate the samplings significantly. However, the distributions from most applications do not have exact symmetries. This paper considers the distributions with approximate symmetries. We first construct an exactly symmetric reference distribution from the target one by averaging over the group orbit associated with the approximate symmetry. Next, we can apply the multilevel Monte Carlo methods by constructing a continuation path between the reference and target distributions. We discuss how to implement these steps with annealed importance sampling and tempered transitions. Compared with traditional multilevel methods, the proposed approach can be more effective since the reference and target distributions are much closer. Numerical results of the Ising models are presented to illustrate the efficiency of the proposed method. △ Less

Submitted 4 January, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

MSC Class: 82B20; 82B80

Model-Free, Regret-Optimal Best Policy Identification in Online CMDPs

Authors: Zihan Zhou, Honghao Wei, Lei Ying

Abstract: This paper considers the best policy identification (BPI) problem in online Constrained Markov Decision Processes (CMDPs). We are interested in algorithms that are model-free, have low regret, and identify an approximately optimal policy with a high probability. Existing model-free algorithms for online CMDPs with sublinear regret and constraint violation do not provide any convergence guarantee t… ▽ More This paper considers the best policy identification (BPI) problem in online Constrained Markov Decision Processes (CMDPs). We are interested in algorithms that are model-free, have low regret, and identify an approximately optimal policy with a high probability. Existing model-free algorithms for online CMDPs with sublinear regret and constraint violation do not provide any convergence guarantee to an optimal policy and provide only average performance guarantees when a policy is uniformly sampled at random from all previously used policies. In this paper, we develop a new algorithm, named Pruning-Refinement-Identification (PRI), based on a fundamental structural property of CMDPs proved before, which we call limited stochasticity. The property says for a CMDP with $N$ constraints, there exists an optimal policy with at most $N$ stochastic decisions. The proposed algorithm first identifies at which step and in which state a stochastic decision has to be taken and then fine-tunes the distributions of these stochastic decisions. PRI achieves trio objectives: (i) PRI is a model-free algorithm; and (ii) it outputs an approximately optimal policy with a high probability at the end of learning; and (iii) PRI guarantees $\tilde{\mathcal{O}}(H\sqrt{K})$ regret and constraint violation, which significantly improves the best existing regret bound $\tilde{\mathcal{O}}(H^4 \sqrt{SA}K^{\frac{4}{5}})$ under a model-free algorithm, where $H$ is the length of each episode, $S$ is the number of states, $A$ is the number of actions, and the total number of episodes during learning is $2K+\tilde{\cal O}(K^{0.25}).$ We further present a matching lower via an example that shows under any online learning algorithm, there exists a well-separated CMDP instance such that either the regret or violation has to be $Ω(H\sqrt{K}),$ which matches the upper bound by a polylogarithmic factor. △ Less

Submitted 14 April, 2024; v1 submitted 27 September, 2023; originally announced September 2023.

Reviving Static Charts into Live Charts

Authors: Lu Ying, Yun Wang, Haotian Li, Shuguang Dou, Haidong Zhang, Xinyang Jiang, Huamin Qu, Yingcai Wu

Abstract: Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce "Live Charts," a new format of presentation that decomposes complex information within a chart and explains the infor… ▽ More Data charts are prevalent across various fields due to their efficacy in conveying complex data relationships. However, static charts may sometimes struggle to engage readers and efficiently present intricate information, potentially resulting in limited understanding. We introduce "Live Charts," a new format of presentation that decomposes complex information within a chart and explains the information pieces sequentially through rich animations and accompanying audio narration. We propose an automated approach to revive static charts into Live Charts. Our method integrates GNN-based techniques to analyze the chart components and extract data from charts. Then we adopt large natural language models to generate appropriate animated visuals along with a voice-over to produce Live Charts from static ones. We conducted a thorough evaluation of our approach, which involved the model performance, use cases, a crowd-sourced user study, and expert interviews. The results demonstrate Live Charts offer a multi-sensory experience where readers can follow the information and understand the data insights better. We analyze the benefits and drawbacks of Live Charts over static charts as a new information consumption experience. △ Less

Submitted 17 May, 2024; v1 submitted 6 September, 2023; originally announced September 2023.

Comments: Accepted by IEEE TVCG

Fast and Regret Optimal Best Arm Identification: Fundamental Limits and Low-Complexity Algorithms

Authors: Qining Zhang, Lei Ying

Abstract: This paper considers a stochastic Multi-Armed Bandit (MAB) problem with dual objectives: (i) quick identification and commitment to the optimal arm, and (ii) reward maximization throughout a sequence of $T$ consecutive rounds. Though each objective has been individually well-studied, i.e., best arm identification for (i) and regret minimization for (ii), the simultaneous realization of both object… ▽ More This paper considers a stochastic Multi-Armed Bandit (MAB) problem with dual objectives: (i) quick identification and commitment to the optimal arm, and (ii) reward maximization throughout a sequence of $T$ consecutive rounds. Though each objective has been individually well-studied, i.e., best arm identification for (i) and regret minimization for (ii), the simultaneous realization of both objectives remains an open problem, despite its practical importance. This paper introduces \emph{Regret Optimal Best Arm Identification} (ROBAI) which aims to achieve these dual objectives. To solve ROBAI with both pre-determined stop** time and adaptive stop** time requirements, we present an algorithm called EOCP and its variants respectively, which not only achieve asymptotic optimal regret in both Gaussian and general bandits, but also commit to the optimal arm in $\mathcal{O}(\log T)$ rounds with pre-determined stop** time and $\mathcal{O}(\log^2 T)$ rounds with adaptive stop** time. We further characterize lower bounds on the commitment time (equivalent to the sample complexity) of ROBAI, showing that EOCP and its variants are sample optimal with pre-determined stop** time, and almost sample optimal with adaptive stop** time. Numerical results confirm our theoretical analysis and reveal an interesting "over-exploration" phenomenon carried by classic UCB algorithms, such that EOCP has smaller regret even though it stops exploration much earlier than UCB, i.e., $\mathcal{O}(\log T)$ versus $\mathcal{O}(T)$, which suggests over-exploration is unnecessary and potentially harmful to system performance. △ Less

Submitted 29 May, 2024; v1 submitted 1 September, 2023; originally announced September 2023.

Study on many-body phases in Jaynes-Cummings-Hubbard arrays

Authors: **-Lou Ma, Bobo Liu, Qing Li, Zexian Guo, Lei Tan, Lei Ying

Abstract: Disorder in one-dimensional (1D) many-body systems emerges abundant phases such as many-body localization (MBL), and thermalization. However, it remains unclear regarding their existence and behavior within hybrid quantum systems. Here, based on a simple bosonic-spin hybrid model, as known as the Jaynes-Cummings-Hubbard (JCH) array, we investigate the effect of disorder comparing to the phenomena… ▽ More Disorder in one-dimensional (1D) many-body systems emerges abundant phases such as many-body localization (MBL), and thermalization. However, it remains unclear regarding their existence and behavior within hybrid quantum systems. Here, based on a simple bosonic-spin hybrid model, as known as the Jaynes-Cummings-Hubbard (JCH) array, we investigate the effect of disorder comparing to the phenomena in the clean system with the variation of atom-photon coupling strength. By using the level-spacing ratio, entanglement entropy, and the properties of observable diagonal and off-diagonal matrix elements, we find that strong disorder results in the appearance of MBL phase in the JCH model that strongly violate eigenstate thermalization hypothesis (ETH), while a conditional prethermal behavior can exist in weak disorder or weak coupling regime. The conditional prethermal dynamics is based on the choice of initial product states. This work systematically reveals abundant many-body phases in the 1D JCH model and clarifies the discrepancies in the thermalization properties of systems with and without disorder. △ Less

Submitted 23 August, 2023; originally announced August 2023.

Quasiparticle Dynamics in Superconducting Quantum-Classical Hybrid Circuits

Authors: Kuang Liu, Xiaoliang He, Zhengqi Niu, Hang Xue, Wenbing Jiang, Liliang Ying, Wei Peng, Masaaki Maezawa, Zhirong Lin, Xiaoming Xie, Zhen Wang

Abstract: Single flux quantum (SFQ) circuitry is a promising candidate for a scalable and integratable cryogenic quantum control system. However, the operation of SFQ circuits introduces non-equilibrium quasiparticles (QPs), which are a significant source of qubit decoherence. In this study, we investigate QP behavior in a superconducting quantum-classical hybrid chip that comprises an SFQ circuit and a qub… ▽ More Single flux quantum (SFQ) circuitry is a promising candidate for a scalable and integratable cryogenic quantum control system. However, the operation of SFQ circuits introduces non-equilibrium quasiparticles (QPs), which are a significant source of qubit decoherence. In this study, we investigate QP behavior in a superconducting quantum-classical hybrid chip that comprises an SFQ circuit and a qubit circuit. By monitoring qubit relaxation time, we explore the dynamics of SFQ-circuit-induced QPs. Our findings reveal that the QP density near the qubit reaches its peak after several microseconds of SFQ circuit operation, which corresponds to the phonon-mediated propagation time of QPs in the hybrid circuits. This suggests that phonon-mediated propagation dominates the spreading of QPs in the hybrid circuits. Our results lay the foundation to suppress QP poisoning in quantum-classical hybrid systems. △ Less

Submitted 26 July, 2023; originally announced July 2023.

Comments: 11 pages,13 figures

Journal ref: PhysRevB.108.064512 (2023)

Origin of Hilbert space quantum scars in unconstrained models

Authors: Zexian Guo, Bobo Liu, Yu Gao, Ang Yang, Junlin Wang, **lou Ma, Lei Ying

Abstract: Quantum many-body scar is a recently discovered phenomenon weakly violating eigenstate thermalization hypothesis, and it has been extensively studied across various models. However, experimental realizations are mainly based on constrained models such as the $PXP$ model. Inspired by recent experimental observations on the superconducting platform in Refs.~[Nat. Phys. 19, 120 (2022)] and [arXiv:221… ▽ More Quantum many-body scar is a recently discovered phenomenon weakly violating eigenstate thermalization hypothesis, and it has been extensively studied across various models. However, experimental realizations are mainly based on constrained models such as the $PXP$ model. Inspired by recent experimental observations on the superconducting platform in Refs.~[Nat. Phys. 19, 120 (2022)] and [arXiv:2211.05803], we study a distinct class of quantum many-body scars based on a half-filling hard-core Bose-Hubbard model, which is generic to describe in many experimental platforms. It is the so-called Hilbert space quantum scar as it originates from a subspace with a hypercube geometry weakly connecting to other thermalization regions in Hilbert space. Within the hypercube, a pair of collective Fock states do not directly connect to the thermalization region, resulting in slow thermalization dynamics with remarkable fidelity revivals with distinct differences from dynamics of other initial states. This mechanism is generic in various real-space lattice configurations, including one-dimensional Su-Schrieffer-Heeger chain, comb lattice, and even random dimer clusters consisting of dimers. In addition, we develop a toy model based on Hilbert hypercube decay approximation, to explain the spectrum overlap between the collective states and all eigenstates. Furthermore, we explore the Hilbert space quantum scar in two- and three-dimensional Su-Schrieffer-Heeger many-body systems, consisting of tetramers or octamers, respectively. This study makes quantum many-body scar state more realistic in applications such as quantum sensing and quantum metrology. △ Less

Submitted 25 July, 2023; originally announced July 2023.

Heisenberg-limited Hamiltonian learning for interacting bosons

Authors: Haoya Li, Yu Tong, Hongkang Ni, Tuvia Gefen, Lexing Ying

Abstract: We develop a protocol for learning a class of interacting bosonic Hamiltonians from dynamics with Heisenberg-limited scaling. For Hamiltonians with an underlying bounded-degree graph structure, we can learn all parameters with root mean squared error $ε$ using $\mathcal{O}(1/ε)$ total evolution time, which is independent of the system size, in a way that is robust against state-preparation and mea… ▽ More We develop a protocol for learning a class of interacting bosonic Hamiltonians from dynamics with Heisenberg-limited scaling. For Hamiltonians with an underlying bounded-degree graph structure, we can learn all parameters with root mean squared error $ε$ using $\mathcal{O}(1/ε)$ total evolution time, which is independent of the system size, in a way that is robust against state-preparation and measurement error. In the protocol, we only use bosonic coherent states, beam splitters, phase shifters, and homodyne measurements, which are easy to implement on many experimental platforms. A key technique we develop is to apply random unitaries to enforce symmetry in the effective Hamiltonian, which may be of independent interest. △ Less

Submitted 10 July, 2023; originally announced July 2023.

Comments: 14 pages with 21-page appendix

Inferring the Goals of Communicating Agents from Actions and Instructions

Authors: Lance Ying, Tan Zhi-Xuan, Vikash Mansinghka, Joshua B. Tenenbaum

Abstract: When humans cooperate, they frequently coordinate their activity through both verbal communication and non-verbal actions, using this information to infer a shared goal and plan. How can we model this inferential ability? In this paper, we introduce a model of a cooperative team where one agent, the principal, may communicate natural language instructions about their shared plan to another agent,… ▽ More When humans cooperate, they frequently coordinate their activity through both verbal communication and non-verbal actions, using this information to infer a shared goal and plan. How can we model this inferential ability? In this paper, we introduce a model of a cooperative team where one agent, the principal, may communicate natural language instructions about their shared plan to another agent, the assistant, using GPT-3 as a likelihood function for instruction utterances. We then show how a third person observer can infer the team's goal via multi-modal Bayesian inverse planning from actions and instructions, computing the posterior distribution over goals under the assumption that agents will act and communicate rationally to achieve them. We evaluate this approach by comparing it with human goal inferences in a multi-agent gridworld, finding that our model's inferences closely correlate with human judgments (R = 0.96). When compared to inference from actions alone, we also find that instructions lead to more rapid and less uncertain goal inference, highlighting the importance of verbal communication for cooperative agents. △ Less

Submitted 28 June, 2023; originally announced June 2023.

Comments: 8 pages, 5 figures. Accepted to the ICML 2023 Workshop on Theory of Mind in Communicating Agents. Supplementary Information: https://osf.io/gh758/