-
PRoDeliberation: Parallel Robust Deliberation for End-to-End Spoken Language Understanding
Authors:
Trang Le,
Daniel Lazar,
Suyoun Kim,
Shan Jiang,
Duc Le,
Adithya Sagar,
Aleksandr Livshits,
Ahmed Aly,
Akshat Shrivastava
Abstract:
Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explored end-to-end models to improve the quality and robustness of SLU models with Deliberation, however these models have remained autoregressive, resulting in higher latencies. In this work we introduce PRoDeliberation, a no…
▽ More
Spoken Language Understanding (SLU) is a critical component of voice assistants; it consists of converting speech to semantic parses for task execution. Previous works have explored end-to-end models to improve the quality and robustness of SLU models with Deliberation, however these models have remained autoregressive, resulting in higher latencies. In this work we introduce PRoDeliberation, a novel method leveraging a Connectionist Temporal Classification-based decoding strategy as well as a denoising objective to train robust non-autoregressive deliberation models. We show that PRoDeliberation achieves the latency reduction of parallel decoding (2-10x improvement over autoregressive models) while retaining the ability to correct Automatic Speech Recognition (ASR) mistranscriptions of autoregressive deliberation systems. We further show that the design of the denoising training allows PRoDeliberation to overcome the limitations of small ASR devices, and we provide analysis on the necessity of each component of the system.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs
Authors:
Charlie Hou,
Akshat Shrivastava,
Hongyuan Zhan,
Rylan Conway,
Trang Le,
Adithya Sagar,
Giulia Fanti,
Daniel Lazar
Abstract:
On-device training is currently the most common approach for training machine learning (ML) models on private, distributed user data. Despite this, on-device training has several drawbacks: (1) most user devices are too small to train large models on-device, (2) on-device training is communication- and computation-intensive, and (3) on-device training can be difficult to debug and deploy. To addre…
▽ More
On-device training is currently the most common approach for training machine learning (ML) models on private, distributed user data. Despite this, on-device training has several drawbacks: (1) most user devices are too small to train large models on-device, (2) on-device training is communication- and computation-intensive, and (3) on-device training can be difficult to debug and deploy. To address these problems, we propose Private Evolution-Text (PrE-Text), a method for generating differentially private (DP) synthetic textual data. First, we show that across multiple datasets, training small models (models that fit on user devices) with PrE-Text synthetic data outperforms small models trained on-device under practical privacy regimes ($ε=1.29$, $ε=7.58$). We achieve these results while using 9$\times$ fewer rounds, 6$\times$ less client computation per round, and 100$\times$ less communication per round. Second, finetuning large models on PrE-Text's DP synthetic data improves large language model (LLM) performance on private data across the same range of privacy budgets. Altogether, these results suggest that training on DP synthetic data can be a better option than training a model on-device on private distributed data. Code is available at https://github.com/houcharlie/PrE-Text.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Augmenting text for spoken language understanding with Large Language Models
Authors:
Roshan Sharma,
Suyoun Kim,
Daniel Lazar,
Trang Le,
Akshat Shrivastava,
Kwanghoon Ahn,
Piyush Kansal,
Leda Sari,
Ozlem Kalinli,
Michael Seltzer
Abstract:
Spoken semantic parsing (SSP) involves generating machine-comprehensible parses from input speech. Training robust models for existing application domains represented in training data or extending to new domains requires corresponding triplets of speech-transcript-semantic parse data, which is expensive to obtain. In this paper, we address this challenge by examining methods that can use transcrip…
▽ More
Spoken semantic parsing (SSP) involves generating machine-comprehensible parses from input speech. Training robust models for existing application domains represented in training data or extending to new domains requires corresponding triplets of speech-transcript-semantic parse data, which is expensive to obtain. In this paper, we address this challenge by examining methods that can use transcript-semantic parse data (unpaired text) without corresponding speech. First, when unpaired text is drawn from existing textual corpora, Joint Audio Text (JAT) and Text-to-Speech (TTS) are compared as ways to generate speech representations for unpaired text. Experiments on the STOP dataset show that unpaired text from existing and new domains improves performance by 2% and 30% in absolute Exact Match (EM) respectively. Second, we consider the setting when unpaired text is not available in existing textual corpora. We propose to prompt Large Language Models (LLMs) to generate unpaired text for existing and new domains. Experiments show that examples and words that co-occur with intents can be used to generate unpaired text with Llama 2.0. Using the generated text with JAT and TTS for spoken semantic parsing improves EM on STOP by 1.4% and 2.6% absolute for existing and new domains respectively.
△ Less
Submitted 17 September, 2023;
originally announced September 2023.
-
Privately Customizing Prefinetuning to Better Match User Data in Federated Learning
Authors:
Charlie Hou,
Hongyuan Zhan,
Akshat Shrivastava,
Sid Wang,
Aleksandr Livshits,
Giulia Fanti,
Daniel Lazar
Abstract:
In Federated Learning (FL), accessing private client data incurs communication and privacy costs. As a result, FL deployments commonly prefinetune pretrained foundation models on a (large, possibly public) dataset that is held by the central server; they then FL-finetune the model on a private, federated dataset held by clients. Evaluating prefinetuning dataset quality reliably and privately is th…
▽ More
In Federated Learning (FL), accessing private client data incurs communication and privacy costs. As a result, FL deployments commonly prefinetune pretrained foundation models on a (large, possibly public) dataset that is held by the central server; they then FL-finetune the model on a private, federated dataset held by clients. Evaluating prefinetuning dataset quality reliably and privately is therefore of high importance. To this end, we propose FreD (Federated Private Fréchet Distance) -- a privately computed distance between a prefinetuning dataset and federated datasets. Intuitively, it privately computes and compares a Fréchet distance between embeddings generated by a large language model on both the central (public) dataset and the federated private client data. To make this computation privacy-preserving, we use distributed, differentially-private mean and covariance estimators. We show empirically that FreD accurately predicts the best prefinetuning dataset at minimal privacy cost. Altogether, using FreD we demonstrate a proof-of-concept for a new approach in private FL training: (1) customize a prefinetuning dataset to better match user data (2) prefinetune (3) perform FL-finetuning.
△ Less
Submitted 22 February, 2023; v1 submitted 17 February, 2023;
originally announced February 2023.
-
STOP: A dataset for Spoken Task Oriented Semantic Parsing
Authors:
Paden Tomasello,
Akshat Shrivastava,
Daniel Lazar,
Po-Chun Hsu,
Duc Le,
Adithya Sagar,
Ali Elkahky,
Jade Copet,
Wei-Ning Hsu,
Yossi Adi,
Robin Algayres,
Tu Ahn Nguyen,
Emmanuel Dupoux,
Luke Zettlemoyer,
Abdelrahman Mohamed
Abstract:
End-to-end spoken language understanding (SLU) predicts intent directly from audio using a single model. It promises to improve the performance of assistant systems by leveraging acoustic information lost in the intermediate textual representation and preventing cascading errors from Automatic Speech Recognition (ASR). Further, having one unified model has efficiency advantages when deploying assi…
▽ More
End-to-end spoken language understanding (SLU) predicts intent directly from audio using a single model. It promises to improve the performance of assistant systems by leveraging acoustic information lost in the intermediate textual representation and preventing cascading errors from Automatic Speech Recognition (ASR). Further, having one unified model has efficiency advantages when deploying assistant systems on-device. However, the limited number of public audio datasets with semantic parse labels hinders the research progress in this area. In this paper, we release the Spoken Task-Oriented semantic Parsing (STOP) dataset, the largest and most complex SLU dataset to be publicly available. Additionally, we define low-resource splits to establish a benchmark for improving SLU when limited labeled data is available. Furthermore, in addition to the human-recorded audio, we are releasing a TTS-generated version to benchmark the performance for low-resource domain adaptation of end-to-end SLU systems. Initial experimentation show end-to-end SLU models performing slightly worse than their cascaded counterparts, which we hope encourages future work in this direction.
△ Less
Submitted 18 October, 2022; v1 submitted 28 June, 2022;
originally announced July 2022.
-
Incentivizing Efficient Equilibria in Traffic Networks with Mixed Autonomy
Authors:
Erdem Bıyık,
Daniel A. Lazar,
Ramtin Pedarsani,
Dorsa Sadigh
Abstract:
Traffic congestion has large economic and social costs. The introduction of autonomous vehicles can potentially reduce this congestion by increasing road capacity via vehicle platooning and by creating an avenue for influencing people's choice of routes. We consider a network of parallel roads with two modes of transportation: (i) human drivers, who will choose the quickest route available to them…
▽ More
Traffic congestion has large economic and social costs. The introduction of autonomous vehicles can potentially reduce this congestion by increasing road capacity via vehicle platooning and by creating an avenue for influencing people's choice of routes. We consider a network of parallel roads with two modes of transportation: (i) human drivers, who will choose the quickest route available to them, and (ii) a ride hailing service, which provides an array of autonomous vehicle route options, each with different prices, to users. We formalize a model of vehicle flow in mixed autonomy and a model of how autonomous service users make choices between routes with different prices and latencies. Develo** an algorithm to learn the preferences of the users, we formulate a planning optimization that chooses prices to maximize a social objective. We demonstrate the benefit of the proposed scheme by comparing the results to theoretical benchmarks which we show can be efficiently calculated.
△ Less
Submitted 5 May, 2021;
originally announced June 2021.
-
Emergent Prosociality in Multi-Agent Games Through Gifting
Authors:
Woodrow Z. Wang,
Mark Beliaev,
Erdem Bıyık,
Daniel A. Lazar,
Ramtin Pedarsani,
Dorsa Sadigh
Abstract:
Coordination is often critical to forming prosocial behaviors -- behaviors that increase the overall sum of rewards received by all agents in a multi-agent game. However, state of the art reinforcement learning algorithms often suffer from converging to socially less desirable equilibria when multiple equilibria exist. Previous works address this challenge with explicit reward sha**, which requi…
▽ More
Coordination is often critical to forming prosocial behaviors -- behaviors that increase the overall sum of rewards received by all agents in a multi-agent game. However, state of the art reinforcement learning algorithms often suffer from converging to socially less desirable equilibria when multiple equilibria exist. Previous works address this challenge with explicit reward sha**, which requires the strong assumption that agents can be forced to be prosocial. We propose using a less restrictive peer-rewarding mechanism, gifting, that guides the agents toward more socially desirable equilibria while allowing agents to remain selfish and decentralized. Gifting allows each agent to give some of their reward to other agents. We employ a theoretical framework that captures the benefit of gifting in converging to the prosocial equilibrium by characterizing the equilibria's basins of attraction in a dynamical system. With gifting, we demonstrate increased convergence of high risk, general-sum coordination games to the prosocial equilibrium both via numerical analysis and experiments.
△ Less
Submitted 13 May, 2021;
originally announced May 2021.
-
The Role of Differentiation in Tolling of Traffic Networks with Mixed Autonomy
Authors:
Daniel A. Lazar,
Ramtin Pedarsani
Abstract:
With autonomous vehicles now sharing roads with human drivers, the era of mixed autonomy brings new challenges in dealing with congestion. One cause of congestion is when vehicle users choose their routes selfishly to minimize their personal travel delay rather than a global travel delay, and prior works address this phenomenon using tolling to influence routing choices, but do not address the set…
▽ More
With autonomous vehicles now sharing roads with human drivers, the era of mixed autonomy brings new challenges in dealing with congestion. One cause of congestion is when vehicle users choose their routes selfishly to minimize their personal travel delay rather than a global travel delay, and prior works address this phenomenon using tolling to influence routing choices, but do not address the setting of mixed autonomy. Tolls may be differentiated, meaning different users of a road experience different tolls, or they may be anonymous; the latter is desirable to allay concerns of fairness and privacy, as well as logistical challenges. In this work we examine the role of differentiation in traffic networks with mixed autonomy. Specifically, we first establish differentiated tolls which completely eliminate inefficiency due to selfish routing. We then show the fundamental limitations of anonymous tolls in our setting, and we provide anonymous tolls with mild performance guarantees. We show that in parallel networks, an infinitesimal differentiation in tolls is enough to guarantee optimality, and finally we establish a lower bound on the inefficiency of variable marginal cost tolling in the mixed autonomy setting.
△ Less
Submitted 3 August, 2021; v1 submitted 24 March, 2021;
originally announced March 2021.
-
Incentivizing Routing Choices for Safe and Efficient Transportation in the Face of the COVID-19 Pandemic
Authors:
Mark Beliaev,
Erdem Bıyık,
Daniel A. Lazar,
Woodrow Z. Wang,
Dorsa Sadigh,
Ramtin Pedarsani
Abstract:
The COVID-19 pandemic has severely affected many aspects of people's daily lives. While many countries are in a re-opening stage, some effects of the pandemic on people's behaviors are expected to last much longer, including how they choose between different transport options. Experts predict considerably delayed recovery of the public transport options, as people try to avoid crowded places. In t…
▽ More
The COVID-19 pandemic has severely affected many aspects of people's daily lives. While many countries are in a re-opening stage, some effects of the pandemic on people's behaviors are expected to last much longer, including how they choose between different transport options. Experts predict considerably delayed recovery of the public transport options, as people try to avoid crowded places. In turn, significant increases in traffic congestion are expected, since people are likely to prefer using their own vehicles or taxis as opposed to riskier and more crowded options such as the railway. In this paper, we propose to use financial incentives to set the tradeoff between risk of infection and congestion to achieve safe and efficient transportation networks. To this end, we formulate a network optimization problem to optimize taxi fares. For our framework to be useful in various cities and times of the day without much designer effort, we also propose a data-driven approach to learn human preferences about transport options, which is then used in our taxi fare optimization. Our user studies and simulation experiments show our framework is able to minimize congestion and risk of infection.
△ Less
Submitted 17 February, 2021; v1 submitted 28 December, 2020;
originally announced December 2020.
-
Optimal Tolling for Multitype Mixed Autonomous Traffic Networks
Authors:
Daniel A. Lazar,
Ramtin Pedarsani
Abstract:
When selfish users share a road network and minimize their individual travel costs, the equilibrium they reach can be worse than the socially optimal routing. Tolls are often used to mitigate this effect in traditional congestion games, where all vehicle contribute identically to congestion. However, with the proliferation of autonomous vehicles and driver-assistance technology, vehicles become he…
▽ More
When selfish users share a road network and minimize their individual travel costs, the equilibrium they reach can be worse than the socially optimal routing. Tolls are often used to mitigate this effect in traditional congestion games, where all vehicle contribute identically to congestion. However, with the proliferation of autonomous vehicles and driver-assistance technology, vehicles become heterogeneous in how they contribute to road latency. This magnifies the potential inefficiencies due to selfish routing and invalidates traditional tolling methods. To address this, we consider a network of parallel roads where the latency on each road is an affine function of the quantity of flow of each vehicle type. We provide tolls (which differentiate between vehicle types) which are guaranteed to minimize social cost at equilibrium. The tolls are a function of a calculated optimal routing; to enable this tolling, we prove that some element in the set of optimal routings has a lack of cycles in a graph representing the way vehicles types share roads. We then show that unless a planner can differentiate between vehicle types in the tolls given, the resulting equilibrium can be unboundedly worse than the optimal routing, and that marginal cost tolling fails in our setting.
△ Less
Submitted 31 August, 2020;
originally announced September 2020.
-
Learning How to Dynamically Route Autonomous Vehicles on Shared Roads
Authors:
Daniel A. Lazar,
Erdem Bıyık,
Dorsa Sadigh,
Ramtin Pedarsani
Abstract:
Road congestion induces significant costs across the world, and road network disturbances, such as traffic accidents, can cause highly congested traffic patterns. If a planner had control over the routing of all vehicles in the network, they could easily reverse this effect. In a more realistic scenario, we consider a planner that controls autonomous cars, which are a fraction of all present cars.…
▽ More
Road congestion induces significant costs across the world, and road network disturbances, such as traffic accidents, can cause highly congested traffic patterns. If a planner had control over the routing of all vehicles in the network, they could easily reverse this effect. In a more realistic scenario, we consider a planner that controls autonomous cars, which are a fraction of all present cars. We study a dynamic routing game, in which the route choices of autonomous cars can be controlled and the human drivers react selfishly and dynamically. As the problem is prohibitively large, we use deep reinforcement learning to learn a policy for controlling the autonomous vehicles. This policy indirectly influences human drivers to route themselves in such a way that minimizes congestion on the network. To gauge the effectiveness of our learned policies, we establish theoretical results characterizing equilibria and empirically compare the learned policy results with best possible equilibria. We prove properties of equilibria on parallel roads and provide a polynomial-time optimization for computing the most efficient equilibrium. Moreover, we show that in the absence of these policies, high demand and network perturbations would result in large congestion, whereas using the policy greatly decreases the travel times by minimizing the congestion. To the best of our knowledge, this is the first work that employs deep reinforcement learning to reduce congestion by indirectly influencing humans' routing decisions in mixed-autonomy traffic.
△ Less
Submitted 3 June, 2021; v1 submitted 9 September, 2019;
originally announced September 2019.
-
The Green Choice: Learning and Influencing Human Decisions on Shared Roads
Authors:
Erdem Bıyık,
Daniel A. Lazar,
Dorsa Sadigh,
Ramtin Pedarsani
Abstract:
Autonomous vehicles have the potential to increase the capacity of roads via platooning, even when human drivers and autonomous vehicles share roads. However, when users of a road network choose their routes selfishly, the resulting traffic configuration may be very inefficient. Because of this, we consider how to influence human decisions so as to decrease congestion on these roads. We consider a…
▽ More
Autonomous vehicles have the potential to increase the capacity of roads via platooning, even when human drivers and autonomous vehicles share roads. However, when users of a road network choose their routes selfishly, the resulting traffic configuration may be very inefficient. Because of this, we consider how to influence human decisions so as to decrease congestion on these roads. We consider a network of parallel roads with two modes of transportation: (i) human drivers who will choose the quickest route available to them, and (ii) ride hailing service which provides an array of autonomous vehicle ride options, each with different prices, to users. In this work, we seek to design these prices so that when autonomous service users choose from these options and human drivers selfishly choose their resulting routes, road usage is maximized and transit delay is minimized. To do so, we formalize a model of how autonomous service users make choices between routes with different price/delay values. Develo** a preference-based algorithm to learn the preferences of the users, and using a vehicle flow model related to the Fundamental Diagram of Traffic, we formulate a planning optimization to maximize a social objective and demonstrate the benefit of the proposed routing and learning scheme.
△ Less
Submitted 9 April, 2019; v1 submitted 3 April, 2019;
originally announced April 2019.
-
Altruistic Autonomy: Beating Congestion on Shared Roads
Authors:
Erdem Bıyık,
Daniel Lazar,
Ramtin Pedarsani,
Dorsa Sadigh
Abstract:
Traffic congestion has large economic and social costs. The introduction of autonomous vehicles can potentially reduce this congestion, both by increasing network throughput and by enabling a social planner to incentivize users of autonomous vehicles to take longer routes that can alleviate congestion on more direct roads. We formalize the effects of altruistic autonomy on roads shared between hum…
▽ More
Traffic congestion has large economic and social costs. The introduction of autonomous vehicles can potentially reduce this congestion, both by increasing network throughput and by enabling a social planner to incentivize users of autonomous vehicles to take longer routes that can alleviate congestion on more direct roads. We formalize the effects of altruistic autonomy on roads shared between human drivers and autonomous vehicles. In this work, we develop a formal model of road congestion on shared roads based on the fundamental diagram of traffic. We consider a network of parallel roads and provide algorithms that compute optimal equilibria that are robust to additional unforeseen demand. We further plan for optimal routings when users have varying degrees of altruism. We find that even with arbitrarily small altruism, total latency can be unboundedly better than without altruism, and that the best selfish equilibrium can be unboundedly better than the worst selfish equilibrium. We validate our theoretical results through microscopic traffic simulations and show average latency decrease of a factor of 4 from worst-case selfish equilibrium to the optimal equilibrium when autonomous vehicles are altruistic.
△ Less
Submitted 29 October, 2018;
originally announced October 2018.
-
Routing for Traffic Networks with Mixed Autonomy
Authors:
Daniel A. Lazar,
Sam Coogan,
Ramtin Pedarsani
Abstract:
In this work we propose a macroscopic model for studying routing on networks shared between human-driven and autonomous vehicles that captures the effects of autonomous vehicles forming platoons. We use this to study inefficiency due to selfish routing and bound the Price of Anarchy (PoA), the maximum ratio between total delay experienced by selfish users and the minimum possible total delay. To d…
▽ More
In this work we propose a macroscopic model for studying routing on networks shared between human-driven and autonomous vehicles that captures the effects of autonomous vehicles forming platoons. We use this to study inefficiency due to selfish routing and bound the Price of Anarchy (PoA), the maximum ratio between total delay experienced by selfish users and the minimum possible total delay. To do so, we establish two road capacity models, each corresponding to an assumption regarding the platooning capabilities of autonomous vehicles. Using these we develop a class of road delay functions, parameterized by the road capacity, that are polynomial with respect to vehicle flow. We then bound the PoA and the bicriteria, another measure of the inefficiency due to selfish routing. We find these bounds depend on: 1) the degree of the polynomial in the road cost function and 2) the degree of asymmetry, the difference in how human-driven and autonomous traffic affect congestion. We demonstrate that these bounds recover the classical bounds when no asymmetry exists. We show the bounds are tight in certain cases and that the PoA bound is order-optimal with respect to the degree of asymmetry.
△ Less
Submitted 4 September, 2018;
originally announced September 2018.
-
What's a little leakage between friends?
Authors:
Sebastian Angel,
David Lazar,
Ioanna Tzialla
Abstract:
This paper introduces a new attack on recent messaging systems that protect communication metadata. The main observation is that if an adversary manages to compromise a user's friend, it can use this compromised friend to learn information about the user's other ongoing conversations. Specifically, the adversary learns whether a user is sending other messages or not, which opens the door to existi…
▽ More
This paper introduces a new attack on recent messaging systems that protect communication metadata. The main observation is that if an adversary manages to compromise a user's friend, it can use this compromised friend to learn information about the user's other ongoing conversations. Specifically, the adversary learns whether a user is sending other messages or not, which opens the door to existing intersection and disclosure attacks. To formalize this compromised friend attack, we present an abstract scenario called the exclusive call center problem that captures the attack's root cause, and demonstrates that it is independent of the particular design or implementation of existing metadata-private messaging systems. We then introduce a new primitive called a private answering machine that can prevent the attack. Unfortunately, building a secure and efficient instance of this primitive under only computational hardness assumptions does not appear possible. Instead, we give a construction under the assumption that users can place a bound on their maximum number of friends and are okay leaking this information.
△ Less
Submitted 23 October, 2018; v1 submitted 1 September, 2018;
originally announced September 2018.
-
Maximizing Road Capacity Using Cars that Influence People
Authors:
Daniel A. Lazar,
Kabir Chandrasekher,
Ramtin Pedarsani,
Dorsa Sadigh
Abstract:
The emerging technology enabling autonomy in vehicles has led to a variety of new problems in transportation networks, such as planning and perception for autonomous vehicles. Other works consider social objectives such as decreasing fuel consumption and travel time by platooning. However, these strategies are limited by the actions of the surrounding human drivers. In this paper, we consider proa…
▽ More
The emerging technology enabling autonomy in vehicles has led to a variety of new problems in transportation networks, such as planning and perception for autonomous vehicles. Other works consider social objectives such as decreasing fuel consumption and travel time by platooning. However, these strategies are limited by the actions of the surrounding human drivers. In this paper, we consider proactively achieving these social objectives by influencing human behavior through planned interactions. Our key insight is that we can use these social objectives to design local interactions that influence human behavior to achieve these goals. To this end, we characterize the increase in road capacity afforded by platooning, as well as the vehicle configuration that maximizes road capacity. We present a novel algorithm that uses a low-level control framework to leverage local interactions to optimally rearrange vehicles. We showcase our algorithm using a simulated road shared between autonomous and human-driven vehicles, in which we illustrate the reordering in action.
△ Less
Submitted 9 October, 2018; v1 submitted 11 July, 2018;
originally announced July 2018.
