-
Once Burned, Twice Shy? The Effect of Stock Market Bubbles on Traders that Learn by Experience
Authors:
Haibei Zhu,
Svitlana Vyetrenko,
Serafin Grundl,
David Byrd,
Kshama Dwarakanath,
Tucker Balch
Abstract:
We study how experience with asset price bubbles changes the trading strategies of reinforcement learning (RL) traders and ask whether the change in trading strategies helps to prevent future bubbles. We train the RL traders in a multi-agent market simulation platform, ABIDES, and compare the strategies of traders trained with and without bubble experience. We find that RL traders without bubble e…
▽ More
We study how experience with asset price bubbles changes the trading strategies of reinforcement learning (RL) traders and ask whether the change in trading strategies helps to prevent future bubbles. We train the RL traders in a multi-agent market simulation platform, ABIDES, and compare the strategies of traders trained with and without bubble experience. We find that RL traders without bubble experience behave like short-term momentum traders, whereas traders with bubble experience behave like value traders. Therefore, RL traders without bubble experience amplify bubbles, whereas RL traders with bubble experience tend to suppress and sometimes prevent them. This finding suggests that learning from experience is a mechanism for a boom and bust cycle where the experience of a collapsing bubble makes future bubbles less likely for a period of time until the memory fades and bubbles become more likely to form again.
△ Less
Submitted 28 December, 2023;
originally announced December 2023.
-
Learning Not to Spoof
Authors:
David Byrd
Abstract:
As intelligent trading agents based on reinforcement learning (RL) gain prevalence, it becomes more important to ensure that RL agents obey laws, regulations, and human behavioral expectations. There is substantial literature concerning the aversion of obvious catastrophes like crashing a helicopter or bankrupting a trading account, but little around the avoidance of subtle non-normative behavior…
▽ More
As intelligent trading agents based on reinforcement learning (RL) gain prevalence, it becomes more important to ensure that RL agents obey laws, regulations, and human behavioral expectations. There is substantial literature concerning the aversion of obvious catastrophes like crashing a helicopter or bankrupting a trading account, but little around the avoidance of subtle non-normative behavior for which there are examples, but no programmable definition. Such behavior may violate legal or regulatory, rather than physical or monetary, constraints.
In this article, I consider a series of experiments in which an intelligent stock trading agent maximizes profit but may also inadvertently learn to spoof the market in which it participates. I first inject a hand-coded spoofing agent to a multi-agent market simulation and learn to recognize spoofing activity sequences. Then I replace the hand-coded spoofing trader with a simple profit-maximizing RL agent and observe that it independently discovers spoofing as the optimal strategy. Finally, I introduce a method to incorporate the recognizer as normative guide, sha** the agent's perceived rewards and altering its selected actions. The agent remains profitable while avoiding spoofing behaviors that would result in even higher profit. After presenting the empirical results, I conclude with some recommendations. The method should generalize to the reduction of any unwanted behavior for which a recognizer can be learned.
△ Less
Submitted 9 June, 2023;
originally announced June 2023.
-
Collusion Resistant Federated Learning with Oblivious Distributed Differential Privacy
Authors:
David Byrd,
Vaikkunth Mugunthan,
Antigoni Polychroniadou,
Tucker Hybinette Balch
Abstract:
Privacy-preserving federated learning enables a population of distributed clients to jointly learn a shared model while kee** client training data private, even from an untrusted server. Prior works do not provide efficient solutions that protect against collusion attacks in which parties collaborate to expose an honest client's model parameters. We present an efficient mechanism based on oblivi…
▽ More
Privacy-preserving federated learning enables a population of distributed clients to jointly learn a shared model while kee** client training data private, even from an untrusted server. Prior works do not provide efficient solutions that protect against collusion attacks in which parties collaborate to expose an honest client's model parameters. We present an efficient mechanism based on oblivious distributed differential privacy that is the first to protect against such client collusion, including the "Sybil" attack in which a server preferentially selects compromised devices or simulates fake devices. We leverage the novel privacy mechanism to construct a secure federated learning protocol and prove the security of that protocol. We conclude with empirical analysis of the protocol's execution speed, learning accuracy, and privacy performance on two data sets within a realistic simulation of 5,000 distributed network clients.
△ Less
Submitted 20 February, 2022;
originally announced February 2022.
-
A multispeaker dataset of raw and reconstructed speech production real-time MRI video and 3D volumetric images
Authors:
Yongwan Lim,
Asterios Toutios,
Yannick Bliesener,
Ye Tian,
Sajan Goud Lingala,
Colin Vaz,
Tanner Sorensen,
Miran Oh,
Sarah Harper,
Weiyi Chen,
Yoonjeong Lee,
Johannes Töger,
Mairym Lloréns Montesserin,
Caitlin Smith,
Bianca Godinez,
Louis Goldstein,
Dani Byrd,
Krishna S. Nayak,
Shrikanth S. Narayanan
Abstract:
Real-time magnetic resonance imaging (RT-MRI) of human speech production is enabling significant advances in speech science, linguistics, bio-inspired speech technology development, and clinical applications. Easy access to RT-MRI is however limited, and comprehensive datasets with broad access are needed to catalyze research across numerous domains. The imaging of the rapidly moving articulators…
▽ More
Real-time magnetic resonance imaging (RT-MRI) of human speech production is enabling significant advances in speech science, linguistics, bio-inspired speech technology development, and clinical applications. Easy access to RT-MRI is however limited, and comprehensive datasets with broad access are needed to catalyze research across numerous domains. The imaging of the rapidly moving articulators and dynamic airway sha** during speech demands high spatio-temporal resolution and robust reconstruction methods. Further, while reconstructed images have been published, to-date there is no open dataset providing raw multi-coil RT-MRI data from an optimized speech production experimental setup. Such datasets could enable new and improved methods for dynamic image reconstruction, artifact correction, feature extraction, and direct extraction of linguistically-relevant biomarkers. The present dataset offers a unique corpus of 2D sagittal-view RT-MRI videos along with synchronized audio for 75 subjects performing linguistically motivated speech tasks, alongside the corresponding first-ever public domain raw RT-MRI data. The dataset also includes 3D volumetric vocal tract MRI during sustained speech sounds and high-resolution static anatomical T2-weighted upper airway MRI for each subject.
△ Less
Submitted 15 February, 2021;
originally announced February 2021.
-
Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applications
Authors:
David Byrd,
Antigoni Polychroniadou
Abstract:
Federated Learning enables a population of clients, working with a trusted server, to collaboratively learn a shared machine learning model while kee** each client's data within its own local systems. This reduces the risk of exposing sensitive data, but it is still possible to reverse engineer information about a client's private data set from communicated model parameters. Most federated learn…
▽ More
Federated Learning enables a population of clients, working with a trusted server, to collaboratively learn a shared machine learning model while kee** each client's data within its own local systems. This reduces the risk of exposing sensitive data, but it is still possible to reverse engineer information about a client's private data set from communicated model parameters. Most federated learning systems therefore use differential privacy to introduce noise to the parameters. This adds uncertainty to any attempt to reveal private client data, but also reduces the accuracy of the shared model, limiting the useful scale of privacy-preserving noise. A system can further reduce the coordinating server's ability to recover private client information, without additional accuracy loss, by also including secure multiparty computation. An approach combining both techniques is especially relevant to financial firms as it allows new possibilities for collaborative learning without exposing sensitive client data. This could produce more accurate models for important tasks like optimal trade execution, credit origination, or fraud detection. The key contributions of this paper are: We present a privacy-preserving federated learning protocol to a non-specialist audience, demonstrate it using logistic regression on a real-world credit card fraud data set, and evaluate it using an open-source simulation platform which we have adapted for the development of federated learning systems.
△ Less
Submitted 12 October, 2020;
originally announced October 2020.
-
The Importance of Low Latency to Order Book Imbalance Trading Strategies
Authors:
David Byrd,
Sruthi Palaparthi,
Maria Hybinette,
Tucker Hybinette Balch
Abstract:
There is a pervasive assumption that low latency access to an exchange is a key factor in the profitability of many high-frequency trading strategies. This belief is evidenced by the "arms race" undertaken by certain financial firms to co-locate with exchange servers. To the best of our knowledge, our study is the first to validate and quantify this assumption in a continuous double auction market…
▽ More
There is a pervasive assumption that low latency access to an exchange is a key factor in the profitability of many high-frequency trading strategies. This belief is evidenced by the "arms race" undertaken by certain financial firms to co-locate with exchange servers. To the best of our knowledge, our study is the first to validate and quantify this assumption in a continuous double auction market with a single exchange similar to the New York Stock Exchange. It is not feasible to conduct this exploration with historical data in which trader identity and location are not reported. Accordingly, we investigate the relationship between latency of access to order book information and profitability of trading strategies exploiting that information with an agent-based interactive discrete event simulation in which thousands of agents pursue archetypal trading strategies. We introduce experimental traders pursuing a low-latency order book imbalance (OBI) strategy in a controlled manner across thousands of simulated trading days, and analyze OBI trader profit while varying distance (latency) from the exchange. Our experiments support that latency is inversely related to profit for the OBI traders, but more interestingly show that latency rank, rather than absolute magnitude, is the key factor in allocating returns among agents pursuing a similar strategy.
△ Less
Submitted 15 June, 2020;
originally announced June 2020.
-
Get Real: Realism Metrics for Robust Limit Order Book Market Simulations
Authors:
Svitlana Vyetrenko,
David Byrd,
Nick Petosa,
Mahmoud Mahfouz,
Danial Dervovic,
Manuela Veloso,
Tucker Hybinette Balch
Abstract:
Machine learning (especially reinforcement learning) methods for trading are increasingly reliant on simulation for agent training and testing. Furthermore, simulation is important for validation of hand-coded trading strategies and for testing hypotheses about market structure. A challenge, however, concerns the robustness of policies validated in simulation because the simulations lack fidelity.…
▽ More
Machine learning (especially reinforcement learning) methods for trading are increasingly reliant on simulation for agent training and testing. Furthermore, simulation is important for validation of hand-coded trading strategies and for testing hypotheses about market structure. A challenge, however, concerns the robustness of policies validated in simulation because the simulations lack fidelity. In fact, researchers have shown that many market simulation approaches fail to reproduce statistics and stylized facts seen in real markets. As a step towards addressing this we surveyed the literature to collect a set of reference metrics and applied them to real market data and simulation output. Our paper provides a comprehensive catalog of these metrics including mathematical formulations where appropriate. Our results show that there are still significant discrepancies between simulated markets and real ones. However, this work serves as a benchmark against which we can measure future improvement.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
Explaining Agent-Based Financial Market Simulation
Authors:
David Byrd
Abstract:
This paper is intended to explain, in simple terms, some of the mechanisms and agents common to multiagent financial market simulations. We first discuss the necessity to include an exogenous price time series ("the fundamental value") for each asset and three methods for generating that series. We then illustrate one process by which a Bayesian agent may receive limited observations of the fundam…
▽ More
This paper is intended to explain, in simple terms, some of the mechanisms and agents common to multiagent financial market simulations. We first discuss the necessity to include an exogenous price time series ("the fundamental value") for each asset and three methods for generating that series. We then illustrate one process by which a Bayesian agent may receive limited observations of the fundamental series and estimate its current and future values. Finally, we present two such agents widely examined in the literature, the Zero Intelligence agent and the Heuristic Belief Learning agent, which implement different approaches to order placement.
△ Less
Submitted 25 September, 2019;
originally announced September 2019.
-
Intra-day Equity Price Prediction using Deep Learning as a Measure of Market Efficiency
Authors:
David Byrd,
Tucker Hybinette Balch
Abstract:
In finance, the weak form of the Efficient Market Hypothesis asserts that historic stock price and volume data cannot inform predictions of future prices. In this paper we show that, to the contrary, future intra-day stock prices could be predicted effectively until 2009. We demonstrate this using two different profitable machine learning-based trading strategies. However, the effectiveness of bot…
▽ More
In finance, the weak form of the Efficient Market Hypothesis asserts that historic stock price and volume data cannot inform predictions of future prices. In this paper we show that, to the contrary, future intra-day stock prices could be predicted effectively until 2009. We demonstrate this using two different profitable machine learning-based trading strategies. However, the effectiveness of both approaches diminish over time, and neither of them are profitable after 2009. We present our implementation and results in detail for the period 2003-2017 and propose a novel idea: the use of such flexible machine learning methods as an objective measure of relative market efficiency. We conclude with a candidate explanation, comparing our returns over time with high-frequency trading volume, and suggest concrete steps for further investigation.
△ Less
Submitted 21 August, 2019;
originally announced August 2019.
-
How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?
Authors:
Tucker Hybinette Balch,
Mahmoud Mahfouz,
Joshua Lockhart,
Maria Hybinette,
David Byrd
Abstract:
We show how a multi-agent simulator can support two important but distinct methods for assessing a trading strategy: Market Replay and Interactive Agent-Based Simulation (IABS). Our solution is important because each method offers strengths and weaknesses that expose or conceal flaws in the subject strategy. A key weakness of Market Replay is that the simulated market does not substantially adapt…
▽ More
We show how a multi-agent simulator can support two important but distinct methods for assessing a trading strategy: Market Replay and Interactive Agent-Based Simulation (IABS). Our solution is important because each method offers strengths and weaknesses that expose or conceal flaws in the subject strategy. A key weakness of Market Replay is that the simulated market does not substantially adapt to or respond to the presence of the experimental strategy. IABS methods provide an artificial market for the experimental strategy using a population of background trading agents. Because the background agents attend to market conditions and current price as part of their strategy, the overall market is responsive to the presence of the experimental strategy. Even so, IABS methods have their own weaknesses, primarily that it is unclear if the market environment they provide is realistic. We describe our approach in detail, and illustrate its use in an example application: The evaluation of market impact for various size orders.
△ Less
Submitted 27 June, 2019;
originally announced June 2019.
-
ABIDES: Towards High-Fidelity Market Simulation for AI Research
Authors:
David Byrd,
Maria Hybinette,
Tucker Hybinette Balch
Abstract:
We introduce ABIDES, an Agent-Based Interactive Discrete Event Simulation environment. ABIDES is designed from the ground up to support AI agent research in market applications. While simulations are certainly available within trading firms for their own internal use, there are no broadly available high-fidelity market simulation environments. We hope that the availability of such a platform will…
▽ More
We introduce ABIDES, an Agent-Based Interactive Discrete Event Simulation environment. ABIDES is designed from the ground up to support AI agent research in market applications. While simulations are certainly available within trading firms for their own internal use, there are no broadly available high-fidelity market simulation environments. We hope that the availability of such a platform will facilitate AI research in this important area. ABIDES currently enables the simulation of tens of thousands of trading agents interacting with an exchange agent to facilitate transactions. It supports configurable pairwise network latencies between each individual agent as well as the exchange. Our simulator's message-based design is modeled after NASDAQ's published equity trading protocols ITCH and OUCH. We introduce the design of the simulator and illustrate its use and configuration with sample code, validating the environment with example trading scenarios. The utility of ABIDES is illustrated through experiments to develop a market impact model. We close with discussion of future experimental problems it can be used to explore, such as the development of ML-based trading algorithms.
△ Less
Submitted 26 April, 2019;
originally announced April 2019.
-
A Scrollbar-based Visualization for Document Navigation
Authors:
Donald Byrd
Abstract:
We are interested in questions of improving user control in best-match text-retrieval systems, specifically questions as to whether simple visualizations that nonetheless go beyond the minimal ones generally available can significantly help users. Recently, we have been investigating ways to help users decide-given a set of documents retrieved by a query-which documents and passages are worth cl…
▽ More
We are interested in questions of improving user control in best-match text-retrieval systems, specifically questions as to whether simple visualizations that nonetheless go beyond the minimal ones generally available can significantly help users. Recently, we have been investigating ways to help users decide-given a set of documents retrieved by a query-which documents and passages are worth closer examination. We built a document viewer incorporating a visualization centered around a novel content-displaying scrollbar and color term highlighting, and studied whether the visualization is helpful to non-expert searchers. Participants' reaction to the visualization was very positive, while the objective results were inconclusive.
△ Less
Submitted 25 February, 1999; v1 submitted 24 February, 1999;
originally announced February 1999.
