-
Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit's Showerthoughts
Authors:
Tolga Buz,
Benjamin Frost,
Nikola Genchev,
Moritz Schneider,
Lucie-Aimée Kaffee,
Gerard de Melo
Abstract:
Recent Large Language Models (LLMs) have shown the ability to generate content that is difficult or impossible to distinguish from human writing. We investigate the ability of differently-sized LLMs to replicate human writing style in short, creative texts in the domain of Showerthoughts, thoughts that may occur during mundane activities. We compare GPT-2 and GPT-Neo fine-tuned on Reddit data as w…
▽ More
Recent Large Language Models (LLMs) have shown the ability to generate content that is difficult or impossible to distinguish from human writing. We investigate the ability of differently-sized LLMs to replicate human writing style in short, creative texts in the domain of Showerthoughts, thoughts that may occur during mundane activities. We compare GPT-2 and GPT-Neo fine-tuned on Reddit data as well as GPT-3.5 invoked in a zero-shot manner, against human-authored texts. We measure human preference on the texts across the specific dimensions that account for the quality of creative, witty texts. Additionally, we compare the ability of humans versus fine-tuned RoBERTa classifiers to detect AI-generated texts. We conclude that human evaluators rate the generated texts slightly worse on average regarding their creative quality, but they are unable to reliably distinguish between human-written and AI-generated texts. We further provide a dataset for creative, witty text generation based on Reddit Showerthoughts posts.
△ Less
Submitted 2 May, 2024;
originally announced May 2024.
-
FARSEC: A Reproducible Framework for Automatic Real-Time Vehicle Speed Estimation Using Traffic Cameras
Authors:
Lucas Liebe,
Franz Sauerwald,
Sylwester Sawicki,
Matthias Schneider,
Leo Schuhmann,
Tolga Buz,
Paul Boes,
Ahmad Ahmadov,
Gerard de Melo
Abstract:
Estimating the speed of vehicles using traffic cameras is a crucial task for traffic surveillance and management, enabling more optimal traffic flow, improved road safety, and lower environmental impact. Transportation-dependent systems, such as for navigation and logistics, have great potential to benefit from reliable speed estimation. While there is prior research in this area reporting competi…
▽ More
Estimating the speed of vehicles using traffic cameras is a crucial task for traffic surveillance and management, enabling more optimal traffic flow, improved road safety, and lower environmental impact. Transportation-dependent systems, such as for navigation and logistics, have great potential to benefit from reliable speed estimation. While there is prior research in this area reporting competitive accuracy levels, their solutions lack reproducibility and robustness across different datasets. To address this, we provide a novel framework for automatic real-time vehicle speed calculation, which copes with more diverse data from publicly available traffic cameras to achieve greater robustness. Our model employs novel techniques to estimate the length of road segments via depth map prediction. Additionally, our framework is capable of handling realistic conditions such as camera movements and different video stream inputs automatically. We compare our model to three well-known models in the field using their benchmark datasets. While our model does not set a new state of the art regarding prediction performance, the results are competitive on realistic CCTV videos. At the same time, our end-to-end pipeline offers more consistent results, an easier implementation, and better compatibility. Its modular structure facilitates reproducibility and future improvements.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
Democratization of Retail Trading: Can Reddit's WallStreetBets Outperform Investment Bank Analysts?
Authors:
Tolga Buz,
Gerard de Melo
Abstract:
The recent hype around Reddit's WallStreetBets (WSB) community has inspired research on its impact on our economy and society. Still, one important question remains: Can WSB's community of anonymous contributors actually provide valuable investment advice and possibly even outperform top financial institutions? We present a data-driven empirical study of investment recommendations of WSB in compar…
▽ More
The recent hype around Reddit's WallStreetBets (WSB) community has inspired research on its impact on our economy and society. Still, one important question remains: Can WSB's community of anonymous contributors actually provide valuable investment advice and possibly even outperform top financial institutions? We present a data-driven empirical study of investment recommendations of WSB in comparison to recommendations made by leading investment banks, based on more than 1.6 million WSB posts published since 2018. %enriched with stock market data. To this end, we extract and evaluate investment recommendations from WSB's raw text for all S&P 500 stocks and compare their performance to more than 16,000 analyst recommendations from the largest investment banks. While not all WSB recommendations prove profitable, our results show that they achieve average returns that compete with the best banks and outperform them in certain cases. Furthermore, the WSB community has been better than almost all investment banks at detecting top-performing stocks. We conclude that WSB may indeed constitute a freely accessible, valuable source of investment advice.
△ Less
Submitted 31 December, 2022;
originally announced January 2023.
-
Should You Take Investment Advice From WallStreetBets? A Data-Driven Approach
Authors:
Tolga Buz,
Gerard de Melo
Abstract:
Reddit's WallStreetBets (WSB) community has come to prominence in light of its notable role in affecting the stock prices of what are now referred to as meme stocks. Yet very little is known about the reliability of the highly speculative investment advice disseminated on WSB. This paper analyses WSB data spanning from January 2019 to April 2021 in order to assess how successful an investment stra…
▽ More
Reddit's WallStreetBets (WSB) community has come to prominence in light of its notable role in affecting the stock prices of what are now referred to as meme stocks. Yet very little is known about the reliability of the highly speculative investment advice disseminated on WSB. This paper analyses WSB data spanning from January 2019 to April 2021 in order to assess how successful an investment strategy relying on the community's recommendations could have been. We detect buy and sell advice and identify the community's most popular stocks, based on which we define a WSB portfolio. Our evaluation shows that this portfolio has grown approx. 200% over the last three years and approx. 480% over the last year, significantly outperforming the S&P500. The average short-term accuracy of buy and sell signals, in contrast, is not found to be significantly better than randomly or equally distributed buy decisions within the same time frame. However, we present a technique for estimating whether posts are proactive as opposed to reactive and show that by focusing on a subset of more promising buy signals, a trader could have made investments yielding higher returns than the broader market or the strategy of trusting all posted buy signals. Lastly, the analysis is also conducted specifically for the period before 2021 in order to factor out the effects of the GameStop hype of January 2021 - the results confirm the conclusions and suggest that the 2021 hype merely amplified pre-existing characteristics.
△ Less
Submitted 6 May, 2021;
originally announced May 2021.
