-
Mind the Gap: Analyzing Lacunae with Transformer-Based Transcription
Authors:
Jaydeep Borkar,
David A. Smith
Abstract:
Historical documents frequently suffer from damage and inconsistencies, including missing or illegible text resulting from issues such as holes, ink problems, and storage damage. These missing portions or gaps are referred to as lacunae. In this study, we employ transformer-based optical character recognition (OCR) models trained on synthetic data containing lacunae in a supervised manner. We demo…
▽ More
Historical documents frequently suffer from damage and inconsistencies, including missing or illegible text resulting from issues such as holes, ink problems, and storage damage. These missing portions or gaps are referred to as lacunae. In this study, we employ transformer-based optical character recognition (OCR) models trained on synthetic data containing lacunae in a supervised manner. We demonstrate their effectiveness in detecting and restoring lacunae, achieving a success rate of 65%, compared to a base model lacking knowledge of lacunae, which achieves only 5% restoration. Additionally, we investigate the mechanistic properties of the model, such as the log probability of transcription, which can identify lacunae and other errors (e.g., mistranscriptions due to complex writing or ink issues) in line images without directly inspecting the image. This capability could be valuable for scholars seeking to distinguish images containing lacunae or errors from clean ones. Although we explore the potential of attention mechanisms in flagging lacunae and transcription errors, our findings suggest it is not a significant factor. Our work highlights a promising direction in utilizing transformer-based OCR models for restoring or analyzing damaged historical documents.
△ Less
Submitted 28 June, 2024;
originally announced July 2024.
-
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
Authors:
USVSN Sai Prashanth,
Alvin Deng,
Kyle O'Brien,
Jyothir S V,
Mohammad Aflah Khan,
Jaydeep Borkar,
Christopher A. Choquette-Choo,
Jacob Ray Fuehne,
Stella Biderman,
Tracy Ke,
Katherine Lee,
Naomi Saphra
Abstract:
Memorization in language models is typically treated as a homogenous phenomenon, neglecting the specifics of the memorized data. We instead model memorization as the effect of a set of complex factors that describe each sample and relate it to the model and corpus. To build intuition around these factors, we break memorization down into a taxonomy: recitation of highly duplicated sequences, recons…
▽ More
Memorization in language models is typically treated as a homogenous phenomenon, neglecting the specifics of the memorized data. We instead model memorization as the effect of a set of complex factors that describe each sample and relate it to the model and corpus. To build intuition around these factors, we break memorization down into a taxonomy: recitation of highly duplicated sequences, reconstruction of inherently predictable sequences, and recollection of sequences that are neither. We demonstrate the usefulness of our taxonomy by using it to construct a predictive model for memorization. By analyzing dependencies and inspecting the weights of the predictive model, we find that different factors influence the likelihood of memorization differently depending on the taxonomic category.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
What can we learn from Data Leakage and Unlearning for Law?
Authors:
Jaydeep Borkar
Abstract:
Large Language Models (LLMs) have a privacy concern because they memorize training data (including personally identifiable information (PII) like emails and phone numbers) and leak it during inference. A company can train an LLM on its domain-customized data which can potentially also include their users' PII. In order to comply with privacy laws such as the "right to be forgotten", the data point…
▽ More
Large Language Models (LLMs) have a privacy concern because they memorize training data (including personally identifiable information (PII) like emails and phone numbers) and leak it during inference. A company can train an LLM on its domain-customized data which can potentially also include their users' PII. In order to comply with privacy laws such as the "right to be forgotten", the data points of users that are most vulnerable to extraction could be deleted. We find that once the most vulnerable points are deleted, a new set of points become vulnerable to extraction. So far, little attention has been given to understanding memorization for fine-tuned models. In this work, we also show that not only do fine-tuned models leak their training data but they also leak the pre-training data (and PII) memorized during the pre-training phase. The property of new data points becoming vulnerable to extraction after unlearning and leakage of pre-training data through fine-tuned models can pose significant privacy and legal concerns for companies that use LLMs to offer services. We hope this work will start an interdisciplinary discussion within AI and law communities regarding the need for policies to tackle these issues.
△ Less
Submitted 19 July, 2023;
originally announced July 2023.
-
Simple Transparent Adversarial Examples
Authors:
Jaydeep Borkar,
Pin-Yu Chen
Abstract:
There has been a rise in the use of Machine Learning as a Service (MLaaS) Vision APIs as they offer multiple services including pre-built models and algorithms, which otherwise take a huge amount of resources if built from scratch. As these APIs get deployed for high-stakes applications, it's very important that they are robust to different manipulations. Recent works have only focused on typical…
▽ More
There has been a rise in the use of Machine Learning as a Service (MLaaS) Vision APIs as they offer multiple services including pre-built models and algorithms, which otherwise take a huge amount of resources if built from scratch. As these APIs get deployed for high-stakes applications, it's very important that they are robust to different manipulations. Recent works have only focused on typical adversarial attacks when evaluating the robustness of vision APIs. We propose two new aspects of adversarial image generation methods and evaluate them on the robustness of Google Cloud Vision API's optical character recognition service and object detection APIs deployed in real-world settings such as sightengine.com, picpurify.com, Google Cloud Vision API, and Microsoft Azure's Computer Vision API. Specifically, we go beyond the conventional small-noise adversarial attacks and introduce secret embedding and transparent adversarial examples as a simpler way to evaluate robustness. These methods are so straightforward that even non-specialists can craft such attacks. As a result, they pose a serious threat where APIs are used for high-stakes applications. Our transparent adversarial examples successfully evade state-of-the art object detections APIs such as Azure Cloud Vision (attack success rate 52%) and Google Cloud Vision (attack success rate 36%). 90% of the images have a secret embedded text that successfully fools the vision of time-limited humans but is detected by Google Cloud Vision API's optical character recognition. Complementing to current research, our results provide simple but unconventional methods on robustness evaluation.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
The Multifrequency Behavior of Sagittarius A*
Authors:
A. Eckart,
M. Zajacek,
M. Parsa,
E. Hosseini N. Fazeli,
G. Busch,
B. Shahzamanian,
M. Subroweit,
F. Peissker,
N. Sabha,
M. Valencia-S.,
M. Horrobin,
C. Straubmeier,
S. Rost,
J. Schneeloch A. Borkar,
V. Karas,
S. Britzen,
A. Zensus,
F. Kamali
Abstract:
The Galactic Center is the closest galactic nucleus that allows us to determine the multi-frequency behavior of the supermassive black hole counterpart Sagittarius A* in great detail. We put SgrA*, as a nucleus with weak activity, into the context of nearby low luminosity nuclei. Possible hints for galaxy evolution of these sources across the [NII]-based diagnostic diagram can be inferred from dep…
▽ More
The Galactic Center is the closest galactic nucleus that allows us to determine the multi-frequency behavior of the supermassive black hole counterpart Sagittarius A* in great detail. We put SgrA*, as a nucleus with weak activity, into the context of nearby low luminosity nuclei. Possible hints for galaxy evolution of these sources across the [NII]-based diagnostic diagram can be inferred from dependencies on the masses, excitation ratios, and radio luminosities within this diagram. For SgrA* we also discuss responsible radiation mechanisms covering results from the radio, infrared, and X-ray regime. We also address the question of justifying the hot-spot model for describing flare profiles in light curves. Since the brightness of LLAGN is also linked to star formation we briefly discuss the possibility of having stars formed in the immediate vicinity of supermassive black holes and possibly even in a mildly relativistic environment.
△ Less
Submitted 1 June, 2018;
originally announced June 2018.
