InvBERT: Reconstructing Text from Contextualized Word Embeddings by inverting the BERT pipeline
Authors:
Kai Kugler,
Simon Münker,
Johannes Höhmann,
Achim Rettinger
Abstract:
Digital Humanities and Computational Literary Studies apply text mining methods to investigate literature. Such automated approaches enable quantitative studies on large corpora which would not be feasible by manual inspection alone. However, due to copyright restrictions, the availability of relevant digitized literary works is limited. Derived Text Formats (DTFs) have been proposed as a solution…
▽ More
Digital Humanities and Computational Literary Studies apply text mining methods to investigate literature. Such automated approaches enable quantitative studies on large corpora which would not be feasible by manual inspection alone. However, due to copyright restrictions, the availability of relevant digitized literary works is limited. Derived Text Formats (DTFs) have been proposed as a solution. Here, textual materials are transformed in such a way that copyright-critical features are removed, but that the use of certain analytical methods remains possible. Contextualized word embeddings produced by transformer-encoders (like BERT) are promising candidates for DTFs because they allow for state-of-the-art performance on various analytical tasks and, at first sight, do not disclose the original text. However, in this paper we demonstrate that under certain conditions the reconstruction of the original copyrighted text becomes feasible and its publication in the form of contextualized token representations is not safe. Our attempts to invert BERT suggest, that publishing the encoder as a black box together with the contextualized embeddings is critical, since it allows to generate data to train a decoder with a reconstruction accuracy sufficient to violate copyright laws.
△ Less
Submitted 22 December, 2022; v1 submitted 21 September, 2021;
originally announced September 2021.
All-Indoor Optical Customer Premises Equipment for Fixed Wireless Access
Authors:
Dominic Schulz,
Julian Hohmann,
Peter Hellwig,
Christoph Kottke,
Ronald Freund,
Volker Jungnickel,
Ralf-Peter Braun,
Frank Geilhardt
Abstract:
We demonstrate an LED-based optical wireless link for fixed wireless access applications, at data rates of 1.5 Gbit/s over 50 m. Transmission between indoor equipment and outdoor access point is possible through metal-coated insulation glass.
We demonstrate an LED-based optical wireless link for fixed wireless access applications, at data rates of 1.5 Gbit/s over 50 m. Transmission between indoor equipment and outdoor access point is possible through metal-coated insulation glass.
△ Less
Submitted 26 May, 2021;
originally announced July 2021.
