Hiding Text in Large Language Models: Introducing Unconditional Token Forcing Confusion
Authors:
Jakub Hoscilowicz,
Pawel Popiolek,
Jan Rudkowski,
Jedrzej Bieniasz,
Artur Janicki
Abstract:
With the help of simple fine-tuning, one can artificially embed hidden text into large language models (LLMs). This text is revealed only when triggered by a specific query to the LLM. Two primary applications are LLM fingerprinting and steganography. In the context of LLM fingerprinting, a unique text identifier (fingerprint) is embedded within the model to verify licensing compliance. In the con…
▽ More
With the help of simple fine-tuning, one can artificially embed hidden text into large language models (LLMs). This text is revealed only when triggered by a specific query to the LLM. Two primary applications are LLM fingerprinting and steganography. In the context of LLM fingerprinting, a unique text identifier (fingerprint) is embedded within the model to verify licensing compliance. In the context of steganography, the LLM serves as a carrier for hidden messages that can be disclosed through a designated trigger.
Our work demonstrates that embedding hidden text in the LLM via fine-tuning, though seemingly secure due to the vast number of potential triggers (any sequence of characters or tokens could serve as a trigger), is susceptible to extraction through analysis of the LLM's output decoding process. We propose a novel approach to extraction called Unconditional Token Forcing. It is premised on the hypothesis that iteratively feeding each token from the LLM's vocabulary into the model should reveal sequences with abnormally high token probabilities, indicating potential embedded text candidates. Additionally, our experiments show that when the first token of a hidden fingerprint is used as an input, the LLM not only produces an output sequence with high token probabilities, but also repetitively generates the fingerprint itself. We also present a method to hide text in such a way that it is resistant to Unconditional Token Forcing, which we named Unconditional Token Forcing Confusion.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
SocialStegDisc: Application of steganography in social networks to create a file system
Authors:
Jedrzej Bieniasz,
Krzysztof Szczypiorski
Abstract:
The concept named SocialStegDisc was introduced as an application of the original idea of StegHash method. This new kind of mass-storage was characterized by unlimited space. The design also attempted to improve the operation of StegHash by trade-off between memory requirements and computation time. Applying the mechanism of linked list provided the set of operations on files: creation, reading, d…
▽ More
The concept named SocialStegDisc was introduced as an application of the original idea of StegHash method. This new kind of mass-storage was characterized by unlimited space. The design also attempted to improve the operation of StegHash by trade-off between memory requirements and computation time. Applying the mechanism of linked list provided the set of operations on files: creation, reading, deletion and modification. Features, limitations and opportunities were discussed.
△ Less
Submitted 29 June, 2017;
originally announced June 2017.
