-
Wavelets Are All You Need for Autoregressive Image Generation
Authors:
Wael Mattar,
Idan Levy,
Nir Sharon,
Shai Dekel
Abstract:
In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visual details of an image from coarse to fine details by ordering the information starting with the most significant bits of the most significant wavelet coefficients. The second is a variant of a language transformer whose…
▽ More
In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visual details of an image from coarse to fine details by ordering the information starting with the most significant bits of the most significant wavelet coefficients. The second is a variant of a language transformer whose architecture is re-designed and optimized for token sequences in this 'wavelet language'. The transformer learns the significant statistical correlations within a token sequence, which are the manifestations of well-known correlations between the wavelet subbands at various resolutions. We show experimental results with conditioning on the generation process.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
Diverse Demonstrations Improve In-context Compositional Generalization
Authors:
Itay Levy,
Ben Bogin,
Jonathan Berant
Abstract:
In-context learning has shown great success in i.i.d semantic parsing splits, where the training and test sets are drawn from the same distribution. In this setup, models are typically prompted with demonstrations that are similar to the input utterance. However, in the setup of compositional generalization, where models are tested on outputs with structures that are absent from the training set,…
▽ More
In-context learning has shown great success in i.i.d semantic parsing splits, where the training and test sets are drawn from the same distribution. In this setup, models are typically prompted with demonstrations that are similar to the input utterance. However, in the setup of compositional generalization, where models are tested on outputs with structures that are absent from the training set, selecting similar demonstrations is insufficient, as often no example will be similar enough to the input. In this work, we propose a method to select diverse demonstrations that aims to collectively cover all of the structures required in the output program, in order to encourage the model to generalize to new structures from these demonstrations. We empirically show that combining diverse demonstrations with in-context learning substantially improves performance across three compositional generalization semantic parsing datasets in the pure in-context learning setup and when combined with finetuning.
△ Less
Submitted 24 June, 2023; v1 submitted 13 December, 2022;
originally announced December 2022.
-
Thoughts on child safety on commodity platforms
Authors:
Ian Levy,
Crispin Robinson
Abstract:
The explosion of global social media and online communication platforms has changed how we interact with each other and as a society, bringing with it new security and privacy challenges. Like all technologies, these platforms can be abused and they are routinely used to attempt to cause harm at scale. One of the most significant offence types that is enabled by these platforms is child sexual abu…
▽ More
The explosion of global social media and online communication platforms has changed how we interact with each other and as a society, bringing with it new security and privacy challenges. Like all technologies, these platforms can be abused and they are routinely used to attempt to cause harm at scale. One of the most significant offence types that is enabled by these platforms is child sexual abuse - both scaling existing abuse and enabling entirely new types of online-only abuse where the impacts on the victim are equally catastrophic. Many platforms invest significantly in combating this crime, referring confirmed evidence of illegality to law enforcement. The introduction of end-to-end encryption and similar technologies breaks many of the mitigations in place today and this has led to a debate around the apparent dichotomy of good child safety and good general user privacy and security. This debate has concentrated on the problem of detecting offenders sharing known abuse imagery using a technique known as client side scanning. We will show that the real problem of online child sexual abuse is much more complex than offender image sharing, providing a new set of 'harm archetypes' to better group harms into categories that have similar technical characteristics and, as far as we are able, bring more clarity to the processes currently used by platforms and law enforcement in relation to child sexual abuse content and the real world impacts. We explore, at a high level, a variety of techniques that could be used as part of any potential solution and examine the benefits and disbenefits that may accrue in various use cases, and use a hypothetical service as an example of how various techniques could be brought together to provide both user privacy and security, while protecting child safety and enabling law enforcement action.
△ Less
Submitted 19 July, 2022;
originally announced July 2022.
