Learning to Decode the Surface Code with a Recurrent, Transformer-Based Neural Network
Authors:
Johannes Bausch,
Andrew W Senior,
Francisco J H Heras,
Thomas Edlich,
Alex Davies,
Michael Newman,
Cody Jones,
Kevin Satzinger,
Murphy Yuezhen Niu,
Sam Blackwell,
George Holland,
Dvir Kafri,
Juan Atalaya,
Craig Gidney,
Demis Hassabis,
Sergio Boixo,
Hartmut Neven,
Pushmeet Kohli
Abstract:
Quantum error-correction is a prerequisite for reliable quantum computation. Towards this goal, we present a recurrent, transformer-based neural network which learns to decode the surface code, the leading quantum error-correction code. Our decoder outperforms state-of-the-art algorithmic decoders on real-world data from Google's Sycamore quantum processor for distance 3 and 5 surface codes. On di…
▽ More
Quantum error-correction is a prerequisite for reliable quantum computation. Towards this goal, we present a recurrent, transformer-based neural network which learns to decode the surface code, the leading quantum error-correction code. Our decoder outperforms state-of-the-art algorithmic decoders on real-world data from Google's Sycamore quantum processor for distance 3 and 5 surface codes. On distances up to 11, the decoder maintains its advantage on simulated data with realistic noise including cross-talk, leakage, and analog readout signals, and sustains its accuracy far beyond the 25 cycles it was trained on. Our work illustrates the ability of machine learning to go beyond human-designed algorithms by learning from data directly, highlighting machine learning as a strong contender for decoding in quantum computers.
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
Molecular representation learning with language models and domain-relevant auxiliary tasks
Authors:
Benedek Fabian,
Thomas Edlich,
Héléna Gaspar,
Marwin Segler,
Joshua Meyers,
Marco Fiscato,
Mohamed Ahmed
Abstract:
We apply a Transformer architecture, specifically BERT, to learn flexible and high quality molecular representations for drug discovery problems. We study the impact of using different combinations of self-supervised tasks for pre-training, and present our results for the established Virtual Screening and QSAR benchmarks. We show that: i) The selection of appropriate self-supervised task(s) for pr…
▽ More
We apply a Transformer architecture, specifically BERT, to learn flexible and high quality molecular representations for drug discovery problems. We study the impact of using different combinations of self-supervised tasks for pre-training, and present our results for the established Virtual Screening and QSAR benchmarks. We show that: i) The selection of appropriate self-supervised task(s) for pre-training has a significant impact on performance in subsequent downstream tasks such as Virtual Screening. ii) Using auxiliary tasks with more domain relevance for Chemistry, such as learning to predict calculated molecular properties, increases the fidelity of our learnt representations. iii) Finally, we show that molecular representations learnt by our model `MolBert' improve upon the current state of the art on the benchmark datasets.
△ Less
Submitted 26 November, 2020;
originally announced November 2020.
