-
Augmenting x-ray single particle imaging reconstruction with self-supervised machine learning
Authors:
Zhantao Chen,
Cong Wang,
Mingye Gao,
Chun Hong Yoon,
Jana B. Thayer,
Joshua J. Turner
Abstract:
The development of X-ray Free Electron Lasers (XFELs) has opened numerous opportunities to probe atomic structure and ultrafast dynamics of various materials. Single Particle Imaging (SPI) with XFELs enables the investigation of biological particles in their natural physiological states with unparalleled temporal resolution, while circumventing the need for cryogenic conditions or crystallization.…
▽ More
The development of X-ray Free Electron Lasers (XFELs) has opened numerous opportunities to probe atomic structure and ultrafast dynamics of various materials. Single Particle Imaging (SPI) with XFELs enables the investigation of biological particles in their natural physiological states with unparalleled temporal resolution, while circumventing the need for cryogenic conditions or crystallization. However, reconstructing real-space structures from reciprocal-space x-ray diffraction data is highly challenging due to the absence of phase and orientation information, which is further complicated by weak scattering signals and considerable fluctuations in the number of photons per pulse. In this work, we present an end-to-end, self-supervised machine learning approach to recover particle orientations and estimate reciprocal space intensities from diffraction images only. Our method demonstrates great robustness under demanding experimental conditions with significantly enhanced reconstruction capabilities compared with conventional algorithms, and signifies a paradigm shift in SPI as currently practiced at XFELs.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Machine learning enabled experimental design and parameter estimation for ultrafast spin dynamics
Authors:
Zhantao Chen,
Cheng Peng,
Alexander N. Petsch,
Sathya R. Chitturi,
Alana Okullo,
Sugata Chowdhury,
Chun Hong Yoon,
Joshua J. Turner
Abstract:
Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and material physics, which often suffer from the scarcity of facility resources and increasing complexities. To address the limitations, we introduce a methodology that combines machine learning with Bayesian optimal experimental design (BOED), exemplified with x-r…
▽ More
Advanced experimental measurements are crucial for driving theoretical developments and unveiling novel phenomena in condensed matter and material physics, which often suffer from the scarcity of facility resources and increasing complexities. To address the limitations, we introduce a methodology that combines machine learning with Bayesian optimal experimental design (BOED), exemplified with x-ray photon fluctuation spectroscopy (XPFS) measurements for spin fluctuations. Our method employs a neural network model for large-scale spin dynamics simulations for precise distribution and utility calculations in BOED. The capability of automatic differentiation from the neural network model is further leveraged for more robust and accurate parameter estimation. Our numerical benchmarks demonstrate the superior performance of our method in guiding XPFS experiments, predicting model parameters, and yielding more informative measurements within limited experimental time. Although focusing on XPFS and spin fluctuations, our method can be adapted to other experiments, facilitating more efficient data collection and accelerating scientific discoveries.
△ Less
Submitted 3 June, 2023;
originally announced June 2023.
-
Neural Fine-Gray: Monotonic neural networks for competing risks
Authors:
Vincent Jeanselme,
Chang Ho Yoon,
Brian Tom,
Jessica Barrett
Abstract:
Time-to-event modelling, known as survival analysis, differs from standard regression as it addresses censoring in patients who do not experience the event of interest. Despite competitive performances in tackling this problem, machine learning methods often ignore other competing risks that preclude the event of interest. This practice biases the survival estimation. Extensions to address this ch…
▽ More
Time-to-event modelling, known as survival analysis, differs from standard regression as it addresses censoring in patients who do not experience the event of interest. Despite competitive performances in tackling this problem, machine learning methods often ignore other competing risks that preclude the event of interest. This practice biases the survival estimation. Extensions to address this challenge often rely on parametric assumptions or numerical estimations leading to sub-optimal survival approximations. This paper leverages constrained monotonic neural networks to model each competing survival distribution. This modelling choice ensures the exact likelihood maximisation at a reduced computational cost by using automatic differentiation. The effectiveness of the solution is demonstrated on one synthetic and three medical datasets. Finally, we discuss the implications of considering competing risks when develo** risk scores for medical practice.
△ Less
Submitted 11 May, 2023;
originally announced May 2023.
-
PeakNet: An Autonomous Bragg Peak Finder with Deep Neural Networks
Authors:
Cong Wang,
Po-Nan Li,
Jana Thayer,
Chun Hong Yoon
Abstract:
Serial crystallography at X-ray free electron laser (XFEL) and synchrotron facilities has experienced tremendous progress in recent times enabling novel scientific investigations into macromolecular structures and molecular processes. However, these experiments generate a significant amount of data posing computational challenges in data reduction and real-time feedback. Bragg peak finding algorit…
▽ More
Serial crystallography at X-ray free electron laser (XFEL) and synchrotron facilities has experienced tremendous progress in recent times enabling novel scientific investigations into macromolecular structures and molecular processes. However, these experiments generate a significant amount of data posing computational challenges in data reduction and real-time feedback. Bragg peak finding algorithm is used to identify useful images and also provide real-time feedback about hit-rate and resolution. Shot-to-shot intensity fluctuations and strong background scattering from buffer solution, injection nozzle and other shielding materials make this a time-consuming optimization problem. Here, we present PeakNet, an autonomous Bragg peak finder that utilizes deep neural networks. The development of this system 1) eliminates the need for manual algorithm parameter tuning, 2) reduces false-positive peaks by adjusting to shot-to-shot variations in strong background scattering in real-time, 3) eliminates the laborious task of manually creating bad pixel masks and the need to store these masks per event since these can be regenerated on demand. PeakNet also exhibits exceptional runtime efficiency, processing a 1920-by-1920 pixel image around 90 ms on an NVIDIA 1080 Ti GPU, with the potential for further enhancements through parallelized analysis or GPU stream processing. PeakNet is well-suited for expert-level real-time serial crystallography data analysis at high data rates.
△ Less
Submitted 29 June, 2023; v1 submitted 24 March, 2023;
originally announced March 2023.
-
SpeckleNN: A unified embedding for real-time speckle pattern classification in X-ray single-particle imaging with limited labeled examples
Authors:
Cong Wang,
Eric Florin,
Hsing-Yin Chang,
Jana Thayer,
Chun Hong Yoon
Abstract:
With X-ray free-electron lasers (XFELs), it is possible to determine the three-dimensional structure of noncrystalline nanoscale particles using X-ray single-particle imaging (SPI) techniques at room temperature. Classifying SPI scattering patterns, or "speckles", to extract single hits that are needed for real-time vetoing and three-dimensional reconstruction poses a challenge for high data rate…
▽ More
With X-ray free-electron lasers (XFELs), it is possible to determine the three-dimensional structure of noncrystalline nanoscale particles using X-ray single-particle imaging (SPI) techniques at room temperature. Classifying SPI scattering patterns, or "speckles", to extract single hits that are needed for real-time vetoing and three-dimensional reconstruction poses a challenge for high data rate facilities like European XFEL and LCLS-II-HE. Here, we introduce SpeckleNN, a unified embedding model for real-time speckle pattern classification with limited labeled examples that can scale linearly with dataset size. Trained with twin neural networks, SpeckleNN maps speckle patterns to a unified embedding vector space, where similarity is measured by Euclidean distance. We highlight its few-shot classification capability on new never-seen samples and its robust performance despite only tens of labels per classification category even in the presence of substantial missing detector areas. Without the need for excessive manual labeling or even a full detector image, our classification method offers a great solution for real-time high-throughput SPI experiments.
△ Less
Submitted 14 February, 2023;
originally announced February 2023.
-
Testing the data framework for an AI algorithm in preparation for high data rate X-ray facilities
Authors:
Hongwei Chen,
Sathya R. Chitturi,
Rajan Plumley,
Lingjia Shen,
Nathan C. Drucker,
Nicolas Burdet,
Cheng Peng,
Sougata Mardanya,
Daniel Ratner,
Aashwin Mishra,
Chun Hong Yoon,
Sanghoon Song,
Matthieu Chollet,
Gilberto Fabbris,
Mike Dunne,
Silke Nelson,
Mingda Li,
Aaron Lindenberg,
Chun**g Jia,
Youssef Nashed,
Arun Bansil,
Sugata Chowdhury,
Adrian E. Feiguin,
Joshua J. Turner,
Jana B. Thayer
Abstract:
The advent of next-generation X-ray free electron lasers will be capable of delivering X-rays at a repetition rate approaching 1 MHz continuously. This will require the development of data systems to handle experiments at these type of facilities, especially for high throughput applications, such as femtosecond X-ray crystallography and X-ray photon fluctuation spectroscopy. Here, we demonstrate a…
▽ More
The advent of next-generation X-ray free electron lasers will be capable of delivering X-rays at a repetition rate approaching 1 MHz continuously. This will require the development of data systems to handle experiments at these type of facilities, especially for high throughput applications, such as femtosecond X-ray crystallography and X-ray photon fluctuation spectroscopy. Here, we demonstrate a framework which captures single shot X-ray data at the LCLS and implements a machine-learning algorithm to automatically extract the contrast parameter from the collected data. We measure the time required to return the results and assess the feasibility of using this framework at high data volume. We use this experiment to determine the feasibility of solutions for `live' data analysis at the MHz repetition rate.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
A machine learning photon detection algorithm for coherent X-ray ultrafast fluctuation analysis
Authors:
Sathya R. Chitturi,
Nicolas G. Burdet,
Youssef Nashed,
Daniel Ratner,
Aashwin Mishra,
TJ Lane,
Matthew Seaberg,
Vincent Esposito,
Chun H. Yoon,
Mike Dunne,
Joshua J. Turner
Abstract:
X-ray free electron laser (XFEL) experiments have brought unique capabilities and opened new directions in research, such as creating new states of matter or directly measuring atomic motion. One such area is the ability to use finely spaced sets of coherent x-ray pulses to be compared after scattering from a dynamic system at different times. This enables the study of fluctuations in many-body qu…
▽ More
X-ray free electron laser (XFEL) experiments have brought unique capabilities and opened new directions in research, such as creating new states of matter or directly measuring atomic motion. One such area is the ability to use finely spaced sets of coherent x-ray pulses to be compared after scattering from a dynamic system at different times. This enables the study of fluctuations in many-body quantum systems at the level of the ultrafast pulse durations, but this method has been limited to a select number of examples and required complex and advanced analytical tools. By applying a new methodology to this problem, we have made qualitative advances in three separate areas that will likely also find application to new fields. As compared to the `droplet-type' models which typically are used to estimate the photon distributions on pixelated detectors to obtain the coherent X-ray speckle patterns, our algorithm pipeline achieves an order of magnitude speedup on CPU hardware and two orders of magnitude improvement on GPU hardware. We also find that it retains accuracy in low-contrast conditions, which is the typical regime for many experiments in structural dynamics. Finally, it can predict photon distributions in high average-intensity applications, a regime which up until now, has not been accessible. Our AI-assisted algorithm will enable a wider adoption of x-ray coherence spectroscopies, by both automating previously challenging analyses and enabling new experiments that were not otherwise feasible without the developments described in this work.
△ Less
Submitted 17 June, 2022;
originally announced June 2022.
-
Scaling and Acceleration of Three-dimensional Structure Determination for Single-Particle Imaging Experiments with SpiniFEL
Authors:
Hsing-Yin Chang,
Elliott Slaughter,
Seema Mirchandaney,
Jeffrey Donatelli,
Chun Hong Yoon
Abstract:
The Linac Coherent Light Source (LCLS) is an X- ray free electron laser (XFEL) facility enabling the study of the structure and dynamics of single macromolecules. A major upgrade will bring the repetition rate of the X-ray source from 120 to 1 million pulses per second. Exascale high performance computing (HPC) capabilities will be required to process the corresponding data rates. We present Spini…
▽ More
The Linac Coherent Light Source (LCLS) is an X- ray free electron laser (XFEL) facility enabling the study of the structure and dynamics of single macromolecules. A major upgrade will bring the repetition rate of the X-ray source from 120 to 1 million pulses per second. Exascale high performance computing (HPC) capabilities will be required to process the corresponding data rates. We present SpiniFEL, an application used for structure determination of proteins from single-particle imaging (SPI) experiments. An emerging technique for imaging individual proteins and other large molecular complexes by outrunning radiation damage, SPI breaks free from the need for crystallization (which is difficult for some proteins) and allows for imaging molecular dynamics at near ambient conditions. SpiniFEL is being developed to run on supercomputers in near real-time while an experiment is taking place, so that the feedback about the data can guide the data collection strategy. We describe here how we reformulated the mathematical framework for parallelizable implementation and accelerated the most compute intensive parts of the application. We also describe the use of Pygion, a Python interface for the Legion task-based programming model and compare to our existing MPI+GPU implementation.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Reconstruction of 3D Image of Nanorice Particle from Randomly Oriented Single-Shot Experimental Diffraction Patterns Using Angular Correlation Method
Authors:
S. S. Kim,
P. Nepal,
D. K. Saldin,
C. H. Yoon
Abstract:
We reconstructed intensities in Fourier space and electron densities in the real space for an azimuthally symmetric object Nanorice particle (Iron Oxide nanoparticle) exposed in the ultrashort, bright and coherent x-ray free electron laser (XFEL) pulses with random unknown orientations through the method of angular correlations among intensities appeared in ninety eight 2D diffraction patterns col…
▽ More
We reconstructed intensities in Fourier space and electron densities in the real space for an azimuthally symmetric object Nanorice particle (Iron Oxide nanoparticle) exposed in the ultrashort, bright and coherent x-ray free electron laser (XFEL) pulses with random unknown orientations through the method of angular correlations among intensities appeared in ninety eight 2D diffraction patterns collected at Linac Coherent Light Source (LCLS).
△ Less
Submitted 12 July, 2020;
originally announced July 2020.
-
An advanced workflow for single particle imaging with the limited data at an X-ray free-electron laser
Authors:
Dameli Assalauova,
Young Yong Kim,
Sergey Bobkov,
Ruslan Khubbutdinov,
Max Rose,
Roberto Alvarez,
Jakob Andreasson,
Eugeniu Balaur,
Alice Contreras,
Hasan DeMirci,
Luca Gelisio,
Janos Hajdu,
Mark S. Hunter,
Ruslan P. Kurta,
Haoyuan Li,
Matthew McFadden,
Reza Nazari,
Peter Schwander,
Anton Teslyuk,
Peter Walter,
P. Lourdu Xavier,
Chun Hong Yoon,
Sahba Zaared,
Viacheslav A. Ilyin,
Richard A. Kirian
, et al. (3 additional authors not shown)
Abstract:
An improved analysis for single particle imaging (SPI) experiments, using the limited data, is presented here. Results are based on a study of bacteriophage PR772 performed at the AMO instrument at the Linac Coherent Light Source (LCLS) as part of the SPI initiative. Existing methods were modified to cope with the shortcomings of the experimental data: inaccessibility of information from the half…
▽ More
An improved analysis for single particle imaging (SPI) experiments, using the limited data, is presented here. Results are based on a study of bacteriophage PR772 performed at the AMO instrument at the Linac Coherent Light Source (LCLS) as part of the SPI initiative. Existing methods were modified to cope with the shortcomings of the experimental data: inaccessibility of information from the half of the detector and small fraction of single hits. General SPI analysis workflow was upgraded with the expectation-maximization based classification of diffraction patterns and mode decomposition on the final virus structure determination step. The presented processing pipeline allowed us to determine the three-dimensional structure of the bacteriophage PR772 without symmetry constraints with a spatial resolution of 6.9 nm. The obtained resolution was limited by the scattering intensity during the experiment and the relatively small number of single hits.
△ Less
Submitted 15 June, 2020;
originally announced June 2020.
-
X-ray diffractive imaging of controlled gas-phase molecules: Toward imaging of dynamics in the molecular frame
Authors:
Thomas Kierspel,
Andrew Morgan,
Joss Wiese,
Terry Mullins,
Andy Aquila,
Anton Barty,
Richard Bean,
Rebecca Boll,
Sébastien Boutet,
Philip Bucksbaum,
Henry N. Chapman,
Lauge Christensen,
Alan Fry,
Mark Hunter,
Jason E. Koglin,
Mengning Liang,
Valerio Mariani,
Adi Natan,
Joseph Robinson,
Daniel Rolles,
Artem Rudenko,
Kirsten Schnorr,
Henrik Stapelfeldt,
Stephan Stern,
Jan Thøgersen
, et al. (3 additional authors not shown)
Abstract:
We report experimental results on the diffractive imaging of three-dimensionally aligned 2,5-diiodothiophene molecules. The molecules were aligned by chirped near-infrared laser pulses, and their structure was probed at a photon energy of 9.5 keV ($λ\approx130 \text{pm}$) provided by the Linac Coherent Light Source. Diffracted photons were recorded on the CSPAD detector and a two-dimensional diffr…
▽ More
We report experimental results on the diffractive imaging of three-dimensionally aligned 2,5-diiodothiophene molecules. The molecules were aligned by chirped near-infrared laser pulses, and their structure was probed at a photon energy of 9.5 keV ($λ\approx130 \text{pm}$) provided by the Linac Coherent Light Source. Diffracted photons were recorded on the CSPAD detector and a two-dimensional diffraction pattern of the equilibrium structure of 2,5-diiodothiophene was recorded. The retrieved distance between the two iodine atoms agrees with the quantum-chemically calculated molecular structure to within 5 %. The experimental approach allows for the imaging of intrinsic molecular dynamics in the molecular frame, albeit this requires more experimental data which should be readily available at upcoming high-repetition-rate facilities.
△ Less
Submitted 7 February, 2020; v1 submitted 29 October, 2019;
originally announced October 2019.
-
Low-signal limit of X-ray single particle imaging
Authors:
Kartik Ayyer,
Andrew J. Morgan,
Andrew A. Aquila,
Hasan DeMirci,
Brenda G. Hogue,
Richard A. Kirian,
P. Lourdu Xavier,
Chun Hong Yoon,
Henry N. Chapman,
Anton Barty
Abstract:
An outstanding question in X-ray single particle imaging experiments has been the feasibility of imaging sub 10-nm-sized biomolecules under realistic experimental conditions where very few photons are expected to be measured in a single snapshot and instrument background may be significant relative to particle scattering. While analyses of simulated data have shown that the determination of an ave…
▽ More
An outstanding question in X-ray single particle imaging experiments has been the feasibility of imaging sub 10-nm-sized biomolecules under realistic experimental conditions where very few photons are expected to be measured in a single snapshot and instrument background may be significant relative to particle scattering. While analyses of simulated data have shown that the determination of an average image should be feasible using Bayesian methods such as the EMC algorithm, this has yet to be demonstrated using experimental data containing realistic non-isotropic instrument background, sample variability and other experimental factors. In this work, we show that the orientation and phase retrieval steps work at photon counts diluted to the signal levels one expects from smaller molecules or with weaker pulses, using data from experimental measurements of 60-nm PR772 viruses. Even when the signal is reduced to a fraction as little as 1/256, the virus electron density determined using ab initio phasing is of almost the same quality as the high-signal data. However, we are still limited by the total number of patterns collected, which may soon be mitigated by the advent of high repetition-rate sources like the European XFEL and LCLS-II.
△ Less
Submitted 27 April, 2020; v1 submitted 10 May, 2019;
originally announced May 2019.
-
SIMEX: Simulation of Experiments at Advanced Light Sources
Authors:
C Fortmann-Grote,
A A Andreev,
R Briggs,
M Bussmann,
A Buzmakov,
M Garten,
A Grund,
A Hübl,
S Hauff,
A Joy,
Z Jurek,
N D Loh,
T Rüter,
L Samoylova,
R Santra,
E A Schneidmiller,
A Sharma,
M Wing,
S Yakubov,
C H Yoon,
M V Yurkov,
B Ziaja,
A P Mancuso
Abstract:
Realistic simulations of experiments at large scale photon facilities, such as optical laser laboratories, synchrotrons, and free electron lasers, are of vital importance for the successful preparation, execution, and analysis of these experiments investigating ever more complex physical systems, e.g. biomolecules, complex materials, and ultra-short lived states of highly excited matter. Tradition…
▽ More
Realistic simulations of experiments at large scale photon facilities, such as optical laser laboratories, synchrotrons, and free electron lasers, are of vital importance for the successful preparation, execution, and analysis of these experiments investigating ever more complex physical systems, e.g. biomolecules, complex materials, and ultra-short lived states of highly excited matter. Traditional photon science modelling takes into account only isolated aspects of an experiment, such as the beam propagation, the photon-matter interaction, or the scattering process, making idealized assumptions about the remaining parts, e.g.\ the source spectrum, temporal structure and coherence properties of the photon beam, or the detector response. In SIMEX, we have implemented a platform for complete start-to-end simulations, following the radiation from the source, through the beam transport optics to the sample or target under investigation, its interaction with and scattering from the sample, and its registration in a photon detector, including a realistic model of the detector response to the radiation. Data analysis tools can be hooked up to the modelling pipeline easily. This allows researchers and facility operators to simulate their experiments and instruments in real life scenarios, identify promising and unattainable regions of the parameter space and ultimately make better use of valuable beamtime.
This work is licensed under the Creative Commons Attribution 3.0 Unported License: http://creativecommons.org/licenses/by/3.0/.
△ Less
Submitted 17 November, 2016; v1 submitted 19 October, 2016;
originally announced October 2016.
-
Strongly aligned gas-phase molecules at Free-Electron Lasers
Authors:
Thomas Kierspel,
Joss Wiese,
Terry Mullins,
Joseph Robinson,
Andy Aquila,
Anton Barty,
Richard Bean,
Rebecca Boll,
Sébastien Boutet,
Philip Bucksbaum,
Henry N. Chapman,
Lauge Christensen,
Alan Fry,
Mark Hunter,
Jason E. Koglin,
Mengning Liang,
Valerio Mariani,
Andrew Morgan,
Adi Natan,
Vladimir Petrovic,
Daniel Rolles,
Artem Rudenko,
Kirsten Schnorr,
Henrik Stapelfeldt,
Stephan Stern
, et al. (5 additional authors not shown)
Abstract:
We demonstrate a novel experimental implementation to strongly align molecules at full repetition rates of free-electron lasers. We utilized the available in-house laser system at the coherent x-ray imaging beamline at the Linac Coherent Light Source. Chirped laser pulses, i. e., the direct output from the regenerative amplifier of the Ti:Sa chirped pulse amplification laser system, were used to s…
▽ More
We demonstrate a novel experimental implementation to strongly align molecules at full repetition rates of free-electron lasers. We utilized the available in-house laser system at the coherent x-ray imaging beamline at the Linac Coherent Light Source. Chirped laser pulses, i. e., the direct output from the regenerative amplifier of the Ti:Sa chirped pulse amplification laser system, were used to strongly align 2,5-diiodothiophene molecules in a molecular beam. The alignment laser pulses had pulse energies of a few mJ and a pulse duration of 94 ps. A degree of alignment of $\left<\cos^2\!θ_{2D}\right>$ = 0.85 was measured, limited by the intrinsic temperature of the molecular beam rather than by the available laser system. With the general availability of synchronized chirped-pulse-amplified near-infrared laser systems at short-wavelength laser facilities, our approach allows for the universal preparation of molecules tightly fixed in space for experiments with x-ray pulses.
△ Less
Submitted 17 October, 2015; v1 submitted 11 June, 2015;
originally announced June 2015.
-
The symmetries of image formation by scattering. II. Applications
Authors:
Peter Schwander,
Chun Hong Yoon,
Abbas Ourmazd,
Dimitrios Giannakis
Abstract:
We show that the symmetries of image formation by scattering enable graph-theoretic manifold-embedding techniques to extract structural and timing information from simulated and experimental snapshots at extremely low signal. The approach constitutes a physically-based, computationally efficient, and noise-robust route to analyzing the large and varied datasets generated by existing and emerging m…
▽ More
We show that the symmetries of image formation by scattering enable graph-theoretic manifold-embedding techniques to extract structural and timing information from simulated and experimental snapshots at extremely low signal. The approach constitutes a physically-based, computationally efficient, and noise-robust route to analyzing the large and varied datasets generated by existing and emerging methods for studying structure and dynamics by scattering. We demonstrate three-dimensional structure recovery from X-ray diffraction and cryo-electron microscope image snapshots of unknown orientation, the latter at 12 times lower dose than currently in use. We also show that ultra-low-signal, random sightings of dynamically evolving systems can be sequenced into high quality movies to reveal their evolution. Our approach offers a route to recovering timing information in time-resolved experiments, and extracting 3D movies from two-dimensional random sightings of dynamic systems.
△ Less
Submitted 24 September, 2011;
originally announced September 2011.