-
Roadmap on Deep Learning for Microscopy
Authors:
Giovanni Volpe,
Carolina Wählby,
Lei Tian,
Michael Hecht,
Artur Yakimovich,
Kristina Monakhova,
Laura Waller,
Ivo F. Sbalzarini,
Christopher A. Metzler,
Mingyang Xie,
Kevin Zhang,
Isaac C. D. Lenton,
Halina Rubinsztein-Dunlop,
Daniel Brunner,
Bijie Bai,
Aydogan Ozcan,
Daniel Midtvedt,
Hao Wang,
Nataša Sladoje,
Joakim Lindblad,
Jason T. Smith,
Marien Ochoa,
Margarida Barroso,
Xavier Intes,
Tong Qiu
, et al. (50 additional authors not shown)
Abstract:
Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the…
▽ More
Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
Playing with Active Matter
Authors:
Angelo Barona Balda,
Aykut Argun,
Agnese Callegari,
Giovanni Volpe
Abstract:
In the last 20 years, active matter has been a very successful research field, bridging the fundamental physics of nonequilibrium thermodynamics with applications in robotics, biology, and medicine. This field deals with active particles, which, differently from passive Brownian particles, can harness energy to generate complex motions and emerging behaviors. Most active-matter experiments are per…
▽ More
In the last 20 years, active matter has been a very successful research field, bridging the fundamental physics of nonequilibrium thermodynamics with applications in robotics, biology, and medicine. This field deals with active particles, which, differently from passive Brownian particles, can harness energy to generate complex motions and emerging behaviors. Most active-matter experiments are performed with microscopic particles and require advanced microfabrication and microscopy techniques. Here, we propose some macroscopic experiments with active matter employing commercially available toy robots, i.e., the Hexbugs. We demonstrate how they can be easily modified to perform regular and chiral active Brownian motion. We also show that Hexbugs can interact with passive objects present in their environment and, depending on their shape, set them in motion and rotation. Furthermore, we show that, by introducing obstacles in the environment, we can sort the robots based on their motility and chirality. Finally, we demonstrate the emergence of Casimir-like activity-induced attraction between planar objects in the presence of active particles in the environment.
△ Less
Submitted 9 September, 2022;
originally announced September 2022.
-
Roadmap for Optical Tweezers
Authors:
Giovanni Volpe,
Onofrio M. Maragò,
Halina Rubinzstein-Dunlop,
Giuseppe Pesce,
Alexander B. Stilgoe,
Giorgio Volpe,
Georgiy Tkachenko,
Viet Giang Truong,
Síle Nic Chormaic,
Fatemeh Kalantarifard,
Parviz Elahi,
Mikael Käll,
Agnese Callegari,
Manuel I. Marqués,
Antonio A. R. Neves,
Wendel L. Moreira,
Adriana Fontes,
Carlos L. Cesar,
Rosalba Saija,
Abir Saidi,
Paul Beck,
Jörg S. Eismann,
Peter Banzer,
Thales F. D. Fernandes,
Francesco Pedaci
, et al. (58 additional authors not shown)
Abstract:
Optical tweezers are tools made of light that enable contactless pushing, trap**, and manipulation of objects ranging from atoms to space light sails. Since the pioneering work by Arthur Ashkin in the 1970s, optical tweezers have evolved into sophisticated instruments and have been employed in a broad range of applications in life sciences, physics, and engineering. These include accurate force…
▽ More
Optical tweezers are tools made of light that enable contactless pushing, trap**, and manipulation of objects ranging from atoms to space light sails. Since the pioneering work by Arthur Ashkin in the 1970s, optical tweezers have evolved into sophisticated instruments and have been employed in a broad range of applications in life sciences, physics, and engineering. These include accurate force and torque measurement at the femtonewton level, microrheology of complex fluids, single micro- and nanoparticle spectroscopy, single-cell analysis, and statistical-physics experiments. This roadmap provides insights into current investigations involving optical forces and optical tweezers from their theoretical foundations to designs and setups. It also offers perspectives for applications to a wide range of research fields, from biophysics to space exploration.
△ Less
Submitted 28 June, 2022;
originally announced June 2022.
-
Objective comparison of methods to decode anomalous diffusion
Authors:
Gorka Muñoz-Gil,
Giovanni Volpe,
Miguel Angel Garcia-March,
Erez Aghion,
Aykut Argun,
Chang Beom Hong,
Tom Bland,
Stefano Bo,
J. Alberto Conejero,
Nicolás Firbas,
Òscar Garibo i Orts,
Alessia Gentili,
Zihan Huang,
Jae-Hyung Jeon,
Hélène Kabbech,
Yeong** Kim,
Patrycja Kowalek,
Diego Krapf,
Hanna Loch-Olszewska,
Michael A. Lomholt,
Jean-Baptiste Masson,
Philipp G. Meyer,
Seongyu Park,
Borja Requena,
Ihor Smal
, et al. (9 additional authors not shown)
Abstract:
Deviations from Brownian motion leading to anomalous diffusion are ubiquitously found in transport dynamics, playing a crucial role in phenomena from quantum physics to life sciences. The detection and characterization of anomalous diffusion from the measurement of an individual trajectory are challenging tasks, which traditionally rely on calculating the mean squared displacement of the trajector…
▽ More
Deviations from Brownian motion leading to anomalous diffusion are ubiquitously found in transport dynamics, playing a crucial role in phenomena from quantum physics to life sciences. The detection and characterization of anomalous diffusion from the measurement of an individual trajectory are challenging tasks, which traditionally rely on calculating the mean squared displacement of the trajectory. However, this approach breaks down for cases of important practical interest, e.g., short or noisy trajectories, ensembles of heterogeneous trajectories, or non-ergodic processes. Recently, several new approaches have been proposed, mostly building on the ongoing machine-learning revolution. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition, the Anomalous Diffusion challenge (AnDi). Participating teams independently applied their own algorithms to a commonly-defined dataset including diverse conditions. Although no single method performed best across all scenarios, the results revealed clear differences between the various approaches, providing practical advice for users and a benchmark for developers.
△ Less
Submitted 14 May, 2021;
originally announced May 2021.
-
Classification, inference and segmentation of anomalous diffusion with recurrent neural networks
Authors:
Aykut Argun,
Giovanni Volpe,
Stefano Bo
Abstract:
Countless systems in biology, physics, and finance undergo diffusive dynamics. Many of these systems, including biomolecules inside cells, active matter systems and foraging animals, exhibit anomalous dynamics where the growth of the mean squared displacement with time follows a power law with an exponent that deviates from $1$. When studying time series recording the evolution of these systems, i…
▽ More
Countless systems in biology, physics, and finance undergo diffusive dynamics. Many of these systems, including biomolecules inside cells, active matter systems and foraging animals, exhibit anomalous dynamics where the growth of the mean squared displacement with time follows a power law with an exponent that deviates from $1$. When studying time series recording the evolution of these systems, it is crucial to precisely measure the anomalous exponent and confidently identify the mechanisms responsible for anomalous diffusion. These tasks can be overwhelmingly difficult when only few short trajectories are available, a situation that is common in the study of non-equilibrium and living systems. Here, we present a data-driven method to analyze single anomalous diffusion trajectories employing recurrent neural networks, which we name RANDI. We show that our method can successfully infer the anomalous exponent, identify the type of anomalous diffusion process, and segment the trajectories of systems switching between different behaviors. We benchmark our performance against the state-of-the art techniques for the study of single short trajectories that participated in the Anomalous Diffusion (AnDi) Challenge. Our method proved to be the most versatile method, being the only one to consistently rank in the top 3 for all tasks proposed in the AnDi Challenge.
△ Less
Submitted 1 April, 2021;
originally announced April 2021.
-
Quantitative Digital Microscopy with Deep Learning
Authors:
Benjamin Midtvedt,
Saga Helgadottir,
Aykut Argun,
Jesús Pineda,
Daniel Midtvedt,
Giovanni Volpe
Abstract:
Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches usi…
▽ More
Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in develo** custom deep-learning solutions. To overcome this issue, we introduce a software, DeepTrack 2.0, to design, train and validate deep-learning solutions for digital microscopy. We use it to exemplify how deep learning can be employed for a broad range of applications, from particle localization, tracking and characterization to cell counting and classification. Thanks to its user-friendly graphical interface, DeepTrack 2.0 can be easily customized for user-specific applications, and, thanks to its open-source object-oriented programming, it can be easily expanded to add features and functionalities, potentially introducing deep-learning-enhanced video microscopy to a far wider audience.
△ Less
Submitted 16 October, 2020;
originally announced October 2020.
-
Enhanced force-field calibration via machine learning
Authors:
Aykut Argun,
Tobias Thalheim,
Stefano Bo,
Frank Cichos,
Giovanni Volpe
Abstract:
The influence of microscopic force fields on the motion of Brownian particles plays a fundamental role in a broad range of fields, including soft matter, biophysics, and active matter. Often, the experimental calibration of these force fields relies on the analysis of the trajectories of these Brownian particles. However, such an analysis is not always straightforward, especially if the underlying…
▽ More
The influence of microscopic force fields on the motion of Brownian particles plays a fundamental role in a broad range of fields, including soft matter, biophysics, and active matter. Often, the experimental calibration of these force fields relies on the analysis of the trajectories of these Brownian particles. However, such an analysis is not always straightforward, especially if the underlying force fields are non-conservative or time-varying, driving the system out of thermodynamic equilibrium. Here, we introduce a toolbox to calibrate microscopic force fields by analyzing the trajectories of a Brownian particle using machine learning, namely recurrent neural networks. We demonstrate that this machine-learning approach outperforms standard methods when characterizing the force fields generated by harmonic potentials if the available data are limited. More importantly, it provides a tool to calibrate force fields in situations for which there are no standard methods, such as non-conservative and time-varying force fields. In order to make this method readily available for other users, we provide a Python software package named DeepCalib, which can be easily personalized and optimized for specific applications.
△ Less
Submitted 16 June, 2020;
originally announced June 2020.
-
Digital video microscopy enhanced by deep learning
Authors:
Saga Helgadottir,
Aykut Argun,
Giovanni Volpe
Abstract:
Single particle tracking is essential in many branches of science and technology, from the measurement of biomolecular forces to the study of colloidal crystals. Standard current methods rely on algorithmic approaches: by fine-tuning several user-defined parameters, these methods can be highly successful at tracking a well-defined kind of particle under low-noise conditions with constant and homog…
▽ More
Single particle tracking is essential in many branches of science and technology, from the measurement of biomolecular forces to the study of colloidal crystals. Standard current methods rely on algorithmic approaches: by fine-tuning several user-defined parameters, these methods can be highly successful at tracking a well-defined kind of particle under low-noise conditions with constant and homogenous illumination. Here, we introduce an alternative data-driven approach based on a convolutional neural network, which we name DeepTrack. We show that DeepTrack outperforms algorithmic approaches, especially in the presence of noise and under poor illumination conditions. We use DeepTrack to track an optically trapped particle under very noisy and unsteady illumination conditions, where standard algorithmic approaches fail. We then demonstrate how DeepTrack can also be used to track multiple particles and non-spherical objects such as bacteria, also at very low signal-to-noise ratios. In order to make DeepTrack readily available for other users, we provide a Python software package, which can be easily personalized and optimized for specific applications.
△ Less
Submitted 6 December, 2018;
originally announced December 2018.