-
SciJava Ops: An Improved Algorithms Framework for Fiji and Beyond
Authors:
Gabriel J. Selzer,
Curtis T. Rueden,
Mark C. Hiner,
Edward L. Evans III,
David Kolb,
Marcel Wiedenmann,
Christian Birkhold,
Tim-Oliver Buchholz,
Stefan Helfrich,
Brian Northan,
Alison Walter,
Johannes Schindelin,
Tobias Pietzsch,
Stephan Saalfeld,
Michael R. Berthold,
Kevin W. Eliceiri
Abstract:
Many scientific software platforms provide plugin mechanisms that simplify the integration, deployment, and execution of externally developed functionality. One of the most widely used platforms in the imaging space is Fiji, a popular open-source application for scientific image analysis. Fiji incorporates and builds on the ImageJ and ImageJ2 platforms, which provide a powerful plugin architecture…
▽ More
Many scientific software platforms provide plugin mechanisms that simplify the integration, deployment, and execution of externally developed functionality. One of the most widely used platforms in the imaging space is Fiji, a popular open-source application for scientific image analysis. Fiji incorporates and builds on the ImageJ and ImageJ2 platforms, which provide a powerful plugin architecture used by thousands of plugins to solve a wide variety of problems. This capability is a major part of Fiji's success, and it has become a widely used biological image analysis tool and a target for new functionality. However, a plugin-based software architecture cannot unify disparate platforms operating on incompatible data structures; interoperability necessitates the creation of adaptation or "bridge" layers to translate data and invoke functionality. As a result, while platforms like Fiji enable a high degree of interconnectivity and extensibility, they were not fundamentally designed to integrate across the many data types, programming languages, and architectural differences of various software platforms.To help address this challenge, we present SciJava Ops, a foundational software library for expressing algorithms as plugins in a unified and extensible way. Continuing the evolution of Fiji's SciJava plugin mechanism, SciJava Ops enables users to harness algorithms from various software platforms within a central execution environment. In addition, SciJava Ops automatically adapts data into the most appropriate structure for each algorithm, allowing users to freely and transparently combine algorithms from otherwise incompatible tools. While SciJava Ops is initially distributed as a Fiji update site, the framework does not require Fiji, ImageJ, or ImageJ2, and would be suitable for integration with additional image analysis platforms.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
Harmonizing the Generation and Pre-publication Stewardship of FAIR Image Data
Authors:
Nikki Bialy,
Frank Alber,
Brenda Andrews,
Michael Angelo,
Brian Beliveau,
Lacramioara Bintu,
Alistair Boettiger,
Ulrike Boehm,
Claire M. Brown,
Mahmoud Bukar Maina,
James J. Chambers,
Beth A. Cimini,
Kevin Eliceiri,
Rachel Errington,
Orestis Faklaris,
Nathalie Gaudreault,
Ronald N. Germain,
Wojtek Goscinski,
David Grunwald,
Michael Halter,
Dorit Hanein,
John W. Hickey,
Judith Lacoste,
Alex Laude,
Emma Lundberg
, et al. (22 additional authors not shown)
Abstract:
Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thu…
▽ More
Together with the molecular knowledge of genes and proteins, biological images promise to significantly enhance the scientific understanding of complex cellular systems and to advance predictive and personalized therapeutic products for human health. For this potential to be realized, quality-assured image data must be shared among labs at a global scale to be compared, pooled, and reanalyzed, thus unleashing untold potential beyond the original purpose for which the data was generated. There are two broad sets of requirements to enable image data sharing in the life sciences. One set of requirements is articulated in the companion White Paper entitled Enabling Global Image Data Sharing in the Life Sciences, which is published in parallel and addresses the need to build the cyberinfrastructure for sharing the digital array data. In this White Paper, we detail a broad set of requirements, which involves collecting, managing, presenting, and propagating contextual information essential to assess the quality, understand the content, interpret the scientific implications, and reuse image data in the context of the experimental details. We start by providing an overview of the main lessons learned to date through international community activities, which have recently made considerable progress toward generating community standard practices for imaging Quality Control (QC) and metadata. We then provide a clear set of recommendations for amplifying this work. The driving goal is to address remaining challenges and democratize access to everyday practices and tools for a spectrum of biomedical researchers, regardless of their expertise, access to resources, and geographical location.
△ Less
Submitted 8 February, 2024; v1 submitted 23 January, 2024;
originally announced January 2024.
-
CellProfiler plugins -- an easy image analysis platform integration for containers and Python tools
Authors:
Erin Weisbart,
Callum Tromans-Coia,
Barbara Diaz-Rohrer,
David R Stirling,
Fernanda Garcia-Fossa,
Rebecca A Senft,
Mark C Hiner,
Marcelo B de Jesus,
Kevin W Eliceiri,
Beth A Cimini
Abstract:
CellProfiler is a widely used software for creating reproducible, reusable image analysis workflows without needing to code. In addition to the >90 modules that make up the main CellProfiler program, CellProfiler has a plugins system that allows for the creation of new modules which integrate with other Python tools or tools that are packaged in software containers. The CellProfiler-plugins reposi…
▽ More
CellProfiler is a widely used software for creating reproducible, reusable image analysis workflows without needing to code. In addition to the >90 modules that make up the main CellProfiler program, CellProfiler has a plugins system that allows for the creation of new modules which integrate with other Python tools or tools that are packaged in software containers. The CellProfiler-plugins repository contains a number of these CellProfiler modules, especially modules that are experimental and/or dependency-heavy. Here, we present an upgraded CellProfiler-plugins repository, an example of accessing containerized tools, improved documentation, and added citation/reference tools to facilitate the use and contribution of the community.
△ Less
Submitted 15 August, 2023; v1 submitted 2 June, 2023;
originally announced June 2023.
-
Community-developed checklists for publishing images and image analysis
Authors:
Christopher Schmied,
Michael Nelson,
Sergiy Avilov,
Gert-Jan Bakker,
Cristina Bertocchi,
Johanna Bischof,
Ulrike Boehm,
Jan Brocher,
Mariana Carvalho,
Catalin Chiritescu,
Jana Christopher,
Beth Cimini,
Eduardo Conde-Sousa,
Michael Ebner,
Rupert Ecker,
Kevin Eliceiri,
Julia Fernandez-Rodriguez,
Nathalie Gaudreault,
Laurent Gelman,
David Grunwald,
Tingting Gu,
Nadia Halidi,
Mathias Hammer,
Matthew Hartley,
Marie Held
, et al. (29 additional authors not shown)
Abstract:
Images document scientific discoveries and are prevalent in modern biomedical research. Microscopy imaging in particular is currently undergoing rapid technological advancements. However for scientists wishing to publish the obtained images and image analyses results, there are to date no unified guidelines. Consequently, microscopy images and image data in publications may be unclear or difficult…
▽ More
Images document scientific discoveries and are prevalent in modern biomedical research. Microscopy imaging in particular is currently undergoing rapid technological advancements. However for scientists wishing to publish the obtained images and image analyses results, there are to date no unified guidelines. Consequently, microscopy images and image data in publications may be unclear or difficult to interpret. Here we present community-developed checklists for preparing light microscopy images and image analysis for publications. These checklists offer authors, readers, and publishers key recommendations for image formatting and annotation, color selection, data availability, and for reporting image analysis workflows. The goal of our guidelines is to increase the clarity and reproducibility of image figures and thereby heighten the quality of microscopy data is in publications.
△ Less
Submitted 14 September, 2023; v1 submitted 14 February, 2023;
originally announced February 2023.
-
Metasurface-enhanced mid-infrared spectrochemical imaging of tissues
Authors:
S. Rosas,
K. A. Schoeller,
E. Chang,
H. Mei,
M. A. Kats,
K. W. Eliceiri,
X. Zhao,
F. Yesilkoy
Abstract:
Label-free and nondestructive mid-infrared vibrational hyperspectral imaging is emerging as an important ex-vivo tissue analysis tool, providing spatially resolved biochemical information critical to understanding physiological and pathological processes. However, the chemically complex and spatially heterogeneous composition of tissue specimens and the inherently weak interaction of infrared ligh…
▽ More
Label-free and nondestructive mid-infrared vibrational hyperspectral imaging is emerging as an important ex-vivo tissue analysis tool, providing spatially resolved biochemical information critical to understanding physiological and pathological processes. However, the chemically complex and spatially heterogeneous composition of tissue specimens and the inherently weak interaction of infrared light with biomolecules limit the analytical performance of infrared absorption spectroscopy. Here, we introduce an advanced mid-infrared spectrochemical tissue imaging modality using metasurfaces that support strong surface-localized electromagnetic fields to capture quantitative molecular maps of large-area murine brain-tissue sections. Our approach leverages polarization-multiplexed multi-resonance plasmonic metasurfaces to simultaneously detect many different functional biomolecules. The resulting surface-enhanced mid-infrared spectral imaging (SE-MIRSI) method eliminates the non-specific effects of bulk tissue morphology on the quantitative analysis of fingerprint spectra and improves the chemical selectivity. We show that the metasurface enhancement increases the retrieval of amide I and II absorption bands associated with secondary structures of proteins. Moreover, we demonstrate that plasmonic metasurfaces enhance the chemical contrast in infrared images and enable the detection of ultrathin tissue regions that are not otherwise visible to conventional mid-infrared spectral imaging. While we tested our approach on murine brain tissue sections, this chemical imaging method is well-suited for any tissue type, which significantly broadens the potential impacts of our method for both translational research and clinical histopathology.
△ Less
Submitted 26 April, 2023; v1 submitted 14 January, 2023;
originally announced January 2023.
-
Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning
Authors:
Bin Li,
Yin Li,
Kevin W. Eliceiri
Abstract:
We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method…
▽ More
We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems. GitHub repository: https://github.com/binli123/dsmil-wsi
△ Less
Submitted 2 April, 2021; v1 submitted 17 November, 2020;
originally announced November 2020.
-
Dual-stream Maximum Self-attention Multi-instance Learning
Authors:
Bin Li,
Kevin W. Eliceiri
Abstract:
Multi-instance learning (MIL) is a form of weakly supervised learning where a single class label is assigned to a bag of instances while the instance-level labels are not available. Training classifiers to accurately determine the bag label and instance labels is a challenging but critical task in many practical scenarios, such as computational histopathology. Recently, MIL models fully parameteri…
▽ More
Multi-instance learning (MIL) is a form of weakly supervised learning where a single class label is assigned to a bag of instances while the instance-level labels are not available. Training classifiers to accurately determine the bag label and instance labels is a challenging but critical task in many practical scenarios, such as computational histopathology. Recently, MIL models fully parameterized by neural networks have become popular due to the high flexibility and superior performance. Most of these models rely on attention mechanisms that assign attention scores across the instance embeddings in a bag and produce the bag embedding using an aggregation operator. In this paper, we proposed a dual-stream maximum self-attention MIL model (DSMIL) parameterized by neural networks. The first stream deploys a simple MIL max-pooling while the top-activated instance embedding is determined and used to obtain self-attention scores across instance embeddings in the second stream. Different from most of the previous methods, the proposed model jointly learns an instance classifier and a bag classifier based on the same instance embeddings. The experiments results show that our method achieves superior performance compared to the best MIL methods and demonstrates state-of-the-art performance on benchmark MIL datasets.
△ Less
Submitted 9 June, 2020;
originally announced June 2020.
-
Quantitative Histopathology of Stained Tissues using Color Spatial Light Interference Microscopy (cSLIM)
Authors:
Hassaan Majeed,
Adib Keikhosravi,
Mikhail E. Kandel,
Tan H. Nguyen,
Yuming Liu,
Andre Kajdacsy-Balla,
Krishnarao Tangella,
Kevin W. Eliceiri,
Gabriel Popescu
Abstract:
Tissue biopsy evaluation in the clinic is in need of quantitative disease markers for diagnosis and, most importantly, prognosis. Among the new technologies, quantitative phase imaging (QPI) has demonstrated promise for histopathology because it reveals intrinsic tissue nanoarchitecture through the refractive index. However, a vast majority of past QPI investigations have relied on imaging unstain…
▽ More
Tissue biopsy evaluation in the clinic is in need of quantitative disease markers for diagnosis and, most importantly, prognosis. Among the new technologies, quantitative phase imaging (QPI) has demonstrated promise for histopathology because it reveals intrinsic tissue nanoarchitecture through the refractive index. However, a vast majority of past QPI investigations have relied on imaging unstained tissues, which disrupts the established specimen processing. Here we present color spatial light interference microscopy (cSLIM) as a new whole slide imaging modality that performs interferometric imaging with a color detector array. As a result, cSLIM yields in a single scan both the intrinsic tissue phase map and the standard color bright-field image, familiar to the pathologist. Our results on 196 breast cancer patients indicate that cSLIM can provide not only diagnostic but also prognostic information from the alignment of collagen fibers in the tumor microenvironment. The effects of staining on the tissue phase maps were corrected by a simple mathematical normalization. These characteristics are likely to reduce barriers to clinical translation for the new cSLIM technology.
△ Less
Submitted 10 June, 2018;
originally announced June 2018.
-
Spatially Adaptive Colocalization Analysis in Dual-Color Fluorescence Microscopy
Authors:
Shulei Wang,
Ellen T. Arena,
Jordan T. Becker,
William M. Bement,
Nathan M. Sherer,
Kevin W. Eliceiri,
Ming Yuan
Abstract:
Colocalization analysis aims to study complex spatial associations between bio-molecules via optical imaging techniques. However, existing colocalization analysis workflows only assess an average degree of colocalization within a certain region of interest and ignore the unique and valuable spatial information offered by microscopy. In the current work, we introduce a new framework for colocalizat…
▽ More
Colocalization analysis aims to study complex spatial associations between bio-molecules via optical imaging techniques. However, existing colocalization analysis workflows only assess an average degree of colocalization within a certain region of interest and ignore the unique and valuable spatial information offered by microscopy. In the current work, we introduce a new framework for colocalization analysis that allows us to quantify colocalization levels at each individual location and automatically identify pixels or regions where colocalization occurs. The framework, referred to as spatially adaptive colocalization analysis (SACA), integrates a pixel-wise local kernel model for colocalization quantification and a multi-scale adaptive propagation-separation strategy for utilizing spatial information to detect colocalization in a spatially adaptive fashion. Applications to simulated and real biological datasets demonstrate the practical merits of SACA in what we hope to be an easily applicable and robust colocalization analysis method. In addition, theoretical properties of SACA are investigated to provide rigorous statistical justification.
△ Less
Submitted 20 March, 2019; v1 submitted 31 October, 2017;
originally announced November 2017.
-
Automated and Robust Quantification of Colocalization in Dual-Color Fluorescence Microscopy: A Nonparametric Statistical Approach
Authors:
Shulei Wang,
Ellen T. Arena,
Kevin W. Eliceiri,
Ming Yuan
Abstract:
Colocalization is a powerful tool to study the interactions between fluorescently labeled molecules in biological fluorescence microscopy. However, existing techniques for colocalization analysis have not undergone continued development especially in regards to robust statistical support. In this paper, we examine two of the most popular quantification techniques for colocalization and argue that…
▽ More
Colocalization is a powerful tool to study the interactions between fluorescently labeled molecules in biological fluorescence microscopy. However, existing techniques for colocalization analysis have not undergone continued development especially in regards to robust statistical support. In this paper, we examine two of the most popular quantification techniques for colocalization and argue that they could be improved upon using ideas from nonparametric statistics and scan statistics. In particular, we propose a new colocalization metric that is robust, easily implementable, and optimal in a rigorous statistical testing framework. Application to several benchmark datasets, as well as biological examples, further demonstrates the usefulness of the proposed technique.
△ Less
Submitted 2 October, 2017;
originally announced October 2017.
-
ImageJ2: ImageJ for the next generation of scientific image data
Authors:
Curtis T. Rueden,
Johannes Schindelin,
Mark C. Hiner,
Barry E. DeZonia,
Alison E. Walter,
Ellen T. Arena,
Kevin W. Eliceiri
Abstract:
ImageJ is an image analysis program extensively used in the biological sciences and beyond. Due to its ease of use, recordable macro language, and extensible plug-in architecture, ImageJ enjoys contributions from non-programmers, amateur programmers, and professional developers alike. Enabling such a diversity of contributors has resulted in a large community that spans the biological and physical…
▽ More
ImageJ is an image analysis program extensively used in the biological sciences and beyond. Due to its ease of use, recordable macro language, and extensible plug-in architecture, ImageJ enjoys contributions from non-programmers, amateur programmers, and professional developers alike. Enabling such a diversity of contributors has resulted in a large community that spans the biological and physical sciences. However, a rapidly growing user base, diverging plugin suites, and technical limitations have revealed a clear need for a concerted software engineering effort to support emerging imaging paradigms, to ensure the software's ability to handle the requirements of modern science. Due to these new and emerging challenges in scientific imaging, ImageJ is at a critical development crossroads.
We present ImageJ2, a total redesign of ImageJ offering a host of new functionality. It separates concerns, fully decoupling the data model from the user interface. It emphasizes integration with external applications to maximize interoperability. Its robust new plugin framework allows everything from image formats, to scripting languages, to visualization to be extended by the community. The redesigned data model supports arbitrarily large, N-dimensional datasets, which are increasingly common in modern image acquisition. Despite the scope of these changes, backwards compatibility is maintained such that this new functionality can be seamlessly integrated with the classic ImageJ interface, allowing users and developers to migrate to these new methods at their own pace. ImageJ2 provides a framework engineered for flexibility, intended to support these requirements as well as accommodate future needs.
△ Less
Submitted 3 November, 2017; v1 submitted 20 January, 2017;
originally announced January 2017.
-
OpenSPIM - an open access platform for light sheet microscopy
Authors:
Peter Pitrone,
Johannes Schindelin,
Luke Stuyvenberg,
Stephan Preibisch,
Michael Weber,
Kevin W. Eliceiri,
Jan Huisken,
Pavel Tomancak
Abstract:
Light sheet microscopy promises to revolutionize developmental biology by enabling live in toto imaging of entire embryos with minimal phototoxicity. We present detailed instructions for building a compact and customizable Selective Plane Illumination Microscopy (SPIM) system. The integrated OpenSPIM hardware and software platform is shared with the scientific community through a public website, t…
▽ More
Light sheet microscopy promises to revolutionize developmental biology by enabling live in toto imaging of entire embryos with minimal phototoxicity. We present detailed instructions for building a compact and customizable Selective Plane Illumination Microscopy (SPIM) system. The integrated OpenSPIM hardware and software platform is shared with the scientific community through a public website, thereby making light sheet microscopy accessible for widespread use and optimization to various applications.
△ Less
Submitted 8 February, 2013;
originally announced February 2013.
