Showing 1–50 of 81 results for author: Martel, A

VertDetect: Fully End-to-End 3D Vertebral Instance Segmentation Model

Authors: Geoff Klein, Michael Hardisty, Cari Whyne, Anne L. Martel

Abstract: Vertebral detection and segmentation are critical steps for treatment planning in spine surgery and radiation therapy. Accurate identification and segmentation are complicated in imaging that does not include the full spine, in cases with variations in anatomy (T13 and/or L6 vertebrae), and in the presence of fracture or hardware. This paper proposes VertDetect, a fully automated end-to-end 3D ver… ▽ More Vertebral detection and segmentation are critical steps for treatment planning in spine surgery and radiation therapy. Accurate identification and segmentation are complicated in imaging that does not include the full spine, in cases with variations in anatomy (T13 and/or L6 vertebrae), and in the presence of fracture or hardware. This paper proposes VertDetect, a fully automated end-to-end 3D vertebral instance segmentation Convolutional Neural Network (CNN) model to predict vertebral level labels and segmentations for all vertebrae present in a CT scan. The utilization of a shared CNN backbone provides the detection and segmentation branches of the network with feature maps containing both spinal and vertebral level information. A Graph Convolutional Network (GCN) layer is used to improve vertebral labelling by using the known structure of the spine. This model achieved a Dice Similarity Coefficient (DSC) of 0.883 (95% CI, 0.843-0.906) and 0.882 (95% CI, 0.835-0.909) in the VerSe 2019 and 0.868 (95\% CI, 0.834-0.890) and 0.869 (95\% CI, 0.832-0.891) in the VerSe 2020 public and hidden test sets, respectively. This model achieved state-of-the-art performance for an end-to-end architecture, whose design facilitates the extraction of features that can be subsequently used for downstream tasks. △ Less

Submitted 16 November, 2023; originally announced November 2023.

Comments: Preprint

Cross-Validation Is All You Need: A Statistical Approach To Label Noise Estimation

Authors: Jianan Chen, Anne Martel

Abstract: Label noise is prevalent in machine learning datasets. It is crucial to identify and remove label noise because models trained on noisy data can have substantially reduced accuracy and generalizability. Most existing label noise detection approaches are designed for classification tasks, and data cleaning for outcome prediction analysis is relatively unexplored. Inspired by the fluctuations in per… ▽ More Label noise is prevalent in machine learning datasets. It is crucial to identify and remove label noise because models trained on noisy data can have substantially reduced accuracy and generalizability. Most existing label noise detection approaches are designed for classification tasks, and data cleaning for outcome prediction analysis is relatively unexplored. Inspired by the fluctuations in performance across different folds in cross-validation, we propose Repeated Cross-Validations for label noise estimation (ReCoV) to address this gap. ReCoV constructs a noise histogram that ranks the noise level of samples based on a large number of cross-validations by recording sample IDs in each worst-performing fold. We further propose three approaches for identifying noisy samples based on noise histograms to address increasingly complex noise distributions. We show that ReCoV outperforms state-of-the-art algorithms for label cleaning in a classification task benchmark. More importantly, we show that removing ReCoV-identified noisy samples in two medical imaging outcome prediction datasets significantly improves model performance on test sets. As a statistical approach that does not rely on hyperparameters, noise distributions, or model structures, ReCoV is compatible with any machine learning analysis. △ Less

Submitted 24 June, 2023; originally announced June 2023.

The Near Infrared Imager and Slitless Spectrograph for the James Webb Space Telescope -- III. Single Object Slitless Spectroscopy

Authors: Loic Albert, David Lafreniere, Rene Doyon, Etienne Artigau, Kevin Volk, Paul Goudfrooij, Andre R. Martel, Michael Radica, Jason Rowe, Nestor Espinoza, Arpita Roy, Joseph C. Filippazzo, Antoine Darveau-Bernier, Geert Jan Talens, Anand Sivaramakrishnan, Chris J. Willott, Alexander W. Fullerton, Stephanie LaMassa, John B. Hutchings, Neil Rowlands, M. Begona Vila, Julia Zhou, David Aldridge, Michael Maszkiewicz, Mathilde Beaulieu , et al. (15 additional authors not shown)

Abstract: The Near Infrared Imager and Slitless Spectrograph instrument (NIRISS) is the Canadian Space Agency (CSA) contribution to the suite of four science instruments of JWST. As one of the three NIRISS observing modes, the Single Object Slitless Spectroscopy (SOSS) mode is tailor-made to undertake time-series observations of exoplanets to perform transit spectroscopy. The SOSS permits observing point so… ▽ More The Near Infrared Imager and Slitless Spectrograph instrument (NIRISS) is the Canadian Space Agency (CSA) contribution to the suite of four science instruments of JWST. As one of the three NIRISS observing modes, the Single Object Slitless Spectroscopy (SOSS) mode is tailor-made to undertake time-series observations of exoplanets to perform transit spectroscopy. The SOSS permits observing point sources between 0.6 and 2.8 um at a resolving power of 650 at 1.25 um using a slit-less cross-dispersing grism while its defocussing cylindrical lens enables observing targets as bright as J=6.7 by spreading light across 23 pixels along the cross-dispersion axis. This paper officially presents the design of the SOSS mode, its operation, characterization, and its performance, from ground-based testing and flight-based Commissioning. On-sky measurements demonstrate a peak photon conversion efficiency of 55% at 1.2 um. The first time-series on the A-type star BD+60o1753 achieves a flux stability close to the photon-noise limit, so far tested to a level of 20 parts per million on 40-minute time-scales after simply subtracting a long-term trend. Uncorrected 1/f noise residuals underneath the spectral traces add an extra source of noise equivalent to doubling the readout noise. Preliminary analysis of a HAT-P-14b transit time-series indicates that it is difficult to remove all the noise in pixels with partially saturated ramps. Overall, the SOSS delivers performance at the level required to tackle key exoplanet science programs such as detecting secondary atmospheres on terrestrial planets and measuring abundances of several chemical species in gas giants. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Comments: 41 pages, 28 figures, accepted for publication in PASP

The Near Infrared Imager and Slitless Spectrograph for the James Webb Space Telescope -- I. Instrument Overview and in-Flight Performance

Authors: Rene Doyon, C. J Willott, John B. Hutchings, Anand Sivaramakrishnan, Loic Albert, David Lafreniere, Neil Rowlands, M. Begona Vila, Andre R. Martel, Stephanie LaMassa, David Aldridge, Etienne Artigau, Peter Cameron, Pierre Chayer, Neil J. Cook, Rachel A. Cooper, Antoine Darveau-Bernier, Jean Dupuis, Colin Earnshaw, Nestor Espinoza, Joseph C. Filippazzo, Alexander W. Fullerton, Daniel Gaudreau, Roman Gawlik, Paul Goudfrooij , et al. (38 additional authors not shown)

Abstract: The Near-Infrared Imager and Slitless Spectrograph (NIRISS) is the science module of the Canadian-built Fine Guidance Sensor (FGS) onboard the James Webb Space Telescope (JWST). NIRISS has four observing modes: 1) broadband imaging featuring seven of the eight NIRCam broadband filters, 2) wide-field slitless spectroscopy (WFSS) at a resolving power of $\sim$150 between 0.8 and 2.2 $μ$m, 3) single-… ▽ More The Near-Infrared Imager and Slitless Spectrograph (NIRISS) is the science module of the Canadian-built Fine Guidance Sensor (FGS) onboard the James Webb Space Telescope (JWST). NIRISS has four observing modes: 1) broadband imaging featuring seven of the eight NIRCam broadband filters, 2) wide-field slitless spectroscopy (WFSS) at a resolving power of $\sim$150 between 0.8 and 2.2 $μ$m, 3) single-object cross-dispersed slitless spectroscopy (SOSS) enabling simultaneous wavelength coverage between 0.6 and 2.8 $μ$m at R$\sim$700, a mode optimized for exoplanet spectroscopy of relatively bright ($J<6.3$) stars and 4) aperture masking interferometry (AMI) between 2.8 and 4.8 $μ$m enabling high-contrast ($\sim10^{-3}-10^{-4}$) imaging at angular separations between 70 and 400 milliarcsec for relatively bright ($M<8$) sources. This paper presents an overview of the NIRISS instrument, its design, its scientific capabilities, and a summary of in-flight performance. NIRISS shows significantly better response shortward of $\sim2.5\,μ$m resulting in 10-40% sensitivity improvement for broadband and low-resolution spectroscopy compared to pre-flight predictions. Two time-series observations performed during instrument commissioning in the SOSS mode yield very stable spectro-photometry performance within $\sim$10% of the expected noise. The first space-based companion detection of the tight binary star AB Dor AC through AMI was demonstrated. △ Less

Submitted 5 June, 2023; originally announced June 2023.

Distance Rank Score: Unsupervised filter method for feature selection on imbalanced dataset

Authors: Katarina Firdova, Céline Labart, Arthur Martel

Abstract: This paper presents a new filter method for unsupervised feature selection. This method is particularly effective on imbalanced multi-class dataset, as in case of clusters of different anomaly types. Existing methods usually involve the variance of the features, which is not suitable when the different types of observations are not represented equally. Our method, based on Spearman's Rank Correlat… ▽ More This paper presents a new filter method for unsupervised feature selection. This method is particularly effective on imbalanced multi-class dataset, as in case of clusters of different anomaly types. Existing methods usually involve the variance of the features, which is not suitable when the different types of observations are not represented equally. Our method, based on Spearman's Rank Correlation between distances on the observations and on feature values, avoids this drawback. The performance of the method is measured on several clustering problems and is compared with existing filter methods suitable for unsupervised data. △ Less

Submitted 31 May, 2023; originally announced May 2023.

The James Webb Space Telescope Mission

Authors: Jonathan P. Gardner, John C. Mather, Randy Abbott, James S. Abell, Mark Abernathy, Faith E. Abney, John G. Abraham, Roberto Abraham, Yasin M. Abul-Huda, Scott Acton, Cynthia K. Adams, Evan Adams, David S. Adler, Maarten Adriaensen, Jonathan Albert Aguilar, Mansoor Ahmed, Nasif S. Ahmed, Tanjira Ahmed, Rüdeger Albat, Loïc Albert, Stacey Alberts, David Aldridge, Mary Marsha Allen, Shaune S. Allen, Martin Altenburg , et al. (983 additional authors not shown)

Abstract: Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astrono… ▽ More Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit. △ Less

Submitted 10 April, 2023; originally announced April 2023.

Comments: Accepted by PASP for the special issue on The James Webb Space Telescope Overview, 29 pages, 4 figures

Understanding metric-related pitfalls in image analysis validation

Authors: Annika Reinke, Minu D. Tizabi, Michael Baumgartner, Matthias Eisenmann, Doreen Heckmann-Nötzel, A. Emre Kavur, Tim Rädsch, Carole H. Sudre, Laura Acion, Michela Antonelli, Tal Arbel, Spyridon Bakas, Arriel Benis, Matthew Blaschko, Florian Buettner, M. Jorge Cardoso, Veronika Cheplygina, Jianxu Chen, Evangelia Christodoulou, Beth A. Cimini, Gary S. Collins, Keyvan Farahani, Luciana Ferrer, Adrian Galdran, Bram van Ginneken , et al. (53 additional authors not shown)

Abstract: Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibilit… ▽ More Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation. △ Less

Submitted 23 February, 2024; v1 submitted 3 February, 2023; originally announced February 2023.

Comments: Shared first authors: Annika Reinke and Minu D. Tizabi; shared senior authors: Lena Maier-Hein and Paul F. Jäger. Published in Nature Methods. arXiv admin note: text overlap with arXiv:2206.01653

Journal ref: Nature methods, 1-13 (2024)

Biomedical image analysis competitions: The state of current participation practice

Authors: Matthias Eisenmann, Annika Reinke, Vivienn Weru, Minu Dietlinde Tizabi, Fabian Isensee, Tim J. Adler, Patrick Godau, Veronika Cheplygina, Michal Kozubek, Sharib Ali, Anubha Gupta, Jan Kybic, Alison Noble, Carlos Ortiz de Solórzano, Samiksha Pachade, Caroline Petitjean, Daniel Sage, Donglai Wei, Elizabeth Wilden, Deepak Alapatt, Vincent Andrearczyk, Ujjwal Baid, Spyridon Bakas, Niranjan Balu, Sophia Bano , et al. (331 additional authors not shown)

Abstract: The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis,… ▽ More The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps. △ Less

Submitted 12 September, 2023; v1 submitted 16 December, 2022; originally announced December 2022.

The Near Infrared Imager and Slitless Spectrograph for JWST -- V. Kernel Phase Imaging and Data Analysis

Authors: Jens Kammerer, Rachel A. Cooper, Thomas Vandal, Deepashri Thatte, Frantz Martinache, Anand Sivaramakrishnan, Alexander Chaushev, Tomas Stolker, James P. Lloyd, Loïc Albert, René Doyon, Steph Sallum, Marshall D. Perrin, Laurent Pueyo, Antoine Mérand, Alexandre Gallenne, Alexandra Greenbaum, Joel Sanchez-Bermudez, Dori Blakely, Doug Johnstone, Kevin Volk, Andre Martel, Paul Goudfrooij, Michael R. Meyer, Chris J. Willott , et al. (4 additional authors not shown)

Abstract: Kernel phase imaging (KPI) enables the direct detection of substellar companions and circumstellar dust close to and below the classical (Rayleigh) diffraction limit. We present a kernel phase analysis of JWST NIRISS full pupil images taken during the instrument commissioning and compare the performance to closely related NIRISS aperture masking interferometry (AMI) observations. For this purpose,… ▽ More Kernel phase imaging (KPI) enables the direct detection of substellar companions and circumstellar dust close to and below the classical (Rayleigh) diffraction limit. We present a kernel phase analysis of JWST NIRISS full pupil images taken during the instrument commissioning and compare the performance to closely related NIRISS aperture masking interferometry (AMI) observations. For this purpose, we develop and make publicly available the custom "Kpi3Pipeline" enabling the extraction of kernel phase observables from JWST images. The extracted observables are saved into a new and versatile kernel phase FITS file (KPFITS) data exchange format. Furthermore, we present our new and publicly available "fouriever" toolkit which can be used to search for companions and derive detection limits from KPI, AMI, and long-baseline interferometry observations while accounting for correlated uncertainties in the model fitting process. Among the four KPI targets that were observed during NIRISS instrument commissioning, we discover a low-contrast (~1:5) close-in (~1 $λ/D$) companion candidate around CPD-66~562 and a new high-contrast (~1:170) detection separated by ~1.5 $λ/D$ from 2MASS~J062802.01-663738.0. The 5-$σ$ companion detection limits around the other two targets reach ~6.5 mag at ~200 mas and ~7 mag at ~400 mas. Comparing these limits to those obtained from the NIRISS AMI commissioning observations, we find that KPI and AMI perform similar in the same amount of observing time. Due to its 5.6 times higher throughput if compared to AMI, KPI is beneficial for observing faint targets and superior to AMI at separations >325 mas. At very small separations (<100 mas) and between ~250-325 mas, AMI slightly outperforms KPI which suffers from increased photon noise from the core and the first Airy ring of the point-spread function. △ Less

Submitted 3 November, 2022; v1 submitted 31 October, 2022; originally announced October 2022.

Comments: 34 pages, 17 figures, accepted for publication in PASP

The Near Infrared Imager and Slitless Spectrograph for the James Webb Space Telescope -- IV. Aperture Masking Interferometry

Authors: Anand Sivaramakrishnan, Peter Tuthill, James P. Lloyd, Alexandra Z. Greenbaum, Deepashri Thatte, Rachel A. Cooper, Thomas Vandal, Jens Kammerer, Joel Sanchez-Bermudez, Benjamin J. S. Pope, Dori Blakely, Loïc Albert, Neil J. Cook, Doug Johnstone, André R. Martel, Kevin Volk, Anthony Soulain, Étienne Artigau, David Lafrenière, Chris J. Willott, Sébastien Parmentier, K. E. Saavik Ford, Barry McKernan, M. Begoña Vila, Neil Rowlands , et al. (14 additional authors not shown)

Abstract: The James Webb Space Telescope's Near Infrared Imager and Slitless Spectrograph (JWST-NIRISS) flies a 7-hole non-redundant mask (NRM), the first such interferometer in space, operating at 3-5 \micron~wavelengths, and a bright limit of $\simeq 4$ magnitudes in W2. We describe the NIRISS Aperture Masking Interferometry (AMI) mode to help potential observers understand its underlying principles, pres… ▽ More The James Webb Space Telescope's Near Infrared Imager and Slitless Spectrograph (JWST-NIRISS) flies a 7-hole non-redundant mask (NRM), the first such interferometer in space, operating at 3-5 \micron~wavelengths, and a bright limit of $\simeq 4$ magnitudes in W2. We describe the NIRISS Aperture Masking Interferometry (AMI) mode to help potential observers understand its underlying principles, present some sample science cases, explain its operational observing strategies, indicate how AMI proposals can be developed with data simulations, and how AMI data can be analyzed. We also present key results from commissioning AMI. Since the allied Kernel Phase Imaging (KPI) technique benefits from AMI operational strategies, we also cover NIRISS KPI methods and analysis techniques, including a new user-friendly KPI pipeline. The NIRISS KPI bright limit is $\simeq 8$ W2 magnitudes. AMI (and KPI) achieve an inner working angle of $\sim 70$ mas that is well inside the $\sim 400$ mas NIRCam inner working angle for its circular occulter coronagraphs at comparable wavelengths. △ Less

Submitted 7 November, 2022; v1 submitted 31 October, 2022; originally announced October 2022.

Comments: 30 pages, 10 figures

The Science Performance of JWST as Characterized in Commissioning

Authors: Jane Rigby, Marshall Perrin, Michael McElwain, Randy Kimble, Scott Friedman, Matt Lallo, René Doyon, Lee Feinberg, Pierre Ferruit, Alistair Glasse, Marcia Rieke, George Rieke, Gillian Wright, Chris Willott, Knicole Colon, Stefanie Milam, Susan Neff, Christopher Stark, Jeff Valenti, Jim Abell, Faith Abney, Yasin Abul-Huda, D. Scott Acton, Evan Adams, David Adler , et al. (601 additional authors not shown)

Abstract: This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries f… ▽ More This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries for which it was built. Moreover, almost across the board, the science performance of JWST is better than expected; in most cases, JWST will go deeper faster than expected. The telescope and instrument suite have demonstrated the sensitivity, stability, image quality, and spectral range that are necessary to transform our understanding of the cosmos through observations spanning from near-earth asteroids to the most distant galaxies. △ Less

Submitted 10 April, 2023; v1 submitted 12 July, 2022; originally announced July 2022.

Comments: 5th version as accepted to PASP; 31 pages, 18 figures; https://iopscience.iop.org/article/10.1088/1538-3873/acb293

Journal ref: PASP 135 048001 (2023)

Metrics reloaded: Recommendations for image analysis validation

Authors: Lena Maier-Hein, Annika Reinke, Patrick Godau, Minu D. Tizabi, Florian Buettner, Evangelia Christodoulou, Ben Glocker, Fabian Isensee, Jens Kleesiek, Michal Kozubek, Mauricio Reyes, Michael A. Riegler, Manuel Wiesenfarth, A. Emre Kavur, Carole H. Sudre, Michael Baumgartner, Matthias Eisenmann, Doreen Heckmann-Nötzel, Tim Rädsch, Laura Acion, Michela Antonelli, Tal Arbel, Spyridon Bakas, Arriel Benis, Matthew Blaschko , et al. (49 additional authors not shown)

Abstract: Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen performance metrics often do not reflect the domain interest, thus failing to adequately measure scientific progress and hindering translation of ML techniques into practice. To overcome this, our large international ex… ▽ More Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen performance metrics often do not reflect the domain interest, thus failing to adequately measure scientific progress and hindering translation of ML techniques into practice. To overcome this, our large international expert consortium created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint - a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), data set and algorithm output. Based on the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as a classification task at image, object or pixel level, namely image-level classification, object detection, semantic segmentation, and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool, which also provides a point of access to explore weaknesses, strengths and specific recommendations for the most common validation metrics. The broad applicability of our framework across domains is demonstrated by an instantiation for various biological and medical image analysis use cases. △ Less

Submitted 23 February, 2024; v1 submitted 3 June, 2022; originally announced June 2022.

Comments: Shared first authors: Lena Maier-Hein, Annika Reinke. arXiv admin note: substantial text overlap with arXiv:2104.05642 Published in Nature Methods

Journal ref: Nature methods, 1-18 (2024)

Explainable multi-class anomaly detection on functional data

Authors: Mathieu Cura, Katarina Firdova, Céline Labart, Arthur Martel

Abstract: In this paper we describe an approach for anomaly detection and its explainability in multivariate functional data. The anomaly detection procedure consists of transforming the series into a vector of features and using an Isolation forest algorithm. The explainable procedure is based on the computation of the SHAP coefficients and on the use of a supervised decision tree. We apply it on simulated… ▽ More In this paper we describe an approach for anomaly detection and its explainability in multivariate functional data. The anomaly detection procedure consists of transforming the series into a vector of features and using an Isolation forest algorithm. The explainable procedure is based on the computation of the SHAP coefficients and on the use of a supervised decision tree. We apply it on simulated data to measure the performance of our method and on real data coming from industry. △ Less

Submitted 3 May, 2022; originally announced May 2022.

MSC Class: 62P30 62H30

ROOD-MRI: Benchmarking the robustness of deep learning segmentation models to out-of-distribution and corrupted data in MRI

Authors: Lyndon Boone, Mahdi Biparva, Parisa Mojiri Forooshani, Joel Ramirez, Mario Masellis, Robert Bartha, Sean Symons, Stephen Strother, Sandra E. Black, Chris Heyn, Anne L. Martel, Richard H. Swartz, Maged Goubran

Abstract: Deep artificial neural networks (DNNs) have moved to the forefront of medical image analysis due to their success in classification, segmentation, and detection challenges. A principal challenge in large-scale deployment of DNNs in neuroimage analysis is the potential for shifts in signal-to-noise ratio, contrast, resolution, and presence of artifacts from site to site due to variances in scanners… ▽ More Deep artificial neural networks (DNNs) have moved to the forefront of medical image analysis due to their success in classification, segmentation, and detection challenges. A principal challenge in large-scale deployment of DNNs in neuroimage analysis is the potential for shifts in signal-to-noise ratio, contrast, resolution, and presence of artifacts from site to site due to variances in scanners and acquisition protocols. DNNs are famously susceptible to these distribution shifts in computer vision. Currently, there are no benchmarking platforms or frameworks to assess the robustness of new and existing models to specific distribution shifts in MRI, and accessible multi-site benchmarking datasets are still scarce or task-specific. To address these limitations, we propose ROOD-MRI: a platform for benchmarking the Robustness of DNNs to Out-Of-Distribution (OOD) data, corruptions, and artifacts in MRI. The platform provides modules for generating benchmarking datasets using transforms that model distribution shifts in MRI, implementations of newly derived benchmarking metrics for image segmentation, and examples for using the methodology with new models and tasks. We apply our methodology to hippocampus, ventricle, and white matter hyperintensity segmentation in several large studies, providing the hippocampus dataset as a publicly available benchmark. By evaluating modern DNNs on these datasets, we demonstrate that they are highly susceptible to distribution shifts and corruptions in MRI. We show that while data augmentation strategies can substantially improve robustness to OOD data for anatomical segmentation tasks, modern DNNs using augmentation still lack robustness in more challenging lesion-based segmentation tasks. We finally benchmark U-Nets and transformer-based models, finding consistent differences in robustness to particular classes of transforms across architectures. △ Less

Submitted 11 March, 2022; originally announced March 2022.

Comments: 30 pages, 13 figures. For associated GitHub repository, see https://github.com/AICONSlab/roodmri

Metastatic Cancer Outcome Prediction with Injective Multiple Instance Pooling

Authors: Jianan Chen, Anne L. Martel

Abstract: Cancer stage is a large determinant of patient prognosis and management in many cancer types, and is often assessed using medical imaging modalities, such as CT and MRI. These medical images contain rich information that can be explored to stratify patients within each stage group to further improve prognostic algorithms. Although the majority of cancer deaths result from metastatic and multifocal… ▽ More Cancer stage is a large determinant of patient prognosis and management in many cancer types, and is often assessed using medical imaging modalities, such as CT and MRI. These medical images contain rich information that can be explored to stratify patients within each stage group to further improve prognostic algorithms. Although the majority of cancer deaths result from metastatic and multifocal disease, building imaging biomarkers for patients with multiple tumors has been a challenging task due to the lack of annotated datasets and standard study framework. In this paper, we process two public datasets to set up a benchmark cohort of 341 patient in total for studying outcome prediction of multifocal metastatic cancer. We identify the lack of expressiveness in common multiple instance classification networks and propose two injective multiple instance pooling functions that are better suited to outcome prediction. Our results show that multiple instance learning with injective pooling functions can achieve state-of-the-art performance in the non-small-cell lung cancer CT and head and neck CT outcome prediction benchmarking tasks. We will release the processed multifocal datasets, our code and the intermediate files i.e. extracted radiomic features to support further transparent and reproducible research. △ Less

Submitted 9 March, 2022; originally announced March 2022.

The James Webb Space Telescope Aperture Masking Interferometer

Authors: A. Soulain, A. Sivaramakrishnan, P. Tuthill, D. Thatte, K. Volk, R. Cooper, L. Albert, É. Artigau, N. Cook, R. Doyon, D. Johnstone, D. Lafrenière, A. Martel

Abstract: In less than a year, the James Webb Space Telescope (JWST) will inherit the mantle of being the world's pre-eminent infrared observatory. JWST will carry with it an Aperture Masking Interferometer (AMI) as one of the supported operational modes of the Near-InfraRed Imager and Slitless Spectrograph (NIRISS) instrument. Aboard such a powerful platform, the AMI mode will deliver the most advanced and… ▽ More In less than a year, the James Webb Space Telescope (JWST) will inherit the mantle of being the world's pre-eminent infrared observatory. JWST will carry with it an Aperture Masking Interferometer (AMI) as one of the supported operational modes of the Near-InfraRed Imager and Slitless Spectrograph (NIRISS) instrument. Aboard such a powerful platform, the AMI mode will deliver the most advanced and scientifically capable interferometer ever launched into space, exceeding anything that has gone before it by orders of magnitude in sensitivity. Here we present key aspects of the design and commissioning of this facility: data simulations ($\texttt{ami_sim}$), the extraction of interferometeric observables using two different approaches ($\texttt{IMPLANEIA}$ and $\texttt{AMICAL}$), an updated view of AMI's expected performance, and our reference star vetting programs. △ Less

Submitted 5 January, 2022; originally announced January 2022.

Comments: 18 pages, 10 figures, invited paper conference

Journal ref: Proceedings of the SPIE, Volume 11446, id. 1144611 18 pp. (2020)

BI-RADS BERT & Using Section Segmentation to Understand Radiology Reports

Authors: Grey Kuling, Dr. Belinda Curpen, Anne L. Martel

Abstract: Radiology reports are one of the main forms of communication between radiologists and other clinicians and contain important information for patient care. In order to use this information for research and automated patient care programs, it is necessary to convert the raw text into structured data suitable for analysis. State-of-the-art natural language processing (NLP) domain-specific contextual… ▽ More Radiology reports are one of the main forms of communication between radiologists and other clinicians and contain important information for patient care. In order to use this information for research and automated patient care programs, it is necessary to convert the raw text into structured data suitable for analysis. State-of-the-art natural language processing (NLP) domain-specific contextual word embeddings have been shown to achieve impressive accuracy for these tasks in medicine, but have yet to be utilized for section structure segmentation. In this work, we pre-trained a contextual embedding BERT model using breast radiology reports and developed a classifier that incorporated the embedding with auxiliary global textual features in order to perform section segmentation. This model achieved a 98% accuracy at segregating free text reports sentence by sentence into sections of information outlined in the Breast Imaging Reporting and Data System (BI-RADS) lexicon, a significant improvement over the Classic BERT model without auxiliary information. We then evaluated whether using section segmentation improved the downstream extraction of clinically relevant information such as modality/procedure, previous cancer, menopausal status, the purpose of the exam, breast density, and breast MRI background parenchymal enhancement. Using the BERT model pre-trained on breast radiology reports combined with section segmentation resulted in an overall accuracy of 95.9% in the field extraction tasks. This is a 17% improvement compared to an overall accuracy of 78.9% for field extraction with models using Classic BERT embeddings and not using section segmentation. Our work shows the strength of using BERT in radiology report analysis and the advantages of section segmentation in identifying key features of patient factors recorded in breast radiology reports. △ Less

Submitted 30 March, 2022; v1 submitted 14 October, 2021; originally announced October 2021.

Resource and data efficient self supervised learning

Authors: Ozan Ciga, Tony Xu, Anne L. Martel

Abstract: We investigate the utility of pretraining by contrastive self supervised learning on both natural-scene and medical imaging datasets when the unlabeled dataset size is small, or when the diversity within the unlabeled set does not lead to better representations. We use a two step approach which is analogous to supervised training with ImageNet initialization, where we pretrain networks that are al… ▽ More We investigate the utility of pretraining by contrastive self supervised learning on both natural-scene and medical imaging datasets when the unlabeled dataset size is small, or when the diversity within the unlabeled set does not lead to better representations. We use a two step approach which is analogous to supervised training with ImageNet initialization, where we pretrain networks that are already pretrained on ImageNet dataset to improve downstream task performance on the domain of interest. To improve the speed of convergence and the overall performance, we propose weight scaling and filter selection methods prior to second step of pretraining. We demonstrate the utility of this approach on three popular contrastive techniques, namely SimCLR, SWaV and BYOL. Benefits of double pretraining include better performance, faster convergence, ability to train with smaller batch sizes and smaller image dimensions with negligible differences in performance. We hope our work helps democratize self-supervision by enabling researchers to fine-tune models without requiring large clusters or long training times. △ Less

Submitted 3 September, 2021; originally announced September 2021.

Improving Self-supervised Learning with Hardness-aware Dynamic Curriculum Learning: An Application to Digital Pathology

Authors: Chetan L Srinidhi, Anne L Martel

Abstract: Self-supervised learning (SSL) has recently shown tremendous potential to learn generic visual representations useful for many image analysis tasks. Despite their notable success, the existing SSL methods fail to generalize to downstream tasks when the number of labeled training instances is small or if the domain shift between the transfer domains is significant. In this paper, we attempt to impr… ▽ More Self-supervised learning (SSL) has recently shown tremendous potential to learn generic visual representations useful for many image analysis tasks. Despite their notable success, the existing SSL methods fail to generalize to downstream tasks when the number of labeled training instances is small or if the domain shift between the transfer domains is significant. In this paper, we attempt to improve self-supervised pretrained representations through the lens of curriculum learning by proposing a hardness-aware dynamic curriculum learning (HaDCL) approach. To improve the robustness and generalizability of SSL, we dynamically leverage progressive harder examples via easy-to-hard and hard-to-very-hard samples during mini-batch downstream fine-tuning. We discover that by progressive stage-wise curriculum learning, the pretrained representations are significantly enhanced and adaptable to both in-domain and out-of-domain distribution data. We performed extensive validation on three histology benchmark datasets on both patch-wise and slide-level classification problems. Our curriculum based fine-tuning yields a significant improvement over standard fine-tuning, with a minimum improvement in area-under-the-curve (AUC) score of 1.7% and 2.2% on in-domain and out-of-domain distribution data, respectively. Further, we empirically show that our approach is more generic and adaptable to any SSL methods and does not impose any additional overhead complexity. Besides, we also outline the role of patch-based versus slide-based curriculum learning in histopathology to provide practical insights into the success of curriculum based fine-tuning of SSL methods. Code is released at https://github.com/srinidhiPY/ICCV-CDPATH2021-ID-8 △ Less

Submitted 5 October, 2021; v1 submitted 16 August, 2021; originally announced August 2021.

Comments: Accepted at ICCV 2021 CDpath workshop

Common Limitations of Image Processing Metrics: A Picture Story

Authors: Annika Reinke, Minu D. Tizabi, Carole H. Sudre, Matthias Eisenmann, Tim Rädsch, Michael Baumgartner, Laura Acion, Michela Antonelli, Tal Arbel, Spyridon Bakas, Peter Bankhead, Arriel Benis, Matthew Blaschko, Florian Buettner, M. Jorge Cardoso, Jianxu Chen, Veronika Cheplygina, Evangelia Christodoulou, Beth Cimini, Gary S. Collins, Sandy Engelhardt, Keyvan Farahani, Luciana Ferrer, Adrian Galdran, Bram van Ginneken , et al. (68 additional authors not shown)

Abstract: While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using spe… ▽ More While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using specific metrics for a given image analysis task. These are typically related to (1) the disregard of inherent metric properties, such as the behaviour in the presence of class imbalance or small target structures, (2) the disregard of inherent data set properties, such as the non-independence of the test cases, and (3) the disregard of the actual biomedical domain interest that the metrics should reflect. This living dynamically document has the purpose to illustrate important limitations of performance metrics commonly applied in the field of image analysis. In this context, it focuses on biomedical image analysis problems that can be phrased as image-level classification, semantic segmentation, instance segmentation, or object detection task. The current version is based on a Delphi process on metrics conducted by an international consortium of image analysis experts from more than 60 institutions worldwide. △ Less

Submitted 6 December, 2023; v1 submitted 12 April, 2021; originally announced April 2021.

Comments: Shared first authors: Annika Reinke and Minu D. Tizabi. This is a dynamic paper on limitations of commonly used metrics. It discusses metrics for image-level classification, semantic and instance segmentation, and object detection. For missing use cases, comments or questions, please contact [email protected]. Substantial contributions to this document will be acknowledged with a co-authorship

Self-supervised driven consistency training for annotation efficient histopathology image analysis

Authors: Chetan L. Srinidhi, Seung Wook Kim, Fu-Der Chen, Anne L. Martel

Abstract: Training a neural network with a large labeled dataset is still a dominant paradigm in computational histopathology. However, obtaining such exhaustive manual annotations is often expensive, laborious, and prone to inter and Intra-observer variability. While recent self-supervised and semi-supervised methods can alleviate this need by learn-ing unsupervised feature representations, they still stru… ▽ More Training a neural network with a large labeled dataset is still a dominant paradigm in computational histopathology. However, obtaining such exhaustive manual annotations is often expensive, laborious, and prone to inter and Intra-observer variability. While recent self-supervised and semi-supervised methods can alleviate this need by learn-ing unsupervised feature representations, they still struggle to generalize well to downstream tasks when the number of labeled instances is small. In this work, we overcome this challenge by leveraging both task-agnostic and task-specific unlabeled data based on two novel strategies: i) a self-supervised pretext task that harnesses the underlying multi-resolution contextual cues in histology whole-slide images to learn a powerful supervisory signal for unsupervised representation learning; ii) a new teacher-student semi-supervised consistency paradigm that learns to effectively transfer the pretrained representations to downstream tasks based on prediction consistency with the task-specific un-labeled data. We carry out extensive validation experiments on three histopathology benchmark datasets across two classification and one regression-based tasks, i.e., tumor metastasis detection, tissue type classification, and tumor cellularity quantification. Under limited-label data, the proposed method yields tangible improvements, which is close or even outperforming other state-of-the-art self-supervised and supervised baselines. Furthermore, we empirically show that the idea of bootstrap** the self-supervised pretrained features is an effective way to improve the task-specific semi-supervised learning on standard benchmarks. Code and pretrained models will be made available at: https://github.com/srinidhiPY/SSL_CR_Histo △ Less

Submitted 3 October, 2021; v1 submitted 7 February, 2021; originally announced February 2021.

Journal ref: Medical Image Analysis, Volume 75, January 2022

AMINN: Autoencoder-based Multiple Instance Neural Network Improves Outcome Prediction of Multifocal Liver Metastases

Authors: Jianan Chen, Helen M. C. Cheung, Laurent Milot, Anne L. Martel

Abstract: Colorectal cancer is one of the most common and lethal cancers and colorectal cancer liver metastases (CRLM) is the major cause of death in patients with colorectal cancer. Multifocality occurs frequently in CRLM, but is relatively unexplored in CRLM outcome prediction. Most existing clinical and imaging biomarkers do not take the imaging features of all multifocal lesions into account. In this pa… ▽ More Colorectal cancer is one of the most common and lethal cancers and colorectal cancer liver metastases (CRLM) is the major cause of death in patients with colorectal cancer. Multifocality occurs frequently in CRLM, but is relatively unexplored in CRLM outcome prediction. Most existing clinical and imaging biomarkers do not take the imaging features of all multifocal lesions into account. In this paper, we present an end-to-end autoencoder-based multiple instance neural network (AMINN) for the prediction of survival outcomes in multifocal CRLM patients using radiomic features extracted from contrast-enhanced MRIs. Specifically, we jointly train an autoencoder to reconstruct input features and a multiple instance network to make predictions by aggregating information from all tumour lesions of a patient. Also, we incorporate a two-step normalization technique to improve the training of deep neural networks, built on the observation that the distributions of radiomic features are almost always severely skewed. Experimental results empirically validated our hypothesis that incorporating imaging features of all lesions improves outcome prediction for multifocal cancer. The proposed AMINN framework achieved an area under the ROC curve (AUC) of 0.70, which is 11.4% higher than the best baseline method. A risk score based on the outputs of AMINN achieved superior prediction in our multifocal CRLM cohort. The effectiveness of incorporating all lesions and applying two-step normalization is demonstrated by a series of ablation studies. A Keras implementation of AMINN is released. △ Less

Submitted 9 July, 2021; v1 submitted 12 December, 2020; originally announced December 2020.

Comments: Early accepted by MICCAI 2021

Overcoming the limitations of patch-based learning to detect cancer in whole slide images

Authors: Ozan Ciga, Tony Xu, Sharon Nofech-Mozes, Shawna Noy, Fang-I Lu, Anne L. Martel

Abstract: Whole slide images (WSIs) pose unique challenges when training deep learning models. They are very large which makes it necessary to break each image down into smaller patches for analysis, image features have to be extracted at multiple scales in order to capture both detail and context, and extreme class imbalances may exist. Significant progress has been made in the analysis of these images, th… ▽ More Whole slide images (WSIs) pose unique challenges when training deep learning models. They are very large which makes it necessary to break each image down into smaller patches for analysis, image features have to be extracted at multiple scales in order to capture both detail and context, and extreme class imbalances may exist. Significant progress has been made in the analysis of these images, thanks largely due to the availability of public annotated datasets. We postulate, however, that even if a method scores well on a challenge task, this success may not translate to good performance in a more clinically relevant workflow. Many datasets consist of image patches which may suffer from data curation bias; other datasets are only labelled at the whole slide level and the lack of annotations across an image may mask erroneous local predictions so long as the final decision is correct. In this paper, we outline the differences between patch or slide-level classification versus methods that need to localize or segment cancer accurately across the whole slide, and we experimentally verify that best practices differ in both cases. We apply a binary cancer detection network on post neoadjuvant therapy breast cancer WSIs to find the tumor bed outlining the extent of cancer, a task which requires sensitivity and precision across the whole slide. We extensively study multiple design choices and their effects on the outcome, including architectures and augmentations. Furthermore, we propose a negative data sampling strategy, which drastically reduces the false positive rate (7% on slide level) and improves each metric pertinent to our problem, with a 15% reduction in the error of tumor extent. △ Less

Submitted 1 December, 2020; originally announced December 2020.

Self supervised contrastive learning for digital histopathology

Authors: Ozan Ciga, Tony Xu, Anne L. Martel

Abstract: Unsupervised learning has been a long-standing goal of machine learning and is especially important for medical image analysis, where the learning can compensate for the scarcity of labeled datasets. A promising subclass of unsupervised learning is self-supervised learning, which aims to learn salient features using the raw input as the learning signal. In this paper, we use a contrastive self-sup… ▽ More Unsupervised learning has been a long-standing goal of machine learning and is especially important for medical image analysis, where the learning can compensate for the scarcity of labeled datasets. A promising subclass of unsupervised learning is self-supervised learning, which aims to learn salient features using the raw input as the learning signal. In this paper, we use a contrastive self-supervised learning method called SimCLR that achieved state-of-the-art results on natural-scene images and apply this method to digital histopathology by collecting and pretraining on 57 histopathology datasets without any labels. We find that combining multiple multi-organ datasets with different types of staining and resolution properties improves the quality of the learned features. Furthermore, we find using more images for pretraining leads to a better performance in multiple downstream tasks. Linear classifiers trained on top of the learned features show that networks pretrained on digital histopathology datasets perform better than ImageNet pretrained networks, boosting task performances by more than 28% in F1 scores on average. These findings may also be useful when applying newer contrastive techniques to histopathology data. Pretrained PyTorch models are made publicly available at https://github.com/ozanciga/self-supervised-histopathology. △ Less

Submitted 7 September, 2021; v1 submitted 27 November, 2020; originally announced November 2020.

A Novel Experimental Approach for Nanostructure Analysis: Simultaneous Small Angle X-ray and Neutron Scattering (SAXS/SANS)

Authors: Ezzeldin Metwalli, Klaus Götz, Sebastian Lages, Christian Bär, Tobias Zech, Dennis M. Noll, Isabel Schuldes, Torben Schindler, Annemarie Prihoda, Herbert Lang, Jürgen Grasser, Mark Jacques, Luc Didier, Amrouni Cyril, Anne Martel, Lionel Porcar, Tobias Unruh

Abstract: Exploiting small angle X-ray and neutron scattering (SAXS/SANS) on the same sample volume at the same time provides complementary nanoscale structural information at two different contrast situations. Compared with an independent experimental approach, the truly combined SAXS/SANS experimental approach ensures the exactness of the probed samples particularly for in-situ studies. Here, we introduce… ▽ More Exploiting small angle X-ray and neutron scattering (SAXS/SANS) on the same sample volume at the same time provides complementary nanoscale structural information at two different contrast situations. Compared with an independent experimental approach, the truly combined SAXS/SANS experimental approach ensures the exactness of the probed samples particularly for in-situ studies. Here, we introduce an advanced portable SAXS system that is dimensionally suitable for installation at D22 zone of ILL. The SAXS apparatus is based on a RIGAKU copper/molybdenum switchable microfocus rotating anode X-ray generator and a DECTRIS detector with a changeable sample-to-detector distance of up to 1.6 m in a vacuum chamber. A science case has been presented to demonstrate the uniqueness of the newly established method at ILL. Temporal structural rearrangements of both, organic stabilizing agents and organically capped gold colloidal particles during gold nanoparticle growth are simultaneously probed, enabling immediate correlated structural information. The newly established nano-analytical method at ILL will open the way for real time investigations of a wide range of innovative nanomaterials and will enable comprehensive in-situ studies on biological systems. A potential development of a fully-automated SAXS/SANS system with a common control environment and additional sample environments, permitting a continual and efficient operation of the system at the hands of ILL users, has also been introduced. △ Less

Submitted 27 March, 2020; originally announced March 2020.

Comments: 22 pages, 9+2 figures

Learning to segment images with classification labels

Authors: Ozan Ciga, Anne L. Martel

Abstract: Two of the most common tasks in medical imaging are classification and segmentation. Either task requires labeled data annotated by experts, which is scarce and expensive to collect. Annotating data for segmentation is generally considered to be more laborious as the annotator has to draw around the boundaries of regions of interest, as opposed to assigning image patches a class label. Furthermore… ▽ More Two of the most common tasks in medical imaging are classification and segmentation. Either task requires labeled data annotated by experts, which is scarce and expensive to collect. Annotating data for segmentation is generally considered to be more laborious as the annotator has to draw around the boundaries of regions of interest, as opposed to assigning image patches a class label. Furthermore, in tasks such as breast cancer histopathology, any realistic clinical application often includes working with whole slide images, whereas most publicly available training data are in the form of image patches, which are given a class label. We propose an architecture that can alleviate the requirements for segmentation-level ground truth by making use of image-level labels to reduce the amount of time spent on data curation. In addition, this architecture can help unlock the potential of previously acquired image-level datasets on segmentation tasks by annotating a small number of regions of interest. In our experiments, we show using only one segmentation-level annotation per class, we can achieve performance comparable to a fully annotated dataset. △ Less

Submitted 29 November, 2020; v1 submitted 28 December, 2019; originally announced December 2019.

Comments: Published on Elsevier, Medical Image Analysis

Journal ref: Medical Image Analysis, Volume 68, 2021, 101912, ISSN 1361-8415

Deep neural network models for computational histopathology: A survey

Authors: Chetan L. Srinidhi, Ozan Ciga, Anne L. Martel

Abstract: Histopathological images contain rich phenotypic information that can be used to monitor underlying mechanisms contributing to diseases progression and patient survival outcomes. Recently, deep learning has become the mainstream methodological choice for analyzing and interpreting cancer histology images. In this paper, we present a comprehensive review of state-of-the-art deep learning approaches… ▽ More Histopathological images contain rich phenotypic information that can be used to monitor underlying mechanisms contributing to diseases progression and patient survival outcomes. Recently, deep learning has become the mainstream methodological choice for analyzing and interpreting cancer histology images. In this paper, we present a comprehensive review of state-of-the-art deep learning approaches that have been used in the context of histopathological image analysis. From the survey of over 130 papers, we review the fields progress based on the methodological aspect of different machine learning strategies such as supervised, weakly supervised, unsupervised, transfer learning and various other sub-variants of these methods. We also provide an overview of deep learning based survival models that are applicable for disease-specific prognosis tasks. Finally, we summarize several existing open datasets and highlight critical challenges and limitations with current deep learning approaches, along with possible avenues for future research. △ Less

Submitted 26 October, 2020; v1 submitted 27 December, 2019; originally announced December 2019.

Comments: Published in Medical Image Analysis, Vol. 67, Jan 2021. (10.1016/j.media.2020.101813)

Journal ref: Medical Image Analysis, Vol. 67, Jan 2021

BIAS: Transparent reporting of biomedical image analysis challenges

Authors: Lena Maier-Hein, Annika Reinke, Michal Kozubek, Anne L. Martel, Tal Arbel, Matthias Eisenmann, Allan Hanbuary, Pierre Jannin, Henning Müller, Sinan Onogur, Julio Saez-Rodriguez, Bram van Ginneken, Annette Kopp-Schneider, Bennett Landman

Abstract: The number of biomedical image analysis challenges organized per year is steadily increasing. These international competitions have the purpose of benchmarking algorithms on common data sets, typically to identify the best method for a given problem. Recent research, however, revealed that common practice related to challenge reporting does not allow for adequate interpretation and reproducibility… ▽ More The number of biomedical image analysis challenges organized per year is steadily increasing. These international competitions have the purpose of benchmarking algorithms on common data sets, typically to identify the best method for a given problem. Recent research, however, revealed that common practice related to challenge reporting does not allow for adequate interpretation and reproducibility of results. To address the discrepancy between the impact of challenges and the quality (control), the Biomedical I mage Analysis ChallengeS (BIAS) initiative developed a set of recommendations for the reporting of challenges. The BIAS statement aims to improve the transparency of the reporting of a biomedical image analysis challenge regardless of field of application, image modality or task category assessed. This article describes how the BIAS statement was developed and presents a checklist which authors of biomedical image analysis challenges are encouraged to include in their submission when giving a paper on a challenge into review. The purpose of the checklist is to standardize and facilitate the review process and raise interpretability and reproducibility of challenge results by making relevant information explicit. △ Less

Submitted 31 August, 2020; v1 submitted 9 October, 2019; originally announced October 2019.

Comments: 2 Appendices - Appendix A: BIAS reporting guideline for biomedical image analysis challenges, Appendix B: Glossary; 2 Supplements - Suppl 1: Form for summarizing information on challenge organization, Suppl 2: Structured description of a challenge design

Unsupervised Clustering of Quantitative Imaging Phenotypes using Autoencoder and Gaussian Mixture Model

Authors: Jianan Chen, Laurent Milot, Helen M. C. Cheung, Anne L. Martel

Abstract: Quantitative medical image computing (radiomics) has been widely applied to build prediction models from medical images. However, overfitting is a significant issue in conventional radiomics, where a large number of radiomic features are directly used to train and test models that predict genotypes or clinical outcomes. In order to tackle this problem, we propose an unsupervised learning pipeline… ▽ More Quantitative medical image computing (radiomics) has been widely applied to build prediction models from medical images. However, overfitting is a significant issue in conventional radiomics, where a large number of radiomic features are directly used to train and test models that predict genotypes or clinical outcomes. In order to tackle this problem, we propose an unsupervised learning pipeline composed of an autoencoder for representation learning of radiomic features and a Gaussian mixture model based on minimum message length criterion for clustering. By incorporating probabilistic modeling, disease heterogeneity has been taken into account. The performance of the proposed pipeline was evaluated on an institutional MRI cohort of 108 patients with colorectal cancer liver metastases. Our approach is capable of automatically selecting the optimal number of clusters and assigns patients into clusters (imaging subtypes) with significantly different survival rates. Our method outperforms other unsupervised clustering methods that have been used for radiomics analysis and has comparable performance to a state-of-the-art imaging biomarker. △ Less

Submitted 6 September, 2019; originally announced September 2019.

Comments: Accepted at MICCAI 2019

Intensity augmentation for domain transfer of whole breast segmentation in MRI

Authors: Linde S. Hesse, Grey Kuling, Mitko Veta, Anne L. Martel

Abstract: The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar T1-weighted MR protocol is used. There has, however, been little work addressing th… ▽ More The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar T1-weighted MR protocol is used. There has, however, been little work addressing the problem of domain adaptation when image intensities or patient orientation differ markedly between the training set and an unseen test set. To overcome the domain shift we propose to apply extensive intensity augmentation in addition to geometric augmentation during training. We explored both style transfer and a novel intensity remap** approach as intensity augmentation strategies. For our experiments, we trained a 3D U-net on T1-weighted scans and tested on T2-weighted scans. By applying intensity augmentation we increased segmentation performance from a DSC of 0.71 to 0.90. This performance is very close to the baseline performance of training and testing on T2-weighted scans (0.92). Furthermore, we applied our network to an independent test set made up of publicly available scans acquired using a T1-weighted TWIST sequence and a different coil configuration. On this dataset we obtained a performance of 0.89, close to the inter-observer variability of the ground truth segmentations (0.92). Our results show that using intensity augmentation in addition to geometric augmentation is a suitable method to overcome the intensity domain shift and we expect it to be useful for a wide range of segmentation tasks. △ Less

Submitted 5 September, 2019; originally announced September 2019.

Comments: Preprint

Multi-layer Domain Adaptation for Deep Convolutional Networks

Authors: Ozan Ciga, Jianan Chen, Anne Martel

Abstract: Despite their success in many computer vision tasks, convolutional networks tend to require large amounts of labeled data to achieve generalization. Furthermore, the performance is not guaranteed on a sample from an unseen domain at test time, if the network was not exposed to similar samples from that domain at training time. This hinders the adoption of these techniques in clinical setting where… ▽ More Despite their success in many computer vision tasks, convolutional networks tend to require large amounts of labeled data to achieve generalization. Furthermore, the performance is not guaranteed on a sample from an unseen domain at test time, if the network was not exposed to similar samples from that domain at training time. This hinders the adoption of these techniques in clinical setting where the imaging data is scarce, and where the intra- and inter-domain variance of the data can be substantial. We propose a domain adaptation technique that is especially suitable for deep networks to alleviate this requirement of labeled data. Our method utilizes gradient reversal layers and Squeezeand-Excite modules to stabilize the training in deep networks. The proposed method was applied to publicly available histopathology and chest X-ray databases and achieved superior performance to existing state-of-the-art networks with and without domain adaptation. Depending on the application, our method can improve multi-class classification accuracy by 5-20% compared to DANN introduced in (Ganin, 2014). △ Less

Submitted 5 September, 2019; originally announced September 2019.

Comments: To be presented on Domain Adaptation and Representation Transfer 2019

Cluster-Based Learning from Weakly Labeled Bags in Digital Pathology

Authors: Shazia Akbar, Anne L. Martel

Abstract: To alleviate the burden of gathering detailed expert annotations when training deep neural networks, we propose a weakly supervised learning approach to recognize metastases in microscopic images of breast lymph nodes. We describe an alternative training loss which clusters weakly labeled bags in latent space to inform relevance of patch-instances during training of a convolutional neural network.… ▽ More To alleviate the burden of gathering detailed expert annotations when training deep neural networks, we propose a weakly supervised learning approach to recognize metastases in microscopic images of breast lymph nodes. We describe an alternative training loss which clusters weakly labeled bags in latent space to inform relevance of patch-instances during training of a convolutional neural network. We evaluate our method on the Camelyon dataset which contains high-resolution digital slides of breast lymph nodes, where labels are provided at the image-level and only subsets of patches are made available during training. △ Less

Submitted 28 November, 2018; originally announced December 2018.

Comments: Machine Learning for Health (ML4H) Workshop at NeurIPS 2018

Report number: ML4H/2018/27

Kinky DNA in solution: Small angle scattering study of a nucleosome positioning sequence

Authors: Torben Schindler, Adrián González, Ramachandran Boopathi, Marta Marty Roda, Lorena Romero-Santacreu, Andrew Wildes, Lionel Porcar, Anne Martel, Nikos Theodorakopoulos, Santiago Cuesta-López, Dimitar Angelov, Tobias Unruh, Michel Peyrard

Abstract: DNA is a flexible molecule, but the degree of its flexibility is subject to debate. The commonly-accepted persistence length of $l_p \approx 500\,$Å is inconsistent with recent studies on short-chain DNA that show much greater flexibility but do not probe its origin. We have performed X-ray and neutron small-angle scattering on a short DNA sequence containing a strong nucleosome positioning elemen… ▽ More DNA is a flexible molecule, but the degree of its flexibility is subject to debate. The commonly-accepted persistence length of $l_p \approx 500\,$Å is inconsistent with recent studies on short-chain DNA that show much greater flexibility but do not probe its origin. We have performed X-ray and neutron small-angle scattering on a short DNA sequence containing a strong nucleosome positioning element, and analyzed the results using a modified Kratky-Porod model to determine possible conformations. Our results support a hypothesis from Crick and Klug in 1975 that some DNA sequences in solution can have sharp kinks, potentially resolving the discrepancy. Our conclusions are supported by measurements on a radiation-damaged sample, where single-strand breaks lead to increased flexibility and by an analysis of data from another sequence, which does not have kinks, but where our method can detect a locally enhanced flexibility due to an $AT$-domain. △ Less

Submitted 25 September, 2018; originally announced September 2018.

Comments: 24 pages, 10 figures

Journal ref: Phys. Rev. E 98, 042417 (2018)

Transitioning between Convolutional and Fully Connected Layers in Neural Networks

Authors: Shazia Akbar, Mohammad Peikari, Sherine Salama, Sharon Nofech-Mozes, Anne Martel

Abstract: Digital pathology has advanced substantially over the last decade however tumor localization continues to be a challenging problem due to highly complex patterns and textures in the underlying tissue bed. The use of convolutional neural networks (CNNs) to analyze such complex images has been well adopted in digital pathology. However in recent years, the architecture of CNNs have altered with the… ▽ More Digital pathology has advanced substantially over the last decade however tumor localization continues to be a challenging problem due to highly complex patterns and textures in the underlying tissue bed. The use of convolutional neural networks (CNNs) to analyze such complex images has been well adopted in digital pathology. However in recent years, the architecture of CNNs have altered with the introduction of inception modules which have shown great promise for classification tasks. In this paper, we propose a modified "transition" module which learns global average pooling layers from filters of varying sizes to encourage class-specific filters at multiple spatial resolutions. We demonstrate the performance of the transition module in AlexNet and ZFNet, for classifying breast tumors in two independent datasets of scanned histology sections, of which the transition module was superior. △ Less

Submitted 18 July, 2017; originally announced July 2017.

Comments: This work is to appear at the 3rd workshop on Deep Learning in Medical Image Analysis (DLMIA), MICCAI 2017

Interpreting extracted rules from ensemble of trees: Application to computer-aided diagnosis of breast MRI

Authors: Cristina Gallego-Ortiz, Anne L. Martel

Abstract: High predictive performance and ease of use and interpretability are important requirements for the applicability of a computer-aided diagnosis (CAD) to human reading studies. We propose a CAD system specifically designed to be more comprehensible to the radiologist reviewing screening breast MRI studies. Multiparametric imaging features are combined to produce a CAD system for differentiating can… ▽ More High predictive performance and ease of use and interpretability are important requirements for the applicability of a computer-aided diagnosis (CAD) to human reading studies. We propose a CAD system specifically designed to be more comprehensible to the radiologist reviewing screening breast MRI studies. Multiparametric imaging features are combined to produce a CAD system for differentiating cancerous and non-cancerous lesions. The complete system uses a rule-extraction algorithm to present lesion classification results in an easy to understand graph visualization. △ Less

Submitted 27 June, 2016; originally announced June 2016.

Comments: presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY

NIRISS aperture masking interferometry: an overview of science opportunities

Authors: Étienne Artigau, Anand Sivaramakrishnan, Alexandra Z. Greenbaum, René Doyon, Paul Goudfrooij, Alex W. Fullerton, David Lafrenière, Kevin Volk, Loïc Albert, André Martel, K. E. Saavik Ford, Barry L. McKernan

Abstract: JWST's Near-Infrared Imager and Slitless Spectrograph (NIRISS) includes an Aperture Masking Interferometry (AMI) mode designed to be used between 2.7μm and 4.8μm. At these wavelengths, it will have the highest angular resolution of any mode on JWST, and, for faint targets, of any existing or planned infrastructure. NIRISS AMI is uniquely suited to detect thermal emission of young massive planets a… ▽ More JWST's Near-Infrared Imager and Slitless Spectrograph (NIRISS) includes an Aperture Masking Interferometry (AMI) mode designed to be used between 2.7μm and 4.8μm. At these wavelengths, it will have the highest angular resolution of any mode on JWST, and, for faint targets, of any existing or planned infrastructure. NIRISS AMI is uniquely suited to detect thermal emission of young massive planets and will permit the characterization of the mid-IR flux of exoplanets discovered by the GPI and SPHERE adaptive optics surveys. It will also directly detect massive planets found by GAIA through astrometric accelerations, providing the first opportunity ever to get both a mass and a flux measurement for non-transiting giant planets. NIRISS AMI will also enable the study of the nuclear environment of AGNs. △ Less

Submitted 26 June, 2014; originally announced June 2014.

Comments: Presented at SPIE Astronomical Telescopes + Instrumentation 2014

AGN and quasar science with aperture masking interferometry on the James Webb Space Telescope

Authors: K. E. Saavik Ford, Barry McKernan, Anand Sivaramakrishnan, André R. Martel, Anton Koekemoer, David Lafrenière, Sébastien Parmentier

Abstract: Due to feedback from accretion onto supermassive black holes (SMBHs), Active Galactic Nuclei (AGNs) are believed to play a key role in LambdaCDM cosmology and galaxy formation. However, AGNs' extreme luminosities and the small angular size of their accretion flows create a challenging imaging problem. We show James Webb Space Telescope's Near Infrared Imager and Slitless Spectrograph (JWST-NIRISS)… ▽ More Due to feedback from accretion onto supermassive black holes (SMBHs), Active Galactic Nuclei (AGNs) are believed to play a key role in LambdaCDM cosmology and galaxy formation. However, AGNs' extreme luminosities and the small angular size of their accretion flows create a challenging imaging problem. We show James Webb Space Telescope's Near Infrared Imager and Slitless Spectrograph (JWST-NIRISS) Aperture Masking Interferometry (AMI) mode will enable true imaging (i.e. without any requirement of prior assumptions on source geometry) at ~65 mas angular resolution at the centers of AGNs. This is advantageous for studying complex extended accretion flows around SMBHs, and in other areas of angular-resolution-limited astrophysics. By simulating data sequences incorporating expected sources of noise, we demonstrate that JWST-NIRISS AMI mode can map extended structure at a pixel-to-pixel contrast of ~10^{-2} around an L=7.5 point source, using short exposure times (minutes). Such images will test models of AGN feedback, fuelling and structure (complementary with ALMA observations), and are not currently supported by any ground-based IR interferometer or telescope. Binary point source contrast with NIRISS is ~10^{-4} (for observing binary nuclei in merging galaxies), significantly better than current ground-based optical or IR interferometry. JWST-NIRISS' seven-hole non-redundant mask has a throughput of 15%, and utilizes NIRISS' F277W (2.77\micron), F380M (3.8\micron), F430M (4.3\micron), and F480M (4.8\micron) filters. NIRISS' square pixels are 65mas per side, with a field of view ~2\arcmin x 2\arcmin. We also extrapolate our results to AGN science enabled by non-redundant masking on future 2.4m and 16m space telescopes working at long-UV to near-IR wavelengths. △ Less

Submitted 20 March, 2014; v1 submitted 2 January, 2014; originally announced January 2014.

Comments: 18 pages, 17 figures, accepted to ApJ; Revised version fixes typo in section 2.1 (to reflect factor of 2 improvement); faster download for figures

Journal ref: 2014, ApJ, 783, 73

Optical and IR Photometry of Globular Clusters in NGC1399: Evidence for Color-Metallicity Nonlinearity

Authors: John P. Blakeslee, Hyejeon Cho, Eric W. Peng, Laura Ferrarese, Andres Jordan, Andre R. Martel

Abstract: We combine new Wide Field Camera~3 IR Channel (WFC3/IR) F160W (H) imaging data for NGC1399, the central galaxy in the Fornax cluster, with archival F475W (g), F606W (V), F814W (I), and F850LP (z) optical data from the Advanced Camera for Surveys (ACS). The purely optical g-I, V-I, and g-z colors of NGC1399's rich globular cluster (GC) system exhibit clear bimodality, at least for magnitudes… ▽ More We combine new Wide Field Camera~3 IR Channel (WFC3/IR) F160W (H) imaging data for NGC1399, the central galaxy in the Fornax cluster, with archival F475W (g), F606W (V), F814W (I), and F850LP (z) optical data from the Advanced Camera for Surveys (ACS). The purely optical g-I, V-I, and g-z colors of NGC1399's rich globular cluster (GC) system exhibit clear bimodality, at least for magnitudes $I_814 > 21.5$. The optical-IR I-H color distribution appears unimodal, and this impression is confirmed by mixture modeling analysis. The V-H colors show marginal evidence for bimodality, consistent with bimodality in V-I and unimodality in I-H. If bimodality is imposed for I-H with a double Gaussian model, the preferred blue/red split differs from that for optical colors; these "differing bimodalities" mean that the optical and optical-IR colors cannot both be linearly proportional to metallicity. Consistent with the differing color distributions, the dependence of I-H on g-I for the matched GC sample is significantly nonlinear, with an inflection point near the trough in the g-I color distribution; the result is similar for the I-H dependence on g-z colors taken from the ACS Fornax Cluster Survey. These g-z colors have been calibrated empirically against metallicity; applying this calibration yields a continuous, skewed, but single-peaked metallicity distribution. Taken together, these results indicate that nonlinear color-metallicity relations play an important role in sha** the observed bimodal distributions of optical colors in extragalactic GC systems. △ Less

Submitted 4 January, 2012; originally announced January 2012.

Comments: 15 pages, 12 figures, accepted for publication in the Astrophysical Journal

A Multiwavelength Spectral and Polarimetric Study of the Jet of 3C 264

Authors: Eric S. Perlman, C. Alex Padgett, Markos Georganopoulos, Diana M. Worrall, Joel H. Kastner, Geoffrey Franz, Mark Birkinshaw, Fred Dulwich, Christopher P. O'Dea, Stefi A. Baum, William B. Sparks, John A. Biretta, Lucas Lara, Sebastian Jester, Andre Martel

Abstract: We present a comprehensive multiband spectral and polarimetric study of the jet of 3C 264 (NGC 3862). Included in this study are three HST optical and ultraviolet polarimetry data sets, along with new and archival VLA radio imaging and polarimetry, a re-analysis of numerous HST broadband data sets from the near infrared to the far ultraviolet, and a Chandra ACIS-S observation. We investigate sim… ▽ More We present a comprehensive multiband spectral and polarimetric study of the jet of 3C 264 (NGC 3862). Included in this study are three HST optical and ultraviolet polarimetry data sets, along with new and archival VLA radio imaging and polarimetry, a re-analysis of numerous HST broadband data sets from the near infrared to the far ultraviolet, and a Chandra ACIS-S observation. We investigate similarities and differences between optical and radio polarimetry, in both degree of polarization and projected magnetic field direction. We also examine the broadband spectral energy distribution of both the nucleus and jet of 3C 264, from the radio through the X-rays. From this we place constraints on the physics of the 3C 264 system, the jet and its dynamics. We find significant curvature of the spectrum from the near-IR to ultraviolet, and synchrotron breaks steeper than 0.5, a situation also encountered in the jet of M87. This likely indicates velocity and/or magnetic field gradients and more efficient particle acceleration localized in the faster/higher magnetic field parts of the flow. The magnetic field structure of the 3C 264 jet is remarkably smooth; however, we do find complex magnetic field structure that is correlated with changes in the optical spectrum. We find that the X-ray emission is due to the synchrotron process; we model the jet spectrum and discuss mechanisms for accelerating particles to the needed energies, together with implications for the orientation of the jet under a possible spine-sheath model. △ Less

Submitted 9 November, 2009; originally announced November 2009.

Comments: 18 pages, 14 figures, 1 table; Ap J, in press (Jan 1 2010 issue)

Journal ref: Astrophys.J.708:171-187,2010

Polarimetry and the High-Energy Emission Mechanisms in Quasar Jets

Authors: M. Cara, E. S. Perlman, Y. Uchiyama, S. Jester, M. Georganopoulos, C. C. Cheung, R. M. Sambruna, W. B. Sparks, A. Martel, C. P. O'Dea, S. A. Baum, D. Axon, M. Begelman, D. M. Worrall, M. Birkinshaw, C. M. Urry, P. Coppi, L. Stawarz

Abstract: The emission mechanisms in extragalactic jets include synchrotron and various inverse-Compton processes. At low (radio through infrared) energies, it is widely agreed that synchrotron emission dominates in both low-power (FR I) and high-power (FR II and quasar) jets, because of the power-law nature of the spectra observed and high polarizations. However, at higher energies, the emission mechanis… ▽ More The emission mechanisms in extragalactic jets include synchrotron and various inverse-Compton processes. At low (radio through infrared) energies, it is widely agreed that synchrotron emission dominates in both low-power (FR I) and high-power (FR II and quasar) jets, because of the power-law nature of the spectra observed and high polarizations. However, at higher energies, the emission mechanism for high-power jets at kpc scales is hotly debated. Two mechanisms have been proposed: either inverse-Compton of cosmic microwave background photons or synchrotron emission from a second, high-energy population of electrons. Here we discuss optical polarimetry as a method for diagnosing the mechanism for the high-energy emission in quasar jets, as well as revealing the jet's three-dimensional energetic and magnetic field structure. We then discuss high-energy emission mechanisms for powerful jets in the light of the HST polarimetry of PKS 1136-135. △ Less

Submitted 1 September, 2009; originally announced September 2009.

Comments: 4 pages, 1 figure. To appear in proceedings of "The Monster's Fiery Breath: Feedback in galaxies, groups, and clusters" meeting, June 1-5, 2009 held in Madison, WI, USA

Principal Component Analysis of the Time- and Position-Dependent Point Spread Function of the Advanced Camera for Surveys

Authors: M. J. Jee, J. P. Blakeslee, M. Sirianni, A. R. Martel, R. L. White, H. C. Ford

Abstract: We describe the time- and position-dependent point spread function (PSF) variation of the Wide Field Channel (WFC) of the Advanced Camera for Surveys (ACS) with the principal component analysis (PCA) technique. The time-dependent change is caused by the temporal variation of the $HST$ focus whereas the position-dependent PSF variation in ACS/WFC at a given focus is mainly the result of changes i… ▽ More We describe the time- and position-dependent point spread function (PSF) variation of the Wide Field Channel (WFC) of the Advanced Camera for Surveys (ACS) with the principal component analysis (PCA) technique. The time-dependent change is caused by the temporal variation of the $HST$ focus whereas the position-dependent PSF variation in ACS/WFC at a given focus is mainly the result of changes in aberrations and charge diffusion across the detector, which appear as position-dependent changes in elongation of the astigmatic core and blurring of the PSF, respectively. Using >400 archival images of star cluster fields, we construct a ACS PSF library covering diverse environments of the $HST$ observations (e.g., focus values). We find that interpolation of a small number ($\sim20$) of principal components or ``eigen-PSFs'' per exposure can robustly reproduce the observed variation of the ellipticity and size of the PSF. Our primary interest in this investigation is the application of this PSF library to precision weak-lensing analyses, where accurate knowledge of the instrument's PSF is crucial. However, the high-fidelity of the model judged from the nice agreement with observed PSFs suggests that the model is potentially also useful in other applications such as crowded field stellar photometry, galaxy profile fitting, AGN studies, etc., which similarly demand a fair knowledge of the PSFs at objects' locations. Our PSF models, applicable to any WFC image rectified with the Lanczos3 kernel, are publicly available. △ Less

Submitted 29 October, 2007; originally announced October 2007.

Comments: Accepted to PASP. To appear in December issue. Figures are degraded to meet the size limit. High-resolution version can be downloaded at http://acs.pha.jhu.edu/~mkjee/acs_psf/acspsf.pdf

Hubble Space Telescope far-ultraviolet imaging of the jet in 3C273: a common emission component from optical to X-rays

Authors: Sebastian Jester, Klaus Meisenheimer, Andre' Martel, Eric Perlman, Bill Sparks

Abstract: We present far-ultraviolet (UV) observations at 150 nm of the jet of the quasar 3C 273 obtained with the Advanced Camera for Survey's Solar Blind Channel (ACS/SBC) on board the Hubble Space Telescope. While the jet morphology is very similar to that in the optical and near-ultraviolet, the spectral energy distributions (SEDs) of the jet's sub-regions show an upturn in nu f_nu at 150 nm compared… ▽ More We present far-ultraviolet (UV) observations at 150 nm of the jet of the quasar 3C 273 obtained with the Advanced Camera for Survey's Solar Blind Channel (ACS/SBC) on board the Hubble Space Telescope. While the jet morphology is very similar to that in the optical and near-ultraviolet, the spectral energy distributions (SEDs) of the jet's sub-regions show an upturn in nu f_nu at 150 nm compared to 300 nm everywhere in the jet. Moreover, the 150 nm flux is compatible with extrapolating the X-ray power-law down to the ultra-violet region. This constitutes strong support for a common origin of the jet's far-UV and X-ray emission. It implies that even a substantial fraction of the *visible light* in the X-ray brightest parts of the jet arises from the same spectral component as the X-rays, as had been suggested earlier based on Spitzer Space Telescope observations. We argue that the identification of this UV/X-ray component opens up the possibility to establish the synchrotron origin of the X-ray emission by optical polarimetry. △ Less

Submitted 18 June, 2007; originally announced June 2007.

Comments: MNRAS accepted; 8 pages, 4 .eps files

Report number: FERMILAB-PUB-07-071-CD

Journal ref: Mon.Not.Roy.Astron.Soc.380:828-834,2007

Discovery of a Ringlike Dark Matter Structure in the Core of the Galaxy Cluster Cl 0024+17

Authors: M. J. Jee, H. C. Ford, G. D. Illingworth, R. L. White, T. J. Broadhurst, D. A. Coe, G. R. Meurer, A. Van Der Wel, N. Benitez, J. P. Blakeslee, R. J. Bouwens, L. D. Bradley, R. Demarco, N. L. Homeier, A. R. Martel, S. Mei

Abstract: We present a comprehensive mass reconstruction of the rich galaxy cluster Cl 0024+17 at z~0.4 from ACS data, unifying both strong- and weak-lensing constraints. The weak-lensing signal from a dense distribution of background galaxies (~120 per square arcmin) across the cluster enables the derivation of a high-resolution parameter-free mass map. The strongly-lensed objects tightly constrain the m… ▽ More We present a comprehensive mass reconstruction of the rich galaxy cluster Cl 0024+17 at z~0.4 from ACS data, unifying both strong- and weak-lensing constraints. The weak-lensing signal from a dense distribution of background galaxies (~120 per square arcmin) across the cluster enables the derivation of a high-resolution parameter-free mass map. The strongly-lensed objects tightly constrain the mass structure of the cluster inner region on an absolute scale, breaking the mass-sheet degeneracy. The mass reconstruction of Cl 0024+17 obtained in such a way is remarkable. It reveals a ringlike dark matter substructure at r~75" surrounding a soft, dense core at r~50". We interpret this peculiar sub-structure as the result of a high-speed line-of-sight collision of two massive clusters 1-2 Gyr ago. Such an event is also indicated by the cluster velocity distribution. Our numerical simulation with purely collisionless particles demonstrates that such density ripples can arise by radially expanding, decelerating particles that originally comprised the pre-collision cores. Cl 0024+17 can be likened to the bullet cluster 1E0657-56, but viewed $along$ the collision axis at a much later epoch. In addition, we show that the long-standing mass discrepancy for Cl 0024+17 between X-ray and lensing can be resolved by treating the cluster X-ray emission as coming from a superposition of two X-ray systems. The cluster's unusual X-ray surface brightness profile that requires a two isothermal sphere description supports this hypothesis. △ Less

Submitted 15 May, 2007; originally announced May 2007.

Comments: To appear in the June 1 issue of The Astrophysical Journal

Journal ref: Astrophys.J.661:728-749,2007

A Robust Test of Evolution near the Tip of the Red Giant Branch and Missing Giants in NGC 2808

Authors: Eric L. Sandquist, André R. Martel

Abstract: We describe a new method for robustly testing theoretical predictions of red giant evolution near the tip of the giant branch. When theoretical cumulative luminosity functions are shifted to align the tip in I-band and normalized at a luminosity level slightly brighter than the red giant bump, virtually all dependence on age and composition (heavy elements and helium abundance) is eliminated. Wh… ▽ More We describe a new method for robustly testing theoretical predictions of red giant evolution near the tip of the giant branch. When theoretical cumulative luminosity functions are shifted to align the tip in I-band and normalized at a luminosity level slightly brighter than the red giant bump, virtually all dependence on age and composition (heavy elements and helium abundance) is eliminated. While significant comparisons with observations require large samples of giant stars, such samples are available for some of the most massive Milky Way globular clusters. We present comparisons with the clusters NGC 2808 and M5, and find that NGC 2808 has a deficiency of bright giants (with a probability of less than about 3% that a more extreme distribution of giant stars would have happened by chance). We discuss the possibilities that underestimated neutrino losses or strong mass loss could be responsible for the deficit of giants. While we cannot rule out the neutrino hypothesis, it cannot explain the apparent agreement between the M5 observations and models. On the other hand, strong mass loss provides a potential link between the giant star observations and NGC 2808's unusually blue horizontal branch. If the mass loss hypothesis is true, there is likely a significant population of He white dwarfs that could be uncovered with slightly deeper UV observations of the cluster. △ Less

Submitted 8 November, 2006; originally announced November 2006.

Comments: 6 pages, 2 figures, uses emulateapj, accepted for ApJ Letters

Journal ref: Astrophys.J.654:L65-L68,2006

Optical Polarimetry of the Jets of Nearby Radio Galaxies: I. The Data

Authors: Eric S. Perlman, C. A. Padgett, Markos Georganopoulos, William B. Sparks, John A. Biretta, Christopher P. O'Dea, Stefi A. Baum, Mark Birkinshaw, D. M. Worrall, Fred Dulwich, Sebastian Jester, Andre Martel, Alessandro Capetti, J. Patrick Leahy

Abstract: We present an overview of new HST imaging polarimetry of six nearby radio galaxies with optical jets. These observations triple the number of extragalactic jets with subarcsecond-resolution optical polarimetry. We discuss the polarization characteristics and optical morphology of each jet. We find evidence of high optical polarization, averaging 20%, but reaching upwards of $\sim 50%$ in some ob… ▽ More We present an overview of new HST imaging polarimetry of six nearby radio galaxies with optical jets. These observations triple the number of extragalactic jets with subarcsecond-resolution optical polarimetry. We discuss the polarization characteristics and optical morphology of each jet. We find evidence of high optical polarization, averaging 20%, but reaching upwards of $\sim 50%$ in some objects, confirming that the optical emission is synchrotron, and that the components of the magnetic fields perpendicular to the line of sight are well ordered. We find a wide range of polarization morphologies, with each jet having a somewhat different relationship between total intensity and polarized flux and the polarization position angle. We find two trends in all of these jets. First, jet ``edges'' are very often associated with high fractional optical polarizations, as also found in earlier radio observations of these and other radio jets. In these regions, the magnetic field vectors appear to track the jet direction, even at bends, where we see particularly high fractional polarizations. This indicates a strong link between the local magnetic field and jet dynamics. Second, optical flux maximum regions are usually well separated from maxima in fractional polarization and often are associated with polarization minima. This trend is not found in radio data and was found in our optical polarimetry of M87 with HST. However, unlike in M87, we do not find a general trend for near-90$^\circ$ rotations in the optical polarization vectors near flux maxima. We discuss possibilities for interpreting these trends, as well as implications for jet dynamics, magnetic field structure and particle acceleration. △ Less

Submitted 6 June, 2006; originally announced June 2006.

Comments: 32 pages, 8 figures, 1 reduced. ApJ, in press

Journal ref: Astrophys.J.651:735-748,2006

HST/ACS Multiband Coronagraphic Imaging of the Debris Disk around Beta Pictoris

Authors: D. A. Golimowski, D. R. Ardila, J. E. Krist, M. Clampin, H. C. Ford, G. D. Illingworth, F. Bartko, N. Benitez, J. P. Blakeslee, R. J. Bouwens, L. D. Bradley, T. J. Broadhurst, R. A. Brown, C. J. Burrows, E. S. Cheng, N. J. G. Cross, R. Demarco, P. D. Feldman, M. Franx, T. Goto, C. Gronwall, G. F. Hartig, B. P. Holden, N. L. Homeier, L. Infante , et al. (18 additional authors not shown)

Abstract: (Abridged.) We present F435W (B), F606W (Broad V), and F814W (Broad I) coronagraphic images of the debris disk around Beta Pictoris obtained with HST's Advanced Camera for Surveys. We confirm that the previously reported warp in the inner disk is a distinct secondary disk inclined by ~5 deg from the main disk. The main disk's northeast extension is linear from 80 to 250 AU, but the southwest ext… ▽ More (Abridged.) We present F435W (B), F606W (Broad V), and F814W (Broad I) coronagraphic images of the debris disk around Beta Pictoris obtained with HST's Advanced Camera for Surveys. We confirm that the previously reported warp in the inner disk is a distinct secondary disk inclined by ~5 deg from the main disk. The main disk's northeast extension is linear from 80 to 250 AU, but the southwest extension is distinctly bowed with an amplitude of ~1 AU over the same region. Both extensions of the secondary disk appear linear, but not collinear, from 80 to 150 AU. Within ~120 AU of the star, the main disk is ~50% thinner than previously reported. The surface-brightness profiles along the spine of the main disk are fitted with four distinct radial power laws between 40 and 250 AU, while those of the secondary disk between 80 and 150 AU are fitted with single power laws. These discrepancies suggest that the two disks have different grain compositions or size distributions. The F606W/F435W and F814W/F435W flux ratios of the composite disk are nonuniform and asymmetric about both projected axes of the disk. Within ~120 AU, the m_F435W-m_F606W and m_F435W-m_F814W colors along the spine of the main disk are ~10% and ~20% redder, respectively, than those of Beta Pic. These colors increasingly redden beyond ~120 AU, becoming 25% and 40% redder, respectively, than the star at 250 AU. We compare the observed red colors within ~120 AU with the simulated colors of non-icy grains having a radial number density ~r^-3 and different compositions, porosities, and minimum grain sizes. The observed colors are consistent with those of compact or moderately porous grains of astronomical silicate and/or graphite with sizes >0.15-0.20 um, but the colors are inconsistent with the blue colors expected from grains with porosities >90%. △ Less

Submitted 1 March, 2006; v1 submitted 13 February, 2006; originally announced February 2006.

Comments: 38 pages (including 21 figures and 4 tables) in EmulateApJ format, accepted for publication in The Astronomical Journal. Full-resolution figures and fully processed FITS images (with error maps) are available at http://acs.pha.jhu.edu/~dag/betapic Version 2: Added 4 references and some clarifying text. Basic facts and conclusions are unchanged

Evolution of the Color-Magnitude Relation in High-Redshift Clusters: Blue Early-Type Galaxies and Red Pairs in RDCS J0910+5422

Authors: S. Mei, J. P. Blakeslee, S. A. Stanford, B. P. Holden, P. Rosati, V. Strazzullo, N. Homeier, M. Postman, M. Franx, A. Rettura, H. Ford, G. D. Illingworth, S. Ettori, R. J. Bouwens, R. Demarco, A. R. Martel, M. Clampin, G. F. Hartig, P. Eisenhardt, the ACS IDT team

Abstract: The color-magnitude relation has been determined for the RDCS J0910+5422 cluster of galaxies at redshift z = 1.106. Cluster members were selected from HST ACS images, combined with ground--based near--IR imaging and optical spectroscopy. The observed early--type color--magnitude relation (CMR) in (i_775 -z_850) versus z_850 shows intrinsic scatters in color of 0.042 +/- 0.010 mag and 0.044 +/- 0… ▽ More The color-magnitude relation has been determined for the RDCS J0910+5422 cluster of galaxies at redshift z = 1.106. Cluster members were selected from HST ACS images, combined with ground--based near--IR imaging and optical spectroscopy. The observed early--type color--magnitude relation (CMR) in (i_775 -z_850) versus z_850 shows intrinsic scatters in color of 0.042 +/- 0.010 mag and 0.044 +/- 0.020 mag for ellipticals and S0s, respectively. From the scatter about the CMR, a mean luminosity--weighted age t > 3.3 Gyr (z > 3) is derived for the elliptical galaxies. Strikingly, the S0 galaxies in RDCS J0910+5422 are systematically bluer in (i_775 - z_850) by 0.07 +/- 0.02 mag, with respect to the ellipticals. The ellipticity distribution as a function of color indicates that the face-on S0s in this particular cluster have likely been classified as elliptical. Thus, if anything, the offset in color between the elliptical and S0 populations may be even more significant. The color offset between S0 and E corresponds to an age difference of ~1 Gyr, for a single-burst solar metallicity model. A solar metallicity model with an exponential decay in star formation will reproduce the offset for an age of 3.5 Gyr, i.e. the S0s have evolved gradually from star forming progenitors. The early--type population in this cluster appears to be still forming. The blue early-type disk galaxies in RDCS J0910+5422 likely represent the direct progenitors of the more evolved S0s that follow the same red sequence as ellipticals in other clusters. Thirteen red galaxy pairs are observed and the galaxies associated in pairs constitute ~40% of the CMR galaxies in this cluster. △ Less

Submitted 16 January, 2006; originally announced January 2006.

Comments: ApJ, in press

Journal ref: Astrophys.J. 639 (2006) 81-94

Lyman break galaxies, Lya emitters and a radio galaxy in a protocluster at z=4.1

Authors: Roderik A. Overzier, R. J. Bouwens, N. J. G. Cross, B. Venemans, G. K. Miley, A. W. Zirm, N. Benitez, J. P. Blakeslee, D. Coe, R. Demarco, H. Ford, N. Homeier, G. D. Illingworth, J. D. Kurk, A. Martel, S. Mei, I. Oliveira, H. J. A. Rottgering, Z. Tsvetanov, W. Zheng

Abstract: We present deep HST/ACS observations in g,r,i,z towards the z=4.1 radio galaxy TN J1338-1942 and its overdensity of >30 spectroscopically confirmed Lya emitters (LAEs). We select 66 g-band dropouts to z=27, 6 of which are also a LAE. Although our color-color selection results in a relatively broad redshift range centered on z=4.1, the field of TN J1338-1942 is richer than the average field at th… ▽ More We present deep HST/ACS observations in g,r,i,z towards the z=4.1 radio galaxy TN J1338-1942 and its overdensity of >30 spectroscopically confirmed Lya emitters (LAEs). We select 66 g-band dropouts to z=27, 6 of which are also a LAE. Although our color-color selection results in a relatively broad redshift range centered on z=4.1, the field of TN J1338-1942 is richer than the average field at the >5 sigma significance, based on a comparison with GOODS. The angular distribution is filamentary with about half of the objects clustered near the radio galaxy, and a small, excess signal (2 sigma) in the projected pair counts at separations of <10" is interpreted as being due to physical pairs. The LAEs are young (a few x 10^7 yr), small (<r_50> = 0.13") galaxies, and we derive a mean stellar mass of ~10^8-9 Msun based on a stacked K-band image. We determine star formation rates, sizes, morphologies, and color-magnitude relations of the g-dropouts and find no evidence for a difference between galaxies near TN J1338-1942 and in the field. We conclude that environmental trends as observed in clusters at much lower redshift are either not yet present, or are washed out by the relatively broad selection in redshift. The large galaxy overdensity, its corresponding mass overdensity and the sub-clustering at the approximate redshift of TN J1338-1942 suggest the assemblage of a >10^14 Msun structure, confirming that it is possible to find and study cluster progenitors in the linear regime at z>4. △ Less

Submitted 10 October, 2007; v1 submitted 10 January, 2006; originally announced January 2006.

Comments: Accepted for publication in ApJ (33 pages, 19 figures, emulateapj). v3 includes updated methods, discussion and referencing

An Overdensity of Galaxies near the Most Distant Radio-Loud Quasar

Authors: W. Zheng, R. Overzier, R. J. Bouwens, R. L. White, H. C. Ford, N. Benitez, J. P. Blakeslee, L. D. Bradley, M. K. Jee, A. R. Martel, S. Mei, A. W. Zirm, G. D. Illingworth, M. Clampin, G. F. Hartig, D. R. Ardila, F. Bartko, T. J. Broadhurst, R. A. Brown, C. J. Burrows, E. S. Cheng, N. J. G. Cross, R. Demarco, P. D. Feldman, M. Franx , et al. (19 additional authors not shown)

Abstract: A five square arcminute region around the luminous radio-loud quasar SDSS J0836+0054 (z=5.8) hosts a wealth of associated galaxies, characterized by very red (1.3 < i_775 - z_{850} < 2.0) color. The surface density of these z~5.8 candidates is approximately six times higher than the number expected from deep ACS fields. This is one of the highest galaxy overdensities at high redshifts, which may… ▽ More A five square arcminute region around the luminous radio-loud quasar SDSS J0836+0054 (z=5.8) hosts a wealth of associated galaxies, characterized by very red (1.3 < i_775 - z_{850} < 2.0) color. The surface density of these z~5.8 candidates is approximately six times higher than the number expected from deep ACS fields. This is one of the highest galaxy overdensities at high redshifts, which may develop into a group or cluster. We also find evidence for a substructure associated with one of the candidates. It has two very faint companion objects within two arcseconds, which are likely to merge. The finding supports the results of a recent simulation that luminous quasars at high redshifts lie on the most prominent dark-matter filaments and are surrounded by many fainter galaxies. The quasar activity from these regions may signal the buildup of a massive system. △ Less

Submitted 25 November, 2005; originally announced November 2005.

Comments: Four figures. The Astrophysical Journal: in press

Journal ref: Astrophys.J.640:574-578,2006

The Morphological Demographics of galaxies in the ACS Hubble Ultra Deep Parallel Fields

Authors: Felipe Menanteau, Holland C. Ford, Veronica Motta, Narciso Benitez, Andre R. Martel, John P. Blakeslee, Leopoldo Infante

Abstract: We present a morphological analysis of distant field galaxies using the deep ACS images from the public parallel NICMOS observations of the Hubble Ultra Deep Field obtained in the F435W (B), F606W (V), F775W (i) and F850LP (z) filters. We morphologically segregate galaxies using a combination of visual classification and objective machine based selection. We use the Asymmetry (A) and Central Con… ▽ More We present a morphological analysis of distant field galaxies using the deep ACS images from the public parallel NICMOS observations of the Hubble Ultra Deep Field obtained in the F435W (B), F606W (V), F775W (i) and F850LP (z) filters. We morphologically segregate galaxies using a combination of visual classification and objective machine based selection. We use the Asymmetry (A) and Central Concentration (C) parameters to characterize galaxies up to z_AB<25mag. We take advantage of the multicolor dataset and estimate redshifts for our sample using the Bayesian photometric redshift (BPZ) which enables us to investigate the evolution of their morphological demographics with redshift. Using a template fitting model and a maximum likelihood approach, we compute the star-formation rate (SFR) for galaxies up to z~1.3 and its contributions from different morphological types. We report that spirals are the main providers to the total SFR. The E/S0s contribution flattens out at z~1 while the Irr/Pec populations continuously rise to match the spirals contribution at z~1.0. We use the i-z and V-i color-magnitude diagrams to constrain the galaxies' formation histories and find that E/S0s show both a population of luminous red galaxies in place at z~1.2 and a bluer and fainter population resembling those of Irr/Pec at similar redshifts. △ Less

Submitted 26 September, 2005; originally announced September 2005.

Comments: 7 pages, 6 figures. Accepted for publication in the Astronomical Journal. Full resolution paper at: http://acs.pha.jhu.edu/~felipe/e-prints/

Journal ref: Astron.J.131:208-215,2006