-
Leveraging Human-Machine Interactions for Computer Vision Dataset Quality Enhancement
Authors:
Esla Timothy Anzaku,
Hyesoo Hong,
**-Woo Park,
Wonjun Yang,
Kangmin Kim,
JongBum Won,
Deshika Vinoshani Kumari Herath,
Arnout Van Messem,
Wesley De Neve
Abstract:
Large-scale datasets for single-label multi-class classification, such as \emph{ImageNet-1k}, have been instrumental in advancing deep learning and computer vision. However, a critical and often understudied aspect is the comprehensive quality assessment of these datasets, especially regarding potential multi-label annotation errors. In this paper, we introduce a lightweight, user-friendly, and sc…
▽ More
Large-scale datasets for single-label multi-class classification, such as \emph{ImageNet-1k}, have been instrumental in advancing deep learning and computer vision. However, a critical and often understudied aspect is the comprehensive quality assessment of these datasets, especially regarding potential multi-label annotation errors. In this paper, we introduce a lightweight, user-friendly, and scalable framework that synergizes human and machine intelligence for efficient dataset validation and quality enhancement. We term this novel framework \emph{Multilabelfy}. Central to Multilabelfy is an adaptable web-based platform that systematically guides annotators through the re-evaluation process, effectively leveraging human-machine interactions to enhance dataset quality. By using Multilabelfy on the ImageNetV2 dataset, we found that approximately $47.88\%$ of the images contained at least two labels, underscoring the need for more rigorous assessments of such influential datasets. Furthermore, our analysis showed a negative correlation between the number of potential labels per image and model top-1 accuracy, illuminating a crucial factor in model evaluation and selection. Our open-source framework, Multilabelfy, offers a convenient, lightweight solution for dataset enhancement, emphasizing multi-label proportions. This study tackles major challenges in dataset integrity and provides key insights into model performance evaluation. Moreover, it underscores the advantages of integrating human expertise with machine capabilities to produce more robust models and trustworthy data development. The source code for Multilabelfy will be available at https://github.com/esla/Multilabelfy.
\keywords{Computer Vision \and Dataset Quality Enhancement \and Dataset Validation \and Human-Computer Interaction \and Multi-label Annotation.}
△ Less
Submitted 31 January, 2024;
originally announced January 2024.
-
Deep Optical Coding Design in Computational Imaging
Authors:
Henry Arguello,
Jorge Bacca,
Hasindu Kariyawasam,
Edwin Vargas,
Miguel Marquez,
Ramith Hettiarachchi,
Hans Garcia,
Kithmini Herath,
Udith Haputhanthri,
Balpreet Singh Ahluwalia,
Peter So,
Dushan N. Wadduwage,
Chamira U. S. Edussooriya
Abstract:
Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches re…
▽ More
Computational optical imaging (COI) systems leverage optical coding elements (CE) in their setups to encode a high-dimensional scene in a single or multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set the distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical encoder and computational decoder. Specifically, by modeling the COI measurements through a fully differentiable image formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the computational decoder can be optimized in an end-to-end (E2E) manner. Moreover, by optimizing just CEs in the same framework, inference tasks can be performed from pure optics. This work surveys the recent advances on CE data-driven design and provides guidelines on how to parametrize different optical elements to include them in the E2E framework. Since the E2E framework can handle different inference applications by changing the loss function and the DNN, we present low-level tasks such as spectral imaging reconstruction or high-level tasks such as pose estimation with privacy preserving enhanced by using optimal task-based optical architectures. Finally, we illustrate classification and 3D object recognition applications performed at the speed of the light using all-optics DNN.
△ Less
Submitted 17 August, 2022; v1 submitted 27 June, 2022;
originally announced July 2022.
-
From Hours to Seconds: Towards 100x Faster Quantitative Phase Imaging via Differentiable Microscopy
Authors:
Udith Haputhanthri,
Kithmini Herath,
Ramith Hettiarachchi,
Hasindu Kariyawasam,
Azeem Ahmad,
Balpreet S. Ahluwalia,
Chamira U. S. Edussooriya,
Dushan N. Wadduwage
Abstract:
With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. Despite significant advances in fast multiplexed imaging sensors and deep-learning-based inverse solvers, the throughput of QPM is currently limited by the speed of electronic hardware. Complementarily, to improve throughput further, here we propose to acqui…
▽ More
With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. Despite significant advances in fast multiplexed imaging sensors and deep-learning-based inverse solvers, the throughput of QPM is currently limited by the speed of electronic hardware. Complementarily, to improve throughput further, here we propose to acquire images in a compressed form such that more information can be transferred beyond the existing electronic hardware bottleneck. To this end, we present a learnable optical compression-decompression framework that learns content-specific features. The proposed differentiable quantitative phase microscopy ($\partial μ$) first uses learnable optical feature extractors as image compressors. The intensity representation produced by these networks is then captured by the imaging sensor. Finally, a reconstruction network running on electronic hardware decompresses the QPM images. In numerical experiments, the proposed system achieves compression of $\times$ 64 while maintaining the SSIM of $\sim 0.90$ and PSNR of $\sim 30$ dB on cells. The results demonstrated by our experiments open up a new pathway for achieving end-to-end optimized (i.e., optics and electronic) compact QPM systems that may provide unprecedented throughput improvements.
△ Less
Submitted 9 October, 2023; v1 submitted 23 May, 2022;
originally announced May 2022.
-
Differentiable Microscopy Designs an All Optical Phase Retrieval Microscope
Authors:
Kithmini Herath,
Udith Haputhanthri,
Ramith Hettiarachchi,
Hasindu Kariyawasam,
Raja N. Ahmad,
Azeem Ahmad,
Balpreet S. Ahluwalia,
Chamira U. S. Edussooriya,
Dushan N. Wadduwage
Abstract:
Since the late 16th century, scientists have continuously innovated and developed new microscope types for various applications. Creating a new architecture from the ground up requires substantial scientific expertise and creativity, often spanning years or even decades. In this study, we propose an alternative approach called "Differentiable Microscopy," which introduces a top-down design paradig…
▽ More
Since the late 16th century, scientists have continuously innovated and developed new microscope types for various applications. Creating a new architecture from the ground up requires substantial scientific expertise and creativity, often spanning years or even decades. In this study, we propose an alternative approach called "Differentiable Microscopy," which introduces a top-down design paradigm for optical microscopes. Using all-optical phase retrieval as an illustrative example, we demonstrate the effectiveness of data-driven microscopy design through $\partialμ$. Furthermore, we conduct comprehensive comparisons with competing methods, showcasing the consistent superiority of our learned designs across multiple datasets, including biological samples. To substantiate our ideas, we experimentally validate the functionality of one of the learned designs, providing a proof of concept. The proposed differentiable microscopy framework supplements the creative process of designing new optical systems and would perhaps lead to unconventional but better optical designs.
△ Less
Submitted 24 August, 2023; v1 submitted 28 March, 2022;
originally announced March 2022.
-
A Novel Transfer Learning-Based Approach for Screening Pre-existing Heart Diseases Using Synchronized ECG Signals and Heart Sounds
Authors:
Ramith Hettiarachchi,
Udith Haputhanthri,
Kithmini Herath,
Hasindu Kariyawasam,
Shehan Munasinghe,
Kithmin Wickramasinghe,
Duminda Samarasinghe,
Anjula De Silva,
Chamira U. S. Edussooriya
Abstract:
Diagnosing pre-existing heart diseases early in life is important as it helps prevent complications such as pulmonary hypertension, heart rhythm problems, blood clots, heart failure and sudden cardiac arrest. To identify such diseases, phonocardiogram (PCG) and electrocardiogram (ECG) waveforms convey important information. Therefore, effectively using these two modalities of data has the potentia…
▽ More
Diagnosing pre-existing heart diseases early in life is important as it helps prevent complications such as pulmonary hypertension, heart rhythm problems, blood clots, heart failure and sudden cardiac arrest. To identify such diseases, phonocardiogram (PCG) and electrocardiogram (ECG) waveforms convey important information. Therefore, effectively using these two modalities of data has the potential to improve the disease screening process. We evaluate this hypothesis on a subset of the PhysioNet Challenge 2016 Dataset which contains simultaneously acquired PCG and ECG recordings. Our novel Dual-Convolutional Neural Network based approach uses transfer learning to tackle the problem of having limited amounts of simultaneous PCG and ECG data that is publicly available, while having the potential to adapt to larger datasets. In addition, we introduce two main evaluation frameworks named record-wise and sample-wise evaluation which leads to a rich performance evaluation for the transfer learning approach. Comparisons with methods which used single or dual modality data show that our method can lead to better performance. Furthermore, our results show that individually collected ECG or PCG waveforms are able to provide transferable features which could effectively help to make use of a limited number of synchronized PCG and ECG waveforms and still achieve significant classification performance.
△ Less
Submitted 14 February, 2021; v1 submitted 2 February, 2021;
originally announced February 2021.
