-
Efficient Personalization of Amplification in Hearing Aids via Multi-band Bayesian Machine Learning
Authors:
Aoxin Ni,
Edward Lobarinas,
Nasser Kehtarnavaz
Abstract:
Personalization of the amplification function of hearing aids has been shown to be of benefit to hearing aid users in previous studies. Several machine learning-based personalization approaches have been introduced in the literature. This paper presents a machine learning personalization approach with the advantage of being efficient in its training based on paired comparisons which makes it pract…
▽ More
Personalization of the amplification function of hearing aids has been shown to be of benefit to hearing aid users in previous studies. Several machine learning-based personalization approaches have been introduced in the literature. This paper presents a machine learning personalization approach with the advantage of being efficient in its training based on paired comparisons which makes it practical and field deployable. The training efficiency of this approach is the result of treating frequency bands independent of one another and by simultaneously carrying out Bayesian machine learning in each band across all of the frequency bands. Simulation results indicate that this approach leads to an estimated hearing preference function close to the true hearing preference function in fewer number of paired comparisons relative to the previous machine learning approaches. In addition, a clinical experiment conducted on eight subjects with hearing impairment indicate that this training efficient personalization approach provides personalized gain settings which are on average six times more preferred over the standard prescriptive gain settings.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
A Survey of Detection Methods for Die Attachment and Wire Bonding Defects in Integrated Circuit Manufacturing
Authors:
Lamia Alam,
Nasser Kehtarnavaz
Abstract:
Defect detection plays a vital role in the manufacturing process of integrated circuits (ICs). Die attachment and wire bonding are two steps of the manufacturing process that determine the power and signal transmission quality and dependability in an IC. This paper presents a survey or literature review of the methods used for detecting these defects based on different sensing modalities used incl…
▽ More
Defect detection plays a vital role in the manufacturing process of integrated circuits (ICs). Die attachment and wire bonding are two steps of the manufacturing process that determine the power and signal transmission quality and dependability in an IC. This paper presents a survey or literature review of the methods used for detecting these defects based on different sensing modalities used including optical, radiological, acoustical, and infrared thermography. A discussion of the detection methods used is provided in this survey. Both conventional and deep learning approaches for detecting die attachment and wire bonding defects are considered along with challenges and future research directions.
△ Less
Submitted 15 June, 2022; v1 submitted 1 June, 2022;
originally announced June 2022.
-
Probabilistic Neural Network to Quantify Uncertainty of Wind Power Estimation
Authors:
Farzad Karami,
Nasser Kehtarnavaz,
Mario Rotea
Abstract:
Each year a growing number of wind farms are being added to power grids to generate electricity. The power curve of a wind turbine, which exhibits the relationship between generated power and wind speed, plays a major role in assessing the performance of a wind farm. Neural networks have been used for power curve estimation. However, they do not produce a confidence measure for their output, unles…
▽ More
Each year a growing number of wind farms are being added to power grids to generate electricity. The power curve of a wind turbine, which exhibits the relationship between generated power and wind speed, plays a major role in assessing the performance of a wind farm. Neural networks have been used for power curve estimation. However, they do not produce a confidence measure for their output, unless computationally prohibitive Bayesian methods are used. In this paper, a probabilistic neural network with Monte Carlo dropout is considered to quantify the model (epistemic) uncertainty of the power curve estimation. This approach offers a minimal increase in computational complexity over deterministic approaches. Furthermore, by incorporating a probabilistic loss function, the noise or aleatoric uncertainty in the data is estimated. The developed network captures both model and noise uncertainty which is found to be useful tools in assessing performance. Also, the developed network is compared with existing ones across a public domain dataset showing superior performance in terms of prediction accuracy.
△ Less
Submitted 4 June, 2021;
originally announced June 2021.
-
Multitasking Deep Learning Model for Detection of Five Stages of Diabetic Retinopathy
Authors:
Sharmin Majumder,
Nasser Kehtarnavaz
Abstract:
This paper presents a multitask deep learning model to detect all the five stages of diabetic retinopathy (DR) consisting of no DR, mild DR, moderate DR, severe DR, and proliferate DR. This multitask model consists of one classification model and one regression model, each with its own loss function. Noting that a higher severity level normally occurs after a lower severity level, this dependency…
▽ More
This paper presents a multitask deep learning model to detect all the five stages of diabetic retinopathy (DR) consisting of no DR, mild DR, moderate DR, severe DR, and proliferate DR. This multitask model consists of one classification model and one regression model, each with its own loss function. Noting that a higher severity level normally occurs after a lower severity level, this dependency is taken into consideration by concatenating the classification and regression models. The regression model learns the inter-dependency between the stages and outputs a score corresponding to the severity level of DR generating a higher score for a higher severity level. After training the regression model and the classification model separately, the features extracted by these two models are concatenated and inputted to a multilayer perceptron network to classify the five stages of DR. A modified Squeeze Excitation Densely Connected deep neural network is developed to implement this multitasking approach. The developed multitask model is then used to detect the five stages of DR by examining the two large Kaggle datasets of APTOS and EyePACS. A multitasking transfer learning model based on Xception network is also developed to evaluate the proposed approach by classifying DR into five stages. It is found that the developed model achieves a weighted Kappa score of 0.90 and 0.88 for the APTOS and EyePACS datasets, respectively, higher than any existing methods for detection of the five stages of DR
△ Less
Submitted 6 March, 2021;
originally announced March 2021.
-
A Generative Model Method for Unsupervised Multispectral Image Fusion in Remote Sensing
Authors:
Arian Azarang,
Nasser Kehtarnavaz
Abstract:
This paper presents a generative model method for multispectral image fusion in remote sensing which is trained without supervision. This method eases the supervision of learning and it also considers a multi-objective loss function to achieve image fusion. The loss function incorporates both spectral and spatial distortions. Two discriminators are designed to minimize the spectral and spatial dis…
▽ More
This paper presents a generative model method for multispectral image fusion in remote sensing which is trained without supervision. This method eases the supervision of learning and it also considers a multi-objective loss function to achieve image fusion. The loss function incorporates both spectral and spatial distortions. Two discriminators are designed to minimize the spectral and spatial distortions of the generative output. Extensive experimentations are conducted using three public domain datasets. The comparison results across four reduced-resolution and three full-resolution objective metrics show the superiority of the developed method over several recently developed methods.
△ Less
Submitted 7 February, 2021;
originally announced February 2021.
-
Deep Learning-Based Detail Map Estimation for MultiSpectral Image Fusion in Remote Sensing
Authors:
Arian Azarang,
Nasser Kehtarnavaz
Abstract:
This paper presents a deep learning-based estimation of the intensity component of MultiSpectral bands by considering joint multiplication of the neighbouring spectral bands. This estimation is conducted as part of the component substitution approach for fusion of PANchromatic and MultiSpectral images in remote sensing. After computing the band dependent intensity components, a deep neural network…
▽ More
This paper presents a deep learning-based estimation of the intensity component of MultiSpectral bands by considering joint multiplication of the neighbouring spectral bands. This estimation is conducted as part of the component substitution approach for fusion of PANchromatic and MultiSpectral images in remote sensing. After computing the band dependent intensity components, a deep neural network is trained to learn the nonlinear relationship between a PAN image and its nonlinear intensity components. Low Resolution MultiSpectral bands are then fed into the trained network to obtain an estimate of High Resolution MultiSpectral bands. Experiments conducted on three datasets show that the developed deep learning-based estimation approach provides improved performance compared to the existing methods based on three objective metrics.
△ Less
Submitted 7 February, 2021;
originally announced February 2021.
-
Deep Learning-Based Human Pose Estimation: A Survey
Authors:
Ce Zheng,
Wenhan Wu,
Chen Chen,
Taojiannan Yang,
Sijie Zhu,
Ju Shen,
Nasser Kehtarnavaz,
Mubarak Shah
Abstract:
Human pose estimation aims to locate the human body parts and build human body representation (e.g., body skeleton) from input data such as images and videos. It has drawn increasing attention during the past decade and has been utilized in a wide range of applications including human-computer interaction, motion analysis, augmented reality, and virtual reality. Although the recently developed dee…
▽ More
Human pose estimation aims to locate the human body parts and build human body representation (e.g., body skeleton) from input data such as images and videos. It has drawn increasing attention during the past decade and has been utilized in a wide range of applications including human-computer interaction, motion analysis, augmented reality, and virtual reality. Although the recently developed deep learning-based solutions have achieved high performance in human pose estimation, there still remain challenges due to insufficient training data, depth ambiguities, and occlusion. The goal of this survey paper is to provide a comprehensive review of recent deep learning-based solutions for both 2D and 3D pose estimation via a systematic analysis and comparison of these solutions based on their input data and inference procedures. More than 250 research papers since 2014 are covered in this survey. Furthermore, 2D and 3D human pose estimation datasets and evaluation metrics are included. Quantitative performance comparisons of the reviewed methods on popular datasets are summarized and discussed. Finally, the challenges involved, applications, and future research directions are concluded. A regularly updated project page is provided: \url{https://github.com/zczcwh/DL-HPE}
△ Less
Submitted 3 July, 2023; v1 submitted 24 December, 2020;
originally announced December 2020.
-
Vision and Inertial Sensing Fusion for Human Action Recognition : A Review
Authors:
Sharmin Majumder,
Nasser Kehtarnavaz
Abstract:
Human action recognition is used in many applications such as video surveillance, human computer interaction, assistive living, and gaming. Many papers have appeared in the literature showing that the fusion of vision and inertial sensing improves recognition accuracies compared to the situations when each sensing modality is used individually. This paper provides a survey of the papers in which v…
▽ More
Human action recognition is used in many applications such as video surveillance, human computer interaction, assistive living, and gaming. Many papers have appeared in the literature showing that the fusion of vision and inertial sensing improves recognition accuracies compared to the situations when each sensing modality is used individually. This paper provides a survey of the papers in which vision and inertial sensing are used simultaneously within a fusion framework in order to perform human action recognition. The surveyed papers are categorized in terms of fusion approaches, features, classifiers, as well as multimodality datasets considered. Challenges as well as possible future directions are also stated for deploying the fusion of these two sensing modalities under realistic conditions.
△ Less
Submitted 1 August, 2020;
originally announced August 2020.
-
Personalization of Hearing Aid Compression by Human-In-Loop Deep Reinforcement Learning
Authors:
Nasim Alamdari,
Edward Lobarinas,
Nasser Kehtarnavaz
Abstract:
Existing prescriptive compression strategies used in hearing aid fitting are designed based on gain averages from a group of users which are not necessarily optimal for a specific user. Nearly half of hearing aid users prefer settings that differ from the commonly prescribed settings. This paper presents a human-in-loop deep reinforcement learning approach that personalizes hearing aid compression…
▽ More
Existing prescriptive compression strategies used in hearing aid fitting are designed based on gain averages from a group of users which are not necessarily optimal for a specific user. Nearly half of hearing aid users prefer settings that differ from the commonly prescribed settings. This paper presents a human-in-loop deep reinforcement learning approach that personalizes hearing aid compression to achieve improved hearing perception. The developed approach is designed to learn a specific user's hearing preferences in order to optimize compression based on the user's feedbacks. Both simulation and subject testing results are reported which demonstrate the effectiveness of the developed personalized compression.
△ Less
Submitted 30 June, 2020;
originally announced July 2020.
-
Automatic exposure selection and fusion for high-dynamic-range photography via smartphones
Authors:
Reza Pourreza,
Nasser Kehtarnavaz
Abstract:
High-dynamic-range (HDR) photography involves fusing a bracket of images taken at different exposure settings in order to compensate for the low dynamic range of digital cameras such as the ones used in smartphones. In this paper, a method for automatically selecting the exposure settings of such images is introduced based on the camera characteristic function. In addition, a new fusion method is…
▽ More
High-dynamic-range (HDR) photography involves fusing a bracket of images taken at different exposure settings in order to compensate for the low dynamic range of digital cameras such as the ones used in smartphones. In this paper, a method for automatically selecting the exposure settings of such images is introduced based on the camera characteristic function. In addition, a new fusion method is introduced based on an optimization formulation and weighted averaging. Both of these methods are implemented on a smartphone platform as an HDR app to demonstrate the practicality of the introduced methods. Comparison results with several existing methods are presented indicating the effectiveness as well as the computational efficiency of the introduced solution.
△ Less
Submitted 21 April, 2020;
originally announced April 2020.
-
A Review of Multi-Objective Deep Learning Speech Denoising Methods
Authors:
Arian Azarang,
Nasser Kehtarnavaz
Abstract:
This paper presents a review of multi-objective deep learning methods that have been introduced in the literature for speech denoising. After stating an overview of conventional, single objective deep learning, and hybrid or combined conventional and deep learning methods, a review of the mathematical framework of the multi-objective deep learning methods for speech denoising is provided. A repres…
▽ More
This paper presents a review of multi-objective deep learning methods that have been introduced in the literature for speech denoising. After stating an overview of conventional, single objective deep learning, and hybrid or combined conventional and deep learning methods, a review of the mathematical framework of the multi-objective deep learning methods for speech denoising is provided. A representative method from each speech denoising category, whose codes are publicly available, is selected and a comparison is carried out by considering the same public domain dataset and four widely used objective metrics. The comparison results indicate the effectiveness of the multi-objective method compared with the other methods, in particular when the signal-to-noise ratio is low. Possible future improvements that can be achieved are also mentioned.
△ Less
Submitted 26 March, 2020;
originally announced March 2020.
-
Image Segmentation Using Deep Learning: A Survey
Authors:
Shervin Minaee,
Yuri Boykov,
Fatih Porikli,
Antonio Plaza,
Nasser Kehtarnavaz,
Demetri Terzopoulos
Abstract:
Image segmentation is a key topic in image processing and computer vision with applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among many others. Various algorithms for image segmentation have been developed in the literature. Recently, due to the success of deep learning models in a wide range of v…
▽ More
Image segmentation is a key topic in image processing and computer vision with applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among many others. Various algorithms for image segmentation have been developed in the literature. Recently, due to the success of deep learning models in a wide range of vision applications, there has been a substantial amount of works aimed at develo** image segmentation approaches using deep learning models. In this survey, we provide a comprehensive review of the literature at the time of this writing, covering a broad spectrum of pioneering works for semantic and instance-level segmentation, including fully convolutional pixel-labeling networks, encoder-decoder architectures, multi-scale and pyramid based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the similarity, strengths and challenges of these deep learning models, examine the most widely used datasets, report performances, and discuss promising future research directions in this area.
△ Less
Submitted 14 November, 2020; v1 submitted 15 January, 2020;
originally announced January 2020.
-
Computationally Efficient Optic Nerve Head Detection in Retinal Fundus Images
Authors:
Reza Pourreza-Shahri,
Meysam Tavakoli,
Nasser Kehtarnavaz
Abstract:
This paper presents a computationally efficient method for the detection of optic nerve head in both color fundus and fluorescein angiography images. It involves a combination of Radon transformation and multi-overlap** windows within an optimization framework in order to achieve a robust detection in the presence of various structural, color, and intensity variations in such images. Three datab…
▽ More
This paper presents a computationally efficient method for the detection of optic nerve head in both color fundus and fluorescein angiography images. It involves a combination of Radon transformation and multi-overlap** windows within an optimization framework in order to achieve a robust detection in the presence of various structural, color, and intensity variations in such images. Three databases have been examined and it is shown that the introduced method provides high detection rates while achieving faster proceeding rates than the existing algorithms that possess comparable detection rates.
△ Less
Submitted 1 October, 2019; v1 submitted 4 September, 2019;
originally announced September 2019.
-
Improving Deep Speech Denoising by Noisy2Noisy Signal Map**
Authors:
Nasim Alamdari,
Arian Azarang,
Nasser Kehtarnavaz
Abstract:
Existing deep learning-based speech denoising approaches require clean speech signals to be available for training. This paper presents a deep learning-based approach to improve speech denoising in real-world audio environments by not requiring the availability of clean speech signals in a self-supervised manner. A fully convolutional neural network is trained by using two noisy realizations of th…
▽ More
Existing deep learning-based speech denoising approaches require clean speech signals to be available for training. This paper presents a deep learning-based approach to improve speech denoising in real-world audio environments by not requiring the availability of clean speech signals in a self-supervised manner. A fully convolutional neural network is trained by using two noisy realizations of the same speech signal, one used as the input and the other as the output of the network. Extensive experimentations are conducted to show the superiority of the developed deep speech denoising approach over the conventional supervised deep speech denoising approach based on four commonly used performance metrics and also based on actual field-testing outcomes.
△ Less
Submitted 21 February, 2020; v1 submitted 26 April, 2019;
originally announced April 2019.
-
Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps
Authors:
Abhishek Sehgal,
Nasser Kehtarnavaz
Abstract:
Deep learning solutions are being increasingly used in mobile applications. Although there are many open-source software tools for the development of deep learning solutions, there are no guidelines in one place in a unified manner for using these tools towards real-time deployment of these solutions on smartphones. From the variety of available deep learning tools, the most suited ones are used i…
▽ More
Deep learning solutions are being increasingly used in mobile applications. Although there are many open-source software tools for the development of deep learning solutions, there are no guidelines in one place in a unified manner for using these tools towards real-time deployment of these solutions on smartphones. From the variety of available deep learning tools, the most suited ones are used in this paper to enable real-time deployment of deep learning inference networks on smartphones. A uniform flow of implementation is devised for both Android and iOS smartphones. The advantage of using multi-threading to achieve or improve real-time throughputs is also showcased. A benchmarking framework consisting of accuracy, CPU/GPU consumption and real-time throughput is considered for validation purposes. The developed deployment approach allows deep learning models to be turned into real-time smartphone apps with ease based on publicly available deep learning and smartphone software tools. This approach is applied to six popular or representative convolutional neural network models and the validation results based on the benchmarking metrics are reported.
△ Less
Submitted 7 January, 2019;
originally announced January 2019.
