Showing 1–9 of 9 results for author: Heydari, M

Convolutional Neural Networks Towards Facial Skin Lesions Detection

Authors: Reza Sarshar, Mohammad Heydari, Elham Akhondzadeh Noughabi

Abstract: Facial analysis has emerged as a prominent area of research with diverse applications, including cosmetic surgery programs, the beauty industry, photography, and entertainment. Manipulating patient images often necessitates professional image processing software. This study contributes by providing a model that facilitates the detection of blemishes and skin lesions on facial images through a conv… ▽ More Facial analysis has emerged as a prominent area of research with diverse applications, including cosmetic surgery programs, the beauty industry, photography, and entertainment. Manipulating patient images often necessitates professional image processing software. This study contributes by providing a model that facilitates the detection of blemishes and skin lesions on facial images through a convolutional neural network and machine learning approach. The proposed method offers advantages such as simple architecture, speed and suitability for image processing while avoiding the complexities associated with traditional methods. The model comprises four main steps: area selection, scanning the chosen region, lesion diagnosis, and marking the identified lesion. Raw data for this research were collected from a reputable clinic in Tehran specializing in skincare and beauty services. The dataset includes administrative information, clinical data, and facial and profile images. A total of 2300 patient images were extracted from this raw data. A software tool was developed to crop and label lesions, with input from two treatment experts. In the lesion preparation phase, the selected area was standardized to 50 * 50 pixels. Subsequently, a convolutional neural network model was employed for lesion labeling. The classification model demonstrated high accuracy, with a measure of 0.98 for healthy skin and 0.97 for lesioned skin specificity. Internal validation involved performance indicators and cross-validation, while external validation compared the model's performance indicators with those of the transfer learning method using the Vgg16 deep network model. Compared to existing studies, the results of this research showcase the efficacy and desirability of the proposed model and methodology. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Comments: 6 pages, 11 figures

SingFake: Singing Voice Deepfake Detection

Authors: Yongyi Zang, You Zhang, Mojtaba Heydari, Zhiyao Duan

Abstract: The rise of singing voice synthesis presents critical challenges to artists and industry stakeholders over unauthorized voice usage. Unlike synthesized speech, synthesized singing voices are typically released in songs containing strong background music that may hide synthesis artifacts. Additionally, singing voices present different acoustic and linguistic characteristics from speech utterances.… ▽ More The rise of singing voice synthesis presents critical challenges to artists and industry stakeholders over unauthorized voice usage. Unlike synthesized speech, synthesized singing voices are typically released in songs containing strong background music that may hide synthesis artifacts. Additionally, singing voices present different acoustic and linguistic characteristics from speech utterances. These unique properties make singing voice deepfake detection a relevant but significantly different problem from synthetic speech detection. In this work, we propose the singing voice deepfake detection task. We first present SingFake, the first curated in-the-wild dataset consisting of 28.93 hours of bonafide and 29.40 hours of deepfake song clips in five languages from 40 singers. We provide a train/validation/test split where the test sets include various scenarios. We then use SingFake to evaluate four state-of-the-art speech countermeasure systems trained on speech utterances. We find these systems lag significantly behind their performance on speech test data. When trained on SingFake, either using separated vocal tracks or song mixtures, these systems show substantial improvement. However, our evaluations also identify challenges associated with unseen singers, communication codecs, languages, and musical contexts, calling for dedicated research into singing voice deepfake detection. The SingFake dataset and related resources are available at https://www.singfake.org/. △ Less

Submitted 21 January, 2024; v1 submitted 14 September, 2023; originally announced September 2023.

Comments: Accepted at ICASSP 2024

SingNet: A Real-time Singing Voice Beat and Downbeat Tracking System

Authors: Mojtaba Heydari, Ju-Chiang Wang, Zhiyao Duan

Abstract: Singing voice beat and downbeat tracking posses several applications in automatic music production, analysis and manipulation. Among them, some require real-time processing, such as live performance processing and auto-accompaniment for singing inputs. This task is challenging owing to the non-trivial rhythmic and harmonic patterns in singing signals. For real-time processing, it introduces furthe… ▽ More Singing voice beat and downbeat tracking posses several applications in automatic music production, analysis and manipulation. Among them, some require real-time processing, such as live performance processing and auto-accompaniment for singing inputs. This task is challenging owing to the non-trivial rhythmic and harmonic patterns in singing signals. For real-time processing, it introduces further constraints such as inaccessibility to future data and the impossibility to correct the previous results that are inconsistent with the latter ones. In this paper, we introduce the first system that tracks the beats and downbeats of singing voices in real-time. Specifically, we propose a novel dynamic particle filtering approach that incorporates offline historical data to correct the online inference by using a variable number of particles. We evaluate the performance on two datasets: GTZAN with the separated vocal tracks, and an in-house dataset with the original vocal stems. Experimental result demonstrates that our proposed approach outperforms the baseline by 3-5%. △ Less

Submitted 4 June, 2023; originally announced June 2023.

Comments: Accepted for 2023 International Conference on Acoustics, Speech, and Signal Processing (ICASSP-2023)

Singing Beat Tracking With Self-supervised Front-end and Linear Transformers

Authors: Mojtaba Heydari, Zhiyao Duan

Abstract: Tracking beats of singing voices without the presence of musical accompaniment can find many applications in music production, automatic song arrangement, and social media interaction. Its main challenge is the lack of strong rhythmic and harmonic patterns that are important for music rhythmic analysis in general. Even for human listeners, this can be a challenging task. As a result, existing musi… ▽ More Tracking beats of singing voices without the presence of musical accompaniment can find many applications in music production, automatic song arrangement, and social media interaction. Its main challenge is the lack of strong rhythmic and harmonic patterns that are important for music rhythmic analysis in general. Even for human listeners, this can be a challenging task. As a result, existing music beat tracking systems fail to deliver satisfactory performance on singing voices. In this paper, we propose singing beat tracking as a novel task, and propose the first approach to solving this task. Our approach leverages semantic information of singing voices by employing pre-trained self-supervised WavLM and DistilHuBERT speech representations as the front-end and uses a self-attention encoder layer to predict beats. To train and test the system, we obtain separated singing voices and their beat annotations using source separation and beat tracking on complete songs, followed by manual corrections. Experiments on the 741 separated vocal tracks of the GTZAN dataset show that the proposed system outperforms several state-of-the-art music beat tracking methods by a large margin in terms of beat tracking accuracy. Ablation studies also confirm the advantages of pre-trained self-supervised speech representations over generic spectral features. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Comments: 23rd International Society for Music Information Retrieval Conference (ISMIR 2022)

A Novel 1D State Space for Efficient Music Rhythmic Analysis

Authors: Mojtaba Heydari, Matthew McCallum, Andreas Ehmann, Zhiyao Duan

Abstract: Inferring music time structures has a broad range of applications in music production, processing and analysis. Scholars have proposed various methods to analyze different aspects of time structures, such as beat, downbeat, tempo and meter. Many state-of-the-art (SOFA) methods, however, are computationally expensive. This makes them inapplicable in real-world industrial settings where the scale of… ▽ More Inferring music time structures has a broad range of applications in music production, processing and analysis. Scholars have proposed various methods to analyze different aspects of time structures, such as beat, downbeat, tempo and meter. Many state-of-the-art (SOFA) methods, however, are computationally expensive. This makes them inapplicable in real-world industrial settings where the scale of the music collections can be millions. This paper proposes a new state space and a semi-Markov model for music time structure analysis. The proposed approach turns the commonly used 2D state spaces into a 1D model through a jump-back reward strategy. It reduces the state spaces size drastically. We then utilize the proposed method for causal, joint beat, downbeat, tempo, and meter tracking, and compare it against several previous methods. The proposed method delivers similar performance with the SOFA joint causal models with a much smaller state space and a more than 30 times speedup. △ Less

Submitted 20 February, 2022; v1 submitted 1 November, 2021; originally announced November 2021.

Comments: International Conference on Acoustics, Speech and Signal Processing (ICASSP), May. 2022

Learning Sparse Analytic Filters for Piano Transcription

Authors: Frank Cwitkowitz, Mojtaba Heydari, Zhiyao Duan

Abstract: In recent years, filterbank learning has become an increasingly popular strategy for various audio-related machine learning tasks. This is partly due to its ability to discover task-specific audio characteristics which can be leveraged in downstream processing. It is also a natural extension of the nearly ubiquitous deep learning methods employed to tackle a diverse array of audio applications. In… ▽ More In recent years, filterbank learning has become an increasingly popular strategy for various audio-related machine learning tasks. This is partly due to its ability to discover task-specific audio characteristics which can be leveraged in downstream processing. It is also a natural extension of the nearly ubiquitous deep learning methods employed to tackle a diverse array of audio applications. In this work, several variations of a frontend filterbank learning module are investigated for piano transcription, a challenging low-level music information retrieval task. We build upon a standard piano transcription model, modifying only the feature extraction stage. The filterbank module is designed such that its complex filters are unconstrained 1D convolutional kernels with long receptive fields. Additional variations employ the Hilbert transform to render the filters intrinsically analytic and apply variational dropout to promote filterbank sparsity. Transcription results are compared across all experiments, and we offer visualization and analysis of the filterbanks. △ Less

Submitted 10 November, 2022; v1 submitted 23 August, 2021; originally announced August 2021.

Comments: Sound and Music Computing Conference (SMC) 2022

BeatNet: CRNN and Particle Filtering for Online Joint Beat Downbeat and Meter Tracking

Authors: Mojtaba Heydari, Frank Cwitkowitz, Zhiyao Duan

Abstract: The online estimation of rhythmic information, such as beat positions, downbeat positions, and meter, is critical for many real-time music applications. Musical rhythm comprises complex hierarchical relationships across time, rendering its analysis intrinsically challenging and at times subjective. Furthermore, systems which attempt to estimate rhythmic information in real-time must be causal and… ▽ More The online estimation of rhythmic information, such as beat positions, downbeat positions, and meter, is critical for many real-time music applications. Musical rhythm comprises complex hierarchical relationships across time, rendering its analysis intrinsically challenging and at times subjective. Furthermore, systems which attempt to estimate rhythmic information in real-time must be causal and must produce estimates quickly and efficiently. In this work, we introduce an online system for joint beat, downbeat, and meter tracking, which utilizes causal convolutional and recurrent layers, followed by a pair of sequential Monte Carlo particle filters applied during inference. The proposed system does not need to be primed with a time signature in order to perform downbeat tracking, and is instead able to estimate meter and adjust the predictions over time. Additionally, we propose an information gate strategy to significantly decrease the computational cost of particle filtering during the inference step, making the system much faster than previous sampling-based methods. Experiments on the GTZAN dataset, which is unseen during training, show that the system outperforms various online beat and downbeat tracking systems and achieves comparable performance to a baseline offline joint method. △ Less

Submitted 8 August, 2021; originally announced August 2021.

Comments: 22nd International Society for Music Information Retrieval (ISMIR) Conference Paper, Fall 2021. 8 Pages (Total), 3 Figures, 2 Tables, 1 Algorithm

Don't look back: an online beat tracking method using RNN and enhanced particle filtering

Authors: Mojtaba Heydari, Zhiyao Duan

Abstract: Online beat tracking (OBT) has always been a challenging task. Due to the inaccessibility of future data and the need to make inference in real-time. We propose Do not Look back! (DLB), a novel approach optimized for efficiency when performing OBT. DLB feeds the activations of a unidirectional RNN into an enhanced Monte-Carlo localization model to infer beat positions. Most preexisting OBT methods… ▽ More Online beat tracking (OBT) has always been a challenging task. Due to the inaccessibility of future data and the need to make inference in real-time. We propose Do not Look back! (DLB), a novel approach optimized for efficiency when performing OBT. DLB feeds the activations of a unidirectional RNN into an enhanced Monte-Carlo localization model to infer beat positions. Most preexisting OBT methods either apply some offline approaches to a moving window containing past data to make predictions about future beat positions or must be primed with past data at startup to initialize. Meanwhile, our proposed method only uses activation of the current time frame to infer beat positions. As such, without waiting at the beginning to receive a chunk, it provides an immediate beat tracking response, which is critical for many OBT applications. DLB significantly improves beat tracking accuracy over state-of-the-art OBT methods, yielding a similar performance to offline methods. △ Less

Submitted 1 March, 2021; v1 submitted 4 November, 2020; originally announced November 2020.

Comments: IEEE International Conference on Acoustics, Speech and Signal Processing, (ICASSP 2021). (ACCEPTED)

Design and Fabrication of Novel Digital Transcranial Electrical Stimulator for Medical and Psychiatry Applications

Authors: HoseinAli Jafari, Mohammad Bagher Heydari, Niloofar Jafari, Hamid Mirhosseini

Abstract: In this article, we design a novel Transcranial Electrical Stimulator for medical applications, which is very cheap and can produce the desired signals very accurately. Our fabricated stimulator generates all current signals related to Transcranial Electrical Stimulation (TES) methods, i.e. Transcranial Direct Current Stimulation (tDCS), Transcranial Pulsed Current Stimulation (tPCS), Cranial Elec… ▽ More In this article, we design a novel Transcranial Electrical Stimulator for medical applications, which is very cheap and can produce the desired signals very accurately. Our fabricated stimulator generates all current signals related to Transcranial Electrical Stimulation (TES) methods, i.e. Transcranial Direct Current Stimulation (tDCS), Transcranial Pulsed Current Stimulation (tPCS), Cranial Electrotherapy Stimulation (CES), and Microcurrent Electrical Therapy (MET). The proposed device has been constructed of an advanced digital controller which makes it tunable. One of the major advantages of the device is its ability to generate Burst pulses. △ Less

Submitted 9 September, 2020; originally announced September 2020.