-
Recognizing Handwritten Mathematical Expressions of Vertical Addition and Subtraction
Authors:
Daniel Rosa,
Filipe R. Cordeiro,
Ruan Carvalho,
Everton Souza,
Sergio Chevtchenko,
Luiz Rodrigues,
Marcelo Marinho,
Thales Vieira,
Valmir Macario
Abstract:
Handwritten Mathematical Expression Recognition (HMER) is a challenging task with many educational applications. Recent methods for HMER have been developed for complex mathematical expressions in standard horizontal format. However, solutions for elementary mathematical expression, such as vertical addition and subtraction, have not been explored in the literature. This work proposes a new handwr…
▽ More
Handwritten Mathematical Expression Recognition (HMER) is a challenging task with many educational applications. Recent methods for HMER have been developed for complex mathematical expressions in standard horizontal format. However, solutions for elementary mathematical expression, such as vertical addition and subtraction, have not been explored in the literature. This work proposes a new handwritten elementary mathematical expression dataset composed of addition and subtraction expressions in a vertical format. We also extended the MNIST dataset to generate artificial images with this structure. Furthermore, we proposed a solution for offline HMER, able to recognize vertical addition and subtraction expressions. Our analysis evaluated the object detection algorithms YOLO v7, YOLO v8, YOLO-NAS, NanoDet and FCOS for identifying the mathematical symbols. We also proposed a transcription method to map the bounding boxes from the object detection stage to a mathematical expression in the LATEX markup sequence. Results show that our approach is efficient, achieving a high expression recognition rate. The code and dataset are available at https://github.com/Danielgol/HME-VAS
△ Less
Submitted 10 August, 2023;
originally announced August 2023.
-
Improving Mass Detection in Mammography Images: A Study of Weakly Supervised Learning and Class Activation Map Methods
Authors:
Vicente Sampaio,
Filipe R. Cordeiro
Abstract:
In recent years, weakly supervised models have aided in mass detection using mammography images, decreasing the need for pixel-level annotations. However, most existing models in the literature rely on Class Activation Maps (CAM) as the activation method, overlooking the potential benefits of exploring other activation techniques. This work presents a study that explores and compares different act…
▽ More
In recent years, weakly supervised models have aided in mass detection using mammography images, decreasing the need for pixel-level annotations. However, most existing models in the literature rely on Class Activation Maps (CAM) as the activation method, overlooking the potential benefits of exploring other activation techniques. This work presents a study that explores and compares different activation maps in conjunction with state-of-the-art methods for weakly supervised training in mammography images. Specifically, we investigate CAM, GradCAM, GradCAM++, XGradCAM, and LayerCAM methods within the framework of the GMIC model for mass detection in mammography images. The evaluation is conducted on the VinDr-Mammo dataset, utilizing the metrics Accuracy, True Positive Rate (TPR), False Negative Rate (FNR), and False Positive Per Image (FPPI). Results show that using different strategies of activation maps during training and test stages leads to an improvement of the model. With this strategy, we improve the results of the GMIC method, decreasing the FPPI value and increasing TPR.
△ Less
Submitted 7 August, 2023;
originally announced August 2023.
-
A Study on the Impact of Data Augmentation for Training Convolutional Neural Networks in the Presence of Noisy Labels
Authors:
Emeson Santana,
Gustavo Carneiro,
Filipe R. Cordeiro
Abstract:
Label noise is common in large real-world datasets, and its presence harms the training process of deep neural networks. Although several works have focused on the training strategies to address this problem, there are few studies that evaluate the impact of data augmentation as a design choice for training deep neural networks. In this work, we analyse the model robustness when using different da…
▽ More
Label noise is common in large real-world datasets, and its presence harms the training process of deep neural networks. Although several works have focused on the training strategies to address this problem, there are few studies that evaluate the impact of data augmentation as a design choice for training deep neural networks. In this work, we analyse the model robustness when using different data augmentations and their improvement on the training with the presence of noisy labels. We evaluate state-of-the-art and classical data augmentation strategies with different levels of synthetic noise for the datasets MNist, CIFAR-10, CIFAR-100, and the real-world dataset Clothing1M. We evaluate the methods using the accuracy metric. Results show that the appropriate selection of data augmentation can drastically improve the model robustness to label noise, increasing up to 177.84% of relative best test accuracy compared to the baseline with no augmentation, and an increase of up to 6% in absolute value with the state-of-the-art DivideMix training strategy.
△ Less
Submitted 7 August, 2023; v1 submitted 23 August, 2022;
originally announced August 2022.
-
PropMix: Hard Sample Filtering and Proportional MixUp for Learning with Noisy Labels
Authors:
Filipe R. Cordeiro,
Vasileios Belagiannis,
Ian Reid,
Gustavo Carneiro
Abstract:
The most competitive noisy label learning methods rely on an unsupervised classification of clean and noisy samples, where samples classified as noisy are re-labelled and "MixMatched" with the clean samples. These methods have two issues in large noise rate problems: 1) the noisy set is more likely to contain hard samples that are in-correctly re-labelled, and 2) the number of samples produced by…
▽ More
The most competitive noisy label learning methods rely on an unsupervised classification of clean and noisy samples, where samples classified as noisy are re-labelled and "MixMatched" with the clean samples. These methods have two issues in large noise rate problems: 1) the noisy set is more likely to contain hard samples that are in-correctly re-labelled, and 2) the number of samples produced by MixMatch tends to be reduced because it is constrained by the small clean set size. In this paper, we introduce the learning algorithm PropMix to handle the issues above. PropMix filters out hard noisy samples, with the goal of increasing the likelihood of correctly re-labelling the easy noisy samples. Also, PropMix places clean and re-labelled easy noisy samples in a training set that is augmented with MixUp, removing the clean set size constraint and including a large proportion of correctly re-labelled easy noisy samples. We also include self-supervised pre-training to improve robustness to high noisy label scenarios. Our experiments show that PropMix has state-of-the-art (SOTA) results on CIFAR-10/-100(with symmetric, asymmetric and semantic label noise), Red Mini-ImageNet (from the Controlled Noisy Web Labels), Clothing1M and WebVision. In severe label noise bench-marks, our results are substantially better than other methods. The code is available athttps://github.com/filipe-research/PropMix.
△ Less
Submitted 22 October, 2021;
originally announced October 2021.
-
ScanMix: Learning from Severe Label Noise via Semantic Clustering and Semi-Supervised Learning
Authors:
Ragav Sachdeva,
Filipe R Cordeiro,
Vasileios Belagiannis,
Ian Reid,
Gustavo Carneiro
Abstract:
We propose a new training algorithm, ScanMix, that explores semantic clustering and semi-supervised learning (SSL) to allow superior robustness to severe label noise and competitive robustness to non-severe label noise problems, in comparison to the state of the art (SOTA) methods. ScanMix is based on the expectation maximisation framework, where the E-step estimates the latent variable to cluster…
▽ More
We propose a new training algorithm, ScanMix, that explores semantic clustering and semi-supervised learning (SSL) to allow superior robustness to severe label noise and competitive robustness to non-severe label noise problems, in comparison to the state of the art (SOTA) methods. ScanMix is based on the expectation maximisation framework, where the E-step estimates the latent variable to cluster the training images based on their appearance and classification results, and the M-step optimises the SSL classification and learns effective feature representations via semantic clustering. We present a theoretical result that shows the correctness and convergence of ScanMix, and an empirical result that shows that ScanMix has SOTA results on CIFAR-10/-100 (with symmetric, asymmetric and semantic label noise), Red Mini-ImageNet (from the Controlled Noisy Web Labels), Clothing1M and WebVision. In all benchmarks with severe label noise, our results are competitive to the current SOTA.
△ Less
Submitted 16 October, 2022; v1 submitted 21 March, 2021;
originally announced March 2021.
-
LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environment
Authors:
Filipe R. Cordeiro,
Ragav Sachdeva,
Vasileios Belagiannis,
Ian Reid,
Gustavo Carneiro
Abstract:
Deep neural network models are robust to a limited amount of label noise, but their ability to memorise noisy labels in high noise rate problems is still an open issue. The most competitive noisy-label learning algorithms rely on a 2-stage process comprising an unsupervised learning to classify training samples as clean or noisy, followed by a semi-supervised learning that minimises the empirical…
▽ More
Deep neural network models are robust to a limited amount of label noise, but their ability to memorise noisy labels in high noise rate problems is still an open issue. The most competitive noisy-label learning algorithms rely on a 2-stage process comprising an unsupervised learning to classify training samples as clean or noisy, followed by a semi-supervised learning that minimises the empirical vicinal risk (EVR) using a labelled set formed by samples classified as clean, and an unlabelled set with samples classified as noisy. In this paper, we hypothesise that the generalisation of such 2-stage noisy-label learning methods depends on the precision of the unsupervised classifier and the size of the training set to minimise the EVR. We empirically validate these two hypotheses and propose the new 2-stage noisy-label training algorithm LongReMix. We test LongReMix on the noisy-label benchmarks CIFAR-10, CIFAR-100, WebVision, Clothing1M, and Food101-N. The results show that our LongReMix generalises better than competing approaches, particularly in high label noise problems. Furthermore, our approach achieves state-of-the-art performance in most datasets. The code is available at https://github.com/filipe-research/LongReMix.
△ Less
Submitted 4 September, 2022; v1 submitted 6 March, 2021;
originally announced March 2021.
-
Self-supervised Mean Teacher for Semi-supervised Chest X-ray Classification
Authors:
Fengbei Liu,
Yu Tian,
Filipe R. Cordeiro,
Vasileios Belagiannis,
Ian Reid,
Gustavo Carneiro
Abstract:
The training of deep learning models generally requires a large amount of annotated data for effective convergence and generalisation. However, obtaining high-quality annotations is a laboursome and expensive process due to the need of expert radiologists for the labelling task. The study of semi-supervised learning in medical image analysis is then of crucial importance given that it is much less…
▽ More
The training of deep learning models generally requires a large amount of annotated data for effective convergence and generalisation. However, obtaining high-quality annotations is a laboursome and expensive process due to the need of expert radiologists for the labelling task. The study of semi-supervised learning in medical image analysis is then of crucial importance given that it is much less expensive to obtain unlabelled images than to acquire images labelled by expert radiologists. Essentially, semi-supervised methods leverage large sets of unlabelled data to enable better training convergence and generalisation than using only the small set of labelled images. In this paper, we propose Self-supervised Mean Teacher for Semi-supervised (S$^2$MTS$^2$) learning that combines self-supervised mean-teacher pre-training with semi-supervised fine-tuning. The main innovation of S$^2$MTS$^2$ is the self-supervised mean-teacher pre-training based on the joint contrastive learning, which uses an infinite number of pairs of positive query and key features to improve the mean-teacher representation. The model is then fine-tuned using the exponential moving average teacher framework trained with semi-supervised learning. We validate S$^2$MTS$^2$ on the multi-label classification problems from Chest X-ray14 and CheXpert, and the multi-class classification from ISIC2018, where we show that it outperforms the previous SOTA semi-supervised learning methods by a large margin.
△ Less
Submitted 4 November, 2021; v1 submitted 5 March, 2021;
originally announced March 2021.
-
MyFood: A Food Segmentation and Classification System to Aid Nutritional Monitoring
Authors:
Charles N. C. Freitas,
Filipe R. Cordeiro,
Valmir Macario
Abstract:
The absence of food monitoring has contributed significantly to the increase in the population's weight. Due to the lack of time and busy routines, most people do not control and record what is consumed in their diet. Some solutions have been proposed in computer vision to recognize food images, but few are specialized in nutritional monitoring. This work presents the development of an intelligent…
▽ More
The absence of food monitoring has contributed significantly to the increase in the population's weight. Due to the lack of time and busy routines, most people do not control and record what is consumed in their diet. Some solutions have been proposed in computer vision to recognize food images, but few are specialized in nutritional monitoring. This work presents the development of an intelligent system that classifies and segments food presented in images to help the automatic monitoring of user diet and nutritional intake. This work shows a comparative study of state-of-the-art methods for image classification and segmentation, applied to food recognition. In our methodology, we compare the FCN, ENet, SegNet, DeepLabV3+, and Mask RCNN algorithms. We build a dataset composed of the most consumed Brazilian food types, containing nine classes and a total of 1250 images. The models were evaluated using the following metrics: Intersection over Union, Sensitivity, Specificity, Balanced Precision, and Positive Predefined Value. We also propose an system integrated into a mobile application that automatically recognizes and estimates the nutrients in a meal, assisting people with better nutritional monitoring. The proposed solution showed better results than the existing ones in the market. The dataset is publicly available at the following link http://doi.org/10.5281/zenodo.4041488
△ Less
Submitted 5 December, 2020;
originally announced December 2020.
-
A Survey on Deep Learning with Noisy Labels: How to train your model when you cannot trust on the annotations?
Authors:
Filipe R. Cordeiro,
Gustavo Carneiro
Abstract:
Noisy Labels are commonly present in data sets automatically collected from the internet, mislabeled by non-specialist annotators, or even specialists in a challenging task, such as in the medical field. Although deep learning models have shown significant improvements in different domains, an open issue is their ability to memorize noisy labels during training, reducing their generalization poten…
▽ More
Noisy Labels are commonly present in data sets automatically collected from the internet, mislabeled by non-specialist annotators, or even specialists in a challenging task, such as in the medical field. Although deep learning models have shown significant improvements in different domains, an open issue is their ability to memorize noisy labels during training, reducing their generalization potential. As deep learning models depend on correctly labeled data sets and label correctness is difficult to guarantee, it is crucial to consider the presence of noisy labels for deep learning training. Several approaches have been proposed in the literature to improve the training of deep learning models in the presence of noisy labels. This paper presents a survey on the main techniques in literature, in which we classify the algorithm in the following groups: robust losses, sample weighting, sample selection, meta-learning, and combined approaches. We also present the commonly used experimental setup, data sets, and results of the state-of-the-art models.
△ Less
Submitted 5 December, 2020;
originally announced December 2020.
-
EvidentialMix: Learning with Combined Open-set and Closed-set Noisy Labels
Authors:
Ragav Sachdeva,
Filipe R. Cordeiro,
Vasileios Belagiannis,
Ian Reid,
Gustavo Carneiro
Abstract:
The efficacy of deep learning depends on large-scale data sets that have been carefully curated with reliable data acquisition and annotation processes. However, acquiring such large-scale data sets with precise annotations is very expensive and time-consuming, and the cheap alternatives often yield data sets that have noisy labels. The field has addressed this problem by focusing on training mode…
▽ More
The efficacy of deep learning depends on large-scale data sets that have been carefully curated with reliable data acquisition and annotation processes. However, acquiring such large-scale data sets with precise annotations is very expensive and time-consuming, and the cheap alternatives often yield data sets that have noisy labels. The field has addressed this problem by focusing on training models under two types of label noise: 1) closed-set noise, where some training samples are incorrectly annotated to a training label other than their known true class; and 2) open-set noise, where the training set includes samples that possess a true class that is (strictly) not contained in the set of known training labels. In this work, we study a new variant of the noisy label problem that combines the open-set and closed-set noisy labels, and introduce a benchmark evaluation to assess the performance of training algorithms under this setup. We argue that such problem is more general and better reflects the noisy label scenarios in practice. Furthermore, we propose a novel algorithm, called EvidentialMix, that addresses this problem and compare its performance with the state-of-the-art methods for both closed-set and open-set noise on the proposed benchmark. Our results show that our method produces superior classification results and better feature representations than previous state-of-the-art methods. The code is available at https://github.com/ragavsachdeva/EvidentialMix.
△ Less
Submitted 11 November, 2020;
originally announced November 2020.
-
A semi-supervised fuzzy GrowCut algorithm to segment and classify regions of interest of mammographic images
Authors:
Filipe Rolim Cordeiro,
Wellington Pinheiro dos Santos,
Abel Guilhermino da Silva Filho
Abstract:
According to the World Health Organization, breast cancer is the most common form of cancer in women. It is the second leading cause of death among women round the world, becoming the most fatal form of cancer. Mammographic image segmentation is a fundamental task to support image analysis and diagnosis, taking into account shape analysis of mammary lesions and their borders. However, mammogram se…
▽ More
According to the World Health Organization, breast cancer is the most common form of cancer in women. It is the second leading cause of death among women round the world, becoming the most fatal form of cancer. Mammographic image segmentation is a fundamental task to support image analysis and diagnosis, taking into account shape analysis of mammary lesions and their borders. However, mammogram segmentation is a very hard process, once it is highly dependent on the types of mammary tissues. In this work we present a new semi-supervised segmentation algorithm based on the modification of the GrowCut algorithm to perform automatic mammographic image segmentation once a region of interest is selected by a specialist. In our proposal, we used fuzzy Gaussian membership functions to modify the evolution rule of the original GrowCut algorithm, in order to estimate the uncertainty of a pixel being object or background. The main impact of the proposed method is the significant reduction of expert effort in the initialization of seed points of GrowCut to perform accurate segmentation, once it removes the need of selection of background seeds. We also constructed an automatic point selection process based on the simulated annealing optimization method, avoiding the need of human intervention. The proposed approach was qualitatively compared with other state-of-the-art segmentation techniques, considering the shape of segmented regions. In order to validate our proposal, we built an image classifier using a classical multilayer perceptron. We used Zernike moments to extract segmented image features. This analysis employed 685 mammograms from IRMA breast cancer database, using fat and fibroid tissues. Results show that the proposed technique could achieve a classification rate of 91.28\% for fat tissues, evidencing the feasibility of our approach.
△ Less
Submitted 3 December, 2017;
originally announced January 2018.
-
Analysis of supervised and semi-supervised GrowCut applied to segmentation of masses in mammography images
Authors:
Filipe Rolim Cordeiro,
Wellington Pinheiro dos Santos,
Abel Guilhermino da Silva Filho
Abstract:
Breast cancer is already one of the most common form of cancer worldwide. Mammography image analysis is still the most effective diagnostic method to promote the early detection of breast cancer. Accurately segmenting tumors in digital mammography images is important to improve diagnosis capabilities of health specialists and avoid misdiagnosis. In this work, we evaluate the feasibility of applyin…
▽ More
Breast cancer is already one of the most common form of cancer worldwide. Mammography image analysis is still the most effective diagnostic method to promote the early detection of breast cancer. Accurately segmenting tumors in digital mammography images is important to improve diagnosis capabilities of health specialists and avoid misdiagnosis. In this work, we evaluate the feasibility of applying GrowCut to segment regions of tumor and we propose two GrowCut semi-supervised versions. All the analysis was performed by evaluating the application of segmentation techniques to a set of images obtained from the Mini-MIAS mammography image database. GrowCut segmentation was compared to Region Growing, Active Contours, Random Walks and Graph Cut techniques. Experiments showed that GrowCut, when compared to the other techniques, was able to acquire better results for the metrics analyzed. Moreover, the proposed semi-supervised versions of GrowCut was proved to have a clinically satisfactory quality of segmentation.
△ Less
Submitted 19 December, 2017;
originally announced December 2017.
