-
Transfer Learning with Ensembles of Deep Neural Networks for Skin Cancer Detection in Imbalanced Data Sets
Authors:
Aqsa Saeed Qureshi,
Teemu Roos
Abstract:
Several machine learning techniques for accurate detection of skin cancer from medical images have been reported. Many of these techniques are based on pre-trained convolutional neural networks (CNNs), which enable training the models based on limited amounts of training data. However, the classification accuracy of these models still tends to be severely limited by the scarcity of representative…
▽ More
Several machine learning techniques for accurate detection of skin cancer from medical images have been reported. Many of these techniques are based on pre-trained convolutional neural networks (CNNs), which enable training the models based on limited amounts of training data. However, the classification accuracy of these models still tends to be severely limited by the scarcity of representative images from malignant tumours. We propose a novel ensemble-based CNN architecture where multiple CNN models, some of which are pre-trained and some are trained only on the data at hand, along with auxiliary data in the form of metadata associated with the input images, are combined using a meta-learner. The proposed approach improves the model's ability to handle limited and imbalanced data. We demonstrate the benefits of the proposed technique using a dataset with 33126 dermoscopic images from 2056 patients. We evaluate the performance of the proposed technique in terms of the F1-measure, area under the ROC curve (AUC-ROC), and area under the PR-curve (AUC-PR), and compare it with that of seven different benchmark methods, including two recent CNN-based techniques. The proposed technique compares favourably in terms of all the evaluation metrics.
△ Less
Submitted 17 May, 2021; v1 submitted 22 March, 2021;
originally announced March 2021.
-
Extracting Signals of Higgs Boson From Background Noise Using Deep Neural Networks
Authors:
Muhammad Abbas,
Asifullah Khan,
Aqsa Saeed Qureshi,
Muhammad Waleed Khan
Abstract:
Higgs boson is a fundamental particle, and the classification of Higgs signals is a well-known problem in high energy physics. The identification of the Higgs signal is a challenging task because its signal has a resemblance to the background signals. This study proposes a Higgs signal classification using a novel combination of random forest, auto encoder and deep auto encoder to build a robust a…
▽ More
Higgs boson is a fundamental particle, and the classification of Higgs signals is a well-known problem in high energy physics. The identification of the Higgs signal is a challenging task because its signal has a resemblance to the background signals. This study proposes a Higgs signal classification using a novel combination of random forest, auto encoder and deep auto encoder to build a robust and generalized Higgs boson prediction system to discriminate the Higgs signal from the background noise. The proposed ensemble technique is based on achieving diversity in the decision space, and the results show good discrimination power on the private leaderboard; achieving an area under the Receiver Operating Characteristic curve of 0.9 and an Approximate Median Significance score of 3.429.
△ Less
Submitted 16 October, 2020;
originally announced October 2020.
-
Wind Speed Prediction using Deep Ensemble Learning with a Jet-like Architecture
Authors:
Aqsa Saeed Qureshi,
Asifullah Khan,
Muhammad Waleed Khan
Abstract:
The wind is one of the most increasingly used renewable energy resources. Accurate and reliable forecast of wind speed is necessary for efficient power production; however, it is not an easy task because it depends upon meteorological features of the surrounding region. Deep learning is extensively used these days for performing feature extraction. It has also been observed that the integration of…
▽ More
The wind is one of the most increasingly used renewable energy resources. Accurate and reliable forecast of wind speed is necessary for efficient power production; however, it is not an easy task because it depends upon meteorological features of the surrounding region. Deep learning is extensively used these days for performing feature extraction. It has also been observed that the integration of several learning models, known as ensemble learning, generally gives better performance compared to a single model. The design of wings, tail, and nose of a jet improves the aerodynamics resulting in a smooth and controlled flight of the jet against the variations of the air currents. Inspired by the shape and working of a jet, a novel Deep Ensemble Learning using Jet-like Architecture (DEL-Jet) technique is proposed to enhance the diversity and robustness of a learning system against the variations in the input space. The diverse feature spaces of the base-regressors are exploited using the jet-like ensemble architecture. Two Convolutional Neural Networks (as jet wings) and one deep Auto-Encoder (as jet tail) are used to extract the diverse feature spaces from the input data. After that, nonlinear PCA (as jet main body) is employed to reduce the dimensionality of extracted feature space. Finally, both the reduced and the original feature spaces are exploited to train the meta-regressor (as jet nose) for forecasting the wind speed. The performance of the proposed DEL-Jet technique is evaluated for ten independent runs and shows that the deep and jet-like architecture helps in improving the robustness and generalization of the learning system.
△ Less
Submitted 20 March, 2020; v1 submitted 28 February, 2020;
originally announced February 2020.
-
Malware Classification using Deep Learning based Feature Extraction and Wrapper based Feature Selection Technique
Authors:
Muhammad Furqan Rafique,
Muhammad Ali,
Aqsa Saeed Qureshi,
Asifullah Khan,
Anwar Majid Mirza
Abstract:
In the case of malware analysis, categorization of malicious files is an essential part after malware detection. Numerous static and dynamic techniques have been reported so far for categorizing malware. This research presents a deep learning-based malware detection (DLMD) technique based on static methods for classifying different malware families. The proposed DLMD technique uses both the byte a…
▽ More
In the case of malware analysis, categorization of malicious files is an essential part after malware detection. Numerous static and dynamic techniques have been reported so far for categorizing malware. This research presents a deep learning-based malware detection (DLMD) technique based on static methods for classifying different malware families. The proposed DLMD technique uses both the byte and ASM files for feature engineering, thus classifying malware families. First, features are extracted from byte files using two different Deep Convolutional Neural Networks (CNN). After that, essential and discriminative opcode features are selected using a wrapper-based mechanism, where Support Vector Machine (SVM) is used as a classifier. The idea is to construct a hybrid feature space by combining the different feature spaces to overcome the shortcoming of particular feature space and thus, reduce the chances of missing a malware. Finally, the hybrid feature space is used to train a Multilayer Perceptron, which classifies all nine different malware families. Experimental results show that proposed DLMD technique achieves log-loss of 0.09 for ten independent runs. Moreover, the proposed DLMD technique's performance is compared against different classifiers and shows its effectiveness in categorizing malware. The relevant code and database can be found at https://github.com/cyberhunters/Malware-Detection-Using-Machine-Learning.
△ Less
Submitted 26 December, 2020; v1 submitted 24 October, 2019;
originally announced October 2019.
-
A Recent Survey on the Applications of Genetic Programming in Image Processing
Authors:
Asifullah Khan,
Aqsa Saeed Qureshi,
Noorul Wahab,
Mutawara Hussain,
Muhammad Yousaf Hamza
Abstract:
Genetic Programming (GP) has been primarily used to tackle optimization, classification, and feature selection related tasks. The widespread use of GP is due to its flexible and comprehensible tree-type structure. Similarly, research is also gaining momentum in the field of Image Processing, because of its promising results over vast areas of applications ranging from medical Image Processing to m…
▽ More
Genetic Programming (GP) has been primarily used to tackle optimization, classification, and feature selection related tasks. The widespread use of GP is due to its flexible and comprehensible tree-type structure. Similarly, research is also gaining momentum in the field of Image Processing, because of its promising results over vast areas of applications ranging from medical Image Processing to multispectral imaging. Image Processing is mainly involved in applications such as computer vision, pattern recognition, image compression, storage, and medical diagnostics. This universal nature of images and their associated algorithm, i.e., complexities, gave an impetus to the exploration of GP. GP has thus been used in different ways for Image Processing since its inception. Many interesting GP techniques have been developed and employed in the field of Image Processing, and consequently, we aim to provide the research community an extensive view of these techniques. This survey thus presents the diverse applications of GP in Image Processing and provides useful resources for further research. Also, the comparison of different parameters used in different applications of Image Processing is summarized in tabular form. Moreover, analysis of the different parameters used in Image Processing related tasks is carried-out to save the time needed in the future for evaluating the parameters of GP. As more advancement is made in GP methodologies, its success in solving complex tasks, not only in Image Processing but also in other fields, may increase. Additionally, guidelines are provided for applying GP in Image Processing related tasks, the pros and cons of GP techniques are discussed, and some future directions are also set.
△ Less
Submitted 25 June, 2020; v1 submitted 18 January, 2019;
originally announced January 2019.
-
A Survey of the Recent Architectures of Deep Convolutional Neural Networks
Authors:
Asifullah Khan,
Anabia Sohail,
Umme Zahoora,
Aqsa Saeed Qureshi
Abstract:
Deep Convolutional Neural Network (CNN) is a special type of Neural Networks, which has shown exemplary performance on several competitions related to Computer Vision and Image Processing. Some of the exciting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, and Speech Recognition. The powerful learning ability…
▽ More
Deep Convolutional Neural Network (CNN) is a special type of Neural Networks, which has shown exemplary performance on several competitions related to Computer Vision and Image Processing. Some of the exciting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, and Speech Recognition. The powerful learning ability of deep CNN is primarily due to the use of multiple feature extraction stages that can automatically learn representations from the data. The availability of a large amount of data and improvement in the hardware technology has accelerated the research in CNNs, and recently interesting deep CNN architectures have been reported. Several inspiring ideas to bring advancements in CNNs have been explored, such as the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. However, the significant improvement in the representational capacity of the deep CNN is achieved through architectural innovations. Notably, the ideas of exploiting spatial and channel information, depth and width of architecture, and multi-path information processing have gained substantial attention. Similarly, the idea of using a block of layers as a structural unit is also gaining popularity. This survey thus focuses on the intrinsic taxonomy present in the recently reported deep CNN architectures and, consequently, classifies the recent innovations in CNN architectures into seven different categories. These seven categories are based on spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. Additionally, the elementary understanding of CNN components, current challenges, and applications of CNN are also provided.
△ Less
Submitted 10 May, 2020; v1 submitted 17 January, 2019;
originally announced January 2019.
-
Adaptive Transfer Learning in Deep Neural Networks: Wind Power Prediction using Knowledge Transfer from Region to Region and Between Different Task Domains
Authors:
Aqsa Saeed Qureshi,
Asifullah Khan
Abstract:
Transfer Learning (TL) in Deep Neural Networks is gaining importance because in most of the applications, the labeling of data is costly and time-consuming. Additionally, TL also provides an effective weight initialization strategy for Deep Neural Networks . This paper introduces the idea of Adaptive Transfer Learning in Deep Neural Networks (ATL-DNN) for wind power prediction. Specifically, we sh…
▽ More
Transfer Learning (TL) in Deep Neural Networks is gaining importance because in most of the applications, the labeling of data is costly and time-consuming. Additionally, TL also provides an effective weight initialization strategy for Deep Neural Networks . This paper introduces the idea of Adaptive Transfer Learning in Deep Neural Networks (ATL-DNN) for wind power prediction. Specifically, we show in case of wind power prediction that adaptive TL of Deep Neural Networks system can be adaptively modified as regards training on a different wind farm is concerned. The proposed ATL-DNN technique is tested for short-term wind power prediction, where continuously arriving information has to be exploited. Adaptive TL not only helps in providing good weight initialization, but is also helpful to utilize the incoming data for effective learning. Additionally, the proposed ATL-DNN technique is shown to transfer knowledge between different task domains (wind power to wind speed prediction) and from one region to another region. The simulation results show that the proposed ATL-DNN technique achieves average values of 0.0637,0.0986, and 0.0984 for the Mean-Absolute-Error, Root-Mean-Squared-Error, and Standard-Deviation-Error, respectively.
△ Less
Submitted 20 December, 2018; v1 submitted 30 October, 2018;
originally announced October 2018.
