-
A Big Data Based Framework for Executing Complex Query Over COVID-19 Datasets (COVID-QF)
Authors:
Eman A. Khashan,
Ali I. Eldesouky,
M. Fadel,
Sally M. Elghamrawy
Abstract:
COVID-19's rapid global spread has driven innovative tools for Big Data Analytics. These have guided organizations in all fields of the health industry to track and minimized the effects of virus. Researchers are required to detect coronaviruses through artificial intelligence, machine learning, and natural language processing, and to gain a complete understanding of the disease. COVID-19 takes pl…
▽ More
COVID-19's rapid global spread has driven innovative tools for Big Data Analytics. These have guided organizations in all fields of the health industry to track and minimized the effects of virus. Researchers are required to detect coronaviruses through artificial intelligence, machine learning, and natural language processing, and to gain a complete understanding of the disease. COVID-19 takes place in different countries in the world, with which only big data application and the work of NOSQL databases are suitable. There is a great number of platforms used for processing NOSQL Databases model like: Spark, H2O and Hadoop HDFS/MapReduce, which are proper to control and manage the enormous amount of data. Many challenges faced by large applications programmers, especially those that work on the COVID-19 databases through hybrid data models through different APIs and query. In this context, this paper proposes a storage framework to handle both SQL and NOSQL databases named (COVID-QF) for COVID-19 datasets in order to treat and handle the problems caused by virus spreading worldwide by reducing treatment times. In case of NoSQL database, COVID-QF uses Hadoop HDFS/Map Reduce and Apache Spark. The COVID-QF consists of three Layers: data collection layer, storage layer, and query Processing layer. The data is collected in the data collection layer. The storage layer divides data into collection of data-saving and processing blocks, and it connects the Connector of the spark with different databases engine to reduce time of saving and retrieving. While the Processing layer executes the request query and sends results. The proposed framework used three datasets increased for time for COVID-19 data (COVID-19-Merging, COVID-19-inside-Hubei and COVID-19-ex-Hubei) to test experiments of this study. The results obtained insure the superiority of the COVID-QF framework.
△ Less
Submitted 24 May, 2020;
originally announced May 2020.
-
A Multi-Dimensional Big Data Storing System for Generated COVID-19 Large-Scale Data using Apache Spark
Authors:
Manar A. Elmeiligy,
Ali I. El Desouky,
Sally M. Elghamrawy
Abstract:
The ongoing outbreak of coronavirus disease (COVID-19) had burst out in Wuhan China, specifically in December 2019. COVID-19 has caused by a new virus that had not been identified in human previously. This was followed by a widespread and rapid spread of this epidemic throughout the world. Daily, the number of the confirmed cases are increasing rapidly, number of the suspect increases, based on th…
▽ More
The ongoing outbreak of coronavirus disease (COVID-19) had burst out in Wuhan China, specifically in December 2019. COVID-19 has caused by a new virus that had not been identified in human previously. This was followed by a widespread and rapid spread of this epidemic throughout the world. Daily, the number of the confirmed cases are increasing rapidly, number of the suspect increases, based on the symptoms that accompany this disease, and unfortunately number of the deaths also increase. Therefore, with these increases in number of cases around the world, it becomes hard to manage all these cases information with different situations; if the patient either injured or suspect with which symptoms that appeared on the patient. Therefore, there is a critical need to construct a multi-dimensional system to store and analyze the generated large-scale data. In this paper, a Comprehensive Storing System for COVID-19 data using Apache Spark (CSS-COVID) is proposed, to handle and manage the problem caused by increasing the number of COVID-19 daily. CSS-COVID helps in decreasing the processing time for querying and storing COVID-19 daily data. CSS-COVID consists of three stages, namely, inserting and indexing, storing, and querying stage. In the inserting stage, data is divided into subsets and then index each subset separately. The storing stage uses set of storing-nodes to store data, while querying stage is responsible for handling the querying processes. Using Apache Spark in CSS-COVID leverages the performance of dealing with large-scale data of the coronavirus disease injured whom increase daily. A set of experiments are applied, using real COVID-19 Datasets, to prove the efficiency of CSS-COVID in indexing large-scale data.
△ Less
Submitted 30 April, 2020;
originally announced May 2020.
-
Detection of Coronavirus (COVID-19) Associated Pneumonia based on Generative Adversarial Networks and a Fine-Tuned Deep Transfer Learning Model using Chest X-ray Dataset
Authors:
Nour Eldeen M. Khalifa,
Mohamed Hamed N. Taha,
Aboul Ella Hassanien,
Sally Elghamrawy
Abstract:
The COVID-19 coronavirus is one of the devastating viruses according to the world health organization. This novel virus leads to pneumonia, which is an infection that inflames the lungs' air sacs of a human. One of the methods to detect those inflames is by using x-rays for the chest. In this paper, a pneumonia chest x-ray detection based on generative adversarial networks (GAN) with a fine-tuned…
▽ More
The COVID-19 coronavirus is one of the devastating viruses according to the world health organization. This novel virus leads to pneumonia, which is an infection that inflames the lungs' air sacs of a human. One of the methods to detect those inflames is by using x-rays for the chest. In this paper, a pneumonia chest x-ray detection based on generative adversarial networks (GAN) with a fine-tuned deep transfer learning for a limited dataset will be presented. The use of GAN positively affects the proposed model robustness and made it immune to the overfitting problem and helps in generating more images from the dataset. The dataset used in this research consists of 5863 X-ray images with two categories: Normal and Pneumonia. This research uses only 10% of the dataset for training data and generates 90% of images using GAN to prove the efficiency of the proposed model. Through the paper, AlexNet, GoogLeNet, Squeeznet, and Resnet18 are selected as deep transfer learning models to detect the pneumonia from chest x-rays. Those models are selected based on their small number of layers on their architectures, which will reflect in reducing the complexity of the models and the consumed memory and time. Using a combination of GAN and deep transfer models proved it is efficiency according to testing accuracy measurement. The research concludes that the Resnet18 is the most appropriate deep transfer model according to testing accuracy measurement and achieved 99% with the other performance metrics such as precision, recall, and F1 score while using GAN as an image augmenter. Finally, a comparison result was carried out at the end of the research with related work which used the same dataset except that this research used only 10% of original dataset. The presented work achieved a superior result than the related work in terms of testing accuracy.
△ Less
Submitted 2 April, 2020;
originally announced April 2020.
