-
Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining
Authors:
Nour Eddine Zekaoui,
Siham Yousfi,
Maryem Rhanoui,
Mounia Mikram
Abstract:
Opinion mining, also known as sentiment analysis, is a subfield of natural language processing (NLP) that focuses on identifying and extracting subjective information in textual material. This can include determining the overall sentiment of a piece of text (e.g., positive or negative), as well as identifying specific emotions or opinions expressed in the text, that involves the use of advanced ma…
▽ More
Opinion mining, also known as sentiment analysis, is a subfield of natural language processing (NLP) that focuses on identifying and extracting subjective information in textual material. This can include determining the overall sentiment of a piece of text (e.g., positive or negative), as well as identifying specific emotions or opinions expressed in the text, that involves the use of advanced machine and deep learning techniques. Recently, transformer-based language models make this task of human emotion analysis intuitive, thanks to the attention mechanism and parallel computation. These advantages make such models very powerful on linguistic tasks, unlike recurrent neural networks that spend a lot of time on sequential processing, making them prone to fail when it comes to processing long text. The scope of our paper aims to study the behaviour of the cutting-edge Transformer-based language models on opinion mining and provide a high-level comparison between them to highlight their key particularities. Additionally, our comparative study shows leads and paves the way for production engineers regarding the approach to focus on and is useful for researchers as it provides guidelines for future research subjects.
△ Less
Submitted 6 August, 2023;
originally announced August 2023.
-
BERT Based Clinical Knowledge Extraction for Biomedical Knowledge Graph Construction and Analysis
Authors:
Ayoub Harnoune,
Maryem Rhanoui,
Mounia Mikram,
Siham Yousfi,
Zineb Elkaimbillah,
Bouchra El Asri
Abstract:
Background : Knowledge is evolving over time, often as a result of new discoveries or changes in the adopted methods of reasoning. Also, new facts or evidence may become available, leading to new understandings of complex phenomena. This is particularly true in the biomedical field, where scientists and physicians are constantly striving to find new methods of diagnosis, treatment and eventually c…
▽ More
Background : Knowledge is evolving over time, often as a result of new discoveries or changes in the adopted methods of reasoning. Also, new facts or evidence may become available, leading to new understandings of complex phenomena. This is particularly true in the biomedical field, where scientists and physicians are constantly striving to find new methods of diagnosis, treatment and eventually cure. Knowledge Graphs (KGs) offer a real way of organizing and retrieving the massive and growing amount of biomedical knowledge.
Objective : We propose an end-to-end approach for knowledge extraction and analysis from biomedical clinical notes using the Bidirectional Encoder Representations from Transformers (BERT) model and Conditional Random Field (CRF) layer.
Methods : The approach is based on knowledge graphs, which can effectively process abstract biomedical concepts such as relationships and interactions between medical entities. Besides offering an intuitive way to visualize these concepts, KGs can solve more complex knowledge retrieval problems by simplifying them into simpler representations or by transforming the problems into representations from different perspectives. We created a biomedical Knowledge Graph using using Natural Language Processing models for named entity recognition and relation extraction. The generated biomedical knowledge graphs (KGs) are then used for question answering.
Results : The proposed framework can successfully extract relevant structured information with high accuracy (90.7% for Named-entity recognition (NER), 88% for relation extraction (RE)), according to experimental findings based on real-world 505 patient biomedical unstructured clinical notes.
Conclusions : In this paper, we propose a novel end-to-end system for the construction of a biomedical knowledge graph from clinical textual using a variation of BERT models.
△ Less
Submitted 21 April, 2023;
originally announced April 2023.
-
Supervised Machine Learning for Breast Cancer Risk Factors Analysis and Survival Prediction
Authors:
Khaoula Chtouki,
Maryem Rhanoui,
Mounia Mikram,
Kamelia Amazian,
Siham Yousfi
Abstract:
The choice of the most effective treatment may eventually be influenced by breast cancer survival prediction. To predict the chances of a patient surviving, a variety of techniques were employed, such as statistical, machine learning, and deep learning models. In the current study, 1904 patient records from the METABRIC dataset were utilized to predict a 5-year breast cancer survival using a machi…
▽ More
The choice of the most effective treatment may eventually be influenced by breast cancer survival prediction. To predict the chances of a patient surviving, a variety of techniques were employed, such as statistical, machine learning, and deep learning models. In the current study, 1904 patient records from the METABRIC dataset were utilized to predict a 5-year breast cancer survival using a machine learning approach. In this study, we compare the outcomes of seven classification models to evaluate how well they perform using the following metrics: recall, AUC, confusion matrix, accuracy, precision, false positive rate, and true positive rate. The findings demonstrate that the classifiers for Logistic Regression (LR), Support Vector Machines (SVM), Decision Tree (DT), Random Forest (RD), Extremely Randomized Trees (ET), K-Nearest Neighbor (KNN), and Adaptive Boosting (AdaBoost) can accurately predict the survival rate of the tested samples, which is 75,4\%, 74,7\%, 71,5\%, 75,5\%, 70,3\%, and 78 percent.
△ Less
Submitted 13 April, 2023;
originally announced April 2023.
-
Smart Agriculture : A Novel Multilevel Approach for Agricultural Risk Assessment over Unstructured Data
Authors:
Hasna Najmi,
Mounia Mikram,
Maryem Rhanoui,
Siham Yousfi
Abstract:
Detecting opportunities and threats from massive text data is a challenging task for most. Traditionally, companies would rely mainly on structured data to detect and predict risks, losing a huge amount of information that could be extracted from unstructured text data. Fortunately, artificial intelligence came to remedy this issue by innovating in data extraction and processing techniques, allowi…
▽ More
Detecting opportunities and threats from massive text data is a challenging task for most. Traditionally, companies would rely mainly on structured data to detect and predict risks, losing a huge amount of information that could be extracted from unstructured text data. Fortunately, artificial intelligence came to remedy this issue by innovating in data extraction and processing techniques, allowing us to understand and make use of Natural Language data and turning it into structures that a machine can process and extract insight from. Uncertainty refers to a state of not knowing what will happen in the future. This paper aims to leverage natural language processing and machine learning techniques to model uncertainties and evaluate the risk level in each uncertainty cluster using massive text data.
△ Less
Submitted 22 November, 2022;
originally announced November 2022.
-
Towards a Generic Multimodal Architecture for Batch and Streaming Big Data Integration
Authors:
Siham Yousfi,
Maryem Rhanoui,
Dalila Chiadmi
Abstract:
Big Data are rapidly produced from various heterogeneous data sources. They are of different types (text, image, video or audio) and have different levels of reliability and completeness. One of the most interesting architectures that deal with the large amount of emerging data at high velocity is called the lambda architecture. In fact, it combines two different processing layers namely batch and…
▽ More
Big Data are rapidly produced from various heterogeneous data sources. They are of different types (text, image, video or audio) and have different levels of reliability and completeness. One of the most interesting architectures that deal with the large amount of emerging data at high velocity is called the lambda architecture. In fact, it combines two different processing layers namely batch and speed layers, each providing specific views of data while ensuring robustness, fast and scalable data processing. However, most papers dealing with the lambda architecture are focusing one single type of data generally produced by a single data source. Besides, the layers of the architecture are implemented independently, or, at best, are combined to perform basic processing without assessing either the data reliability or completeness. Therefore, inspired by the lambda architecture, we propose in this paper a generic multimodal architecture that combines both batch and streaming processing in order to build a complete, global and accurate insight in near-real-time based on the knowledge extracted from multiple heterogeneous Big Data sources. Our architecture uses batch processing to analyze the data structures and contents, build the learning models and calculate the reliability index of the involved sources, while the streaming processing uses the built-in models of the batch layer to immediately process incoming data and rapidly provide results. We validate our architecture in the context of urban traffic management systems in order to detect congestions.
△ Less
Submitted 9 August, 2021;
originally announced August 2021.
-
Structural behaviour of BiFeO3/SrRuO3 superlattices: an X-ray diffraction and Raman spectroscopy investigation
Authors:
S. Yousfi,
M. El Marssi,
H. Bouyanfif
Abstract:
Epitaxial BiFeO3/SrRuO3 superlattices have been grown by pulsed laser deposition on a (001) oriented LaAlO3 substrate and probed by X-ray diffraction and Raman spectroscopy. To investigate the structural competition between rhombohedral BiFeO3 and orthorhombic SrRuO3 the total thickness of all SLs was kept constant and the bilayer thickness (period) Λ was varied. The interlayer strain effects are…
▽ More
Epitaxial BiFeO3/SrRuO3 superlattices have been grown by pulsed laser deposition on a (001) oriented LaAlO3 substrate and probed by X-ray diffraction and Raman spectroscopy. To investigate the structural competition between rhombohedral BiFeO3 and orthorhombic SrRuO3 the total thickness of all SLs was kept constant and the bilayer thickness (period) Λ was varied. The interlayer strain effects are therefore tuned from large strain effects (short Λ period) to quasi-relaxed structure (large Λ). A complementary investigation using X-ray diffraction and phonon dynamics hints to change from a rhombohedral to a tetragonal structure in the superlattices with the increase of the interlayer strain effect.
△ Less
Submitted 8 July, 2021;
originally announced July 2021.
-
Impedance spectroscopy and conduction mechanism of a BiFe$_{0.95}$Mn$_{0.05}$O$_3$ thin film
Authors:
S. Yousfi,
M. El Marssi,
H. Bouyanfif
Abstract:
Dielectric response and conduction mechanism were investigated for a multiferroic BiFe$_{0.95}$Mn$_{0.05}$O$_3$ epitaxial thin film. A contribution from a thermally activated interface (0.37 eV) and the bulk of the film on the dielectric response were observed through the comparison between experimental results and equivalent circuit model. The low frequency interface relaxation signatures strongl…
▽ More
Dielectric response and conduction mechanism were investigated for a multiferroic BiFe$_{0.95}$Mn$_{0.05}$O$_3$ epitaxial thin film. A contribution from a thermally activated interface (0.37 eV) and the bulk of the film on the dielectric response were observed through the comparison between experimental results and equivalent circuit model. The low frequency interface relaxation signatures strongly suggest a Maxwell-Wagner space charge origin. The alternative current conductivity deduced from the model follows a power law frequency dependence suggesting a polaronic hop** mechanism while the low frequency limit is in perfect agreement with the direct current conduction mechanism. The current-voltage characteristics were indeed correlated with Schottky-Simmons interface limited transport with activation energy of 0.36 eV, close to the one extracted from the impedance analysis. Such analysis of the electrostatic landscape and dielectric behaviour may help to further understanding the anomalous photo-induced properties in the BiFeO$_3$ system.
△ Less
Submitted 6 March, 2021;
originally announced March 2021.
-
Tailoring the photovoltaic effect in (111) oriented BiFeO3/LaFeO3 superlattices
Authors:
J. Belhadi,
S. Yousfi,
M. El Marssi,
D. C. Arnold,
H. Bouyanfif
Abstract:
Ferroelectric and photovoltaic properties of BiFeO3/LaFeO3 superlattices grown by pulsed laser deposition have been investigated being the bilayer thickness). For a high concentration of BiFeO3 a ferroelectric state is observed simultaneously with a switchable photovoltaic response. In contrast for certain concentration of LaFeO3 a non-switchable photovoltaic effect is evidenced. Such modulation o…
▽ More
Ferroelectric and photovoltaic properties of BiFeO3/LaFeO3 superlattices grown by pulsed laser deposition have been investigated being the bilayer thickness). For a high concentration of BiFeO3 a ferroelectric state is observed simultaneously with a switchable photovoltaic response. In contrast for certain concentration of LaFeO3 a non-switchable photovoltaic effect is evidenced. Such modulation of the PV response in the superlattices is attributed to the ferroelectric to paraelectric phase transition which is controlled with the increase of x. Remarkably, concomitant to this change of PV mechanism, a change of the conduction mechanism also seems to take place from a bulk-limited to an interface-limited transport as x increases.
△ Less
Submitted 10 December, 2020;
originally announced December 2020.
-
String Diagrams for Assembly Planning
Authors:
Jade Master,
Evan Patterson,
Shahin Yousfi,
Arquimedes Canedo
Abstract:
Assembly planning is a difficult problem for companies. Many disciplines such as design, planning, scheduling, and manufacturing execution need to be carefully engineered and coordinated to create successful product assembly plans. Recent research in the field of design for assembly has proposed new methodologies to design product structures in such a way that their assembly is easier. However, pr…
▽ More
Assembly planning is a difficult problem for companies. Many disciplines such as design, planning, scheduling, and manufacturing execution need to be carefully engineered and coordinated to create successful product assembly plans. Recent research in the field of design for assembly has proposed new methodologies to design product structures in such a way that their assembly is easier. However, present assembly planning approaches lack the engineering tool support to capture all the constraints associated to assembly planning in a unified manner. This paper proposes CompositionalPlanning, a string diagram based framework for assembly planning. In the proposed framework, string diagrams and their compositional properties serve as the foundation for an engineering tool where CAD designs interact with planning and scheduling algorithms to automatically create high-quality assembly plans. These assembly plans are then executed in simulation to measure their performance and to visualize their key build characteristics. We demonstrate the versatility of this approach in the LEGO assembly domain. We developed two reference LEGO CAD models that are processed by CompositionalPlanning's algorithmic pipeline. We compare sequential and parallel assembly plans in a Minecraft simulation and show that the time-to-build performance can be optimized by our algorithms.
△ Less
Submitted 11 May, 2020; v1 submitted 23 September, 2019;
originally announced September 2019.
-
Conduction mechanism and switchable photovoltaic effect in (111) oriented BiFe$_{0.95}$Mn$_{0.05}$O$_{3}$ thin film
Authors:
Jamal Belhadi,
J. Ruvalcaba,
S. Yousfi,
M. El-Marssi,
T. Fraga Córdova,
S. Matzen,
P. Lecoeur,
H. Bouyanfif
Abstract:
Epitaxial 200nm BiFe$_{0.95}$Mn$_{0.05}$O$_{3}$ (BFO) film was grown by pulsed laser deposition on (111) oriented SrTiO3 substrate buffered with a 50nm thick SrRuO$_{3}$ electrode. The BFO thin film shows a rhombohedral structure and a large remnant polarization of Pr = 104 $μ$C/cm$^{2}$. By comparing I(V) characteristics with different conduction models we reveal the presence of both bulk limited…
▽ More
Epitaxial 200nm BiFe$_{0.95}$Mn$_{0.05}$O$_{3}$ (BFO) film was grown by pulsed laser deposition on (111) oriented SrTiO3 substrate buffered with a 50nm thick SrRuO$_{3}$ electrode. The BFO thin film shows a rhombohedral structure and a large remnant polarization of Pr = 104 $μ$C/cm$^{2}$. By comparing I(V) characteristics with different conduction models we reveal the presence of both bulk limited Poole-Frenkel and Schottky interface mechanisms and each one dominates in a specific range of temperature. At room temperature and under 10mW laser illumination, the as grown BFO film presents short-circuit current density (Jsc) and open circuit voltage (Voc) of 2.25mA/cm$^{2}$ and -0.55V respectively. This PV effect can be switched by applying positive voltage pulses higher than the coercive field. For low temperatures a large Voc value of about -4.5V (-225kV/cm) is observed which suggests a bulk non-centrosymmetric origin of the PV response.
△ Less
Submitted 4 May, 2019;
originally announced May 2019.
-
Structural investigation of (111) oriented (BiFeO3)(1-x)Λ/(LaFeO3)xΛ superlattices by X-ray diffraction and Raman spectroscopy
Authors:
J. Belhadi,
S. Yousfi,
H. Bouyanfif,
M. El Marssi
Abstract:
(BiFeO3)(1-x)Λ/(LaFeO3)xΛ superlattices (SLs) with varying x have been grown by pulsed laser deposition on (111) oriented SrTiO3 substrates. In order to obtain good epitaxy and flat samples a conducting SrRuO3 buffer has been deposited prior to the superlattices to screen the polar mismatch for such (111) SrTiO3 orientation. X-ray diffraction reciprocal space map** on different family of planes…
▽ More
(BiFeO3)(1-x)Λ/(LaFeO3)xΛ superlattices (SLs) with varying x have been grown by pulsed laser deposition on (111) oriented SrTiO3 substrates. In order to obtain good epitaxy and flat samples a conducting SrRuO3 buffer has been deposited prior to the superlattices to screen the polar mismatch for such (111) SrTiO3 orientation. X-ray diffraction reciprocal space map** on different family of planes were collected and evidenced a room temperature structural change at x=0.5 from a rhombohedral/monoclinic structure for rich BiFeO3 to an orthorhombic symmetry for rich LaFeO3. This symmetry change has been confirmed by Raman spectroscopy and demonstrates the different phase stability compared to similar SLs grown on (100) SrTiO3. The strongly anisotropic strain and oxygen octahedral rotation/tilt system compatibility at the interfaces probably explain the orientation dependence of the phase stability in such superlattices.
△ Less
Submitted 6 January, 2019;
originally announced January 2019.
-
Influence of temperature and wavelength on the switchable photovoltaic response of a BiFe0.95Mn0.05O3 thin film
Authors:
Said Yousfi,
Houssny Bouyanfif,
Mimoun El Marssi
Abstract:
Photovoltaic (PV) response of epitaxial BiFe0.95Mn0.05O3 thin film grown by pulsed laser deposition has been investigated on a broad range of temperature. Wavelength dependent photovoltaic effect shows the contribution of in gap level states most likely connected to the manganese do** on the B-site of the perovskite unit cells and presence of defects (Bi and O vacancies). The temperature depende…
▽ More
Photovoltaic (PV) response of epitaxial BiFe0.95Mn0.05O3 thin film grown by pulsed laser deposition has been investigated on a broad range of temperature. Wavelength dependent photovoltaic effect shows the contribution of in gap level states most likely connected to the manganese do** on the B-site of the perovskite unit cells and presence of defects (Bi and O vacancies). The temperature dependent response of the PV response rules out electromigration and/or Schottky barriers as dominant mechanisms. This is corroborated with the observed switchable photovoltaic effect that can be explained either by the depolarizing field or bulk photovoltaic effect. In addition the PV response shows strong correlation with the low temperature polaronic like conduction mechanism and high open circuit voltage (2.5V) is detected in the investigated vertical capacitive geometry.
△ Less
Submitted 5 January, 2019;
originally announced January 2019.
