-
Deep Sketched Output Kernel Regression for Structured Prediction
Authors:
Tamim El Ahmad,
Junjie Yang,
Pierre Laforgue,
Florence d'Alché-Buc
Abstract:
By leveraging the kernel trick in the output space, kernel-induced losses provide a principled way to define structured output prediction tasks for a wide variety of output modalities. In particular, they have been successfully used in the context of surrogate non-parametric regression, where the kernel trick is typically exploited in the input space as well. However, when inputs are images or tex…
▽ More
By leveraging the kernel trick in the output space, kernel-induced losses provide a principled way to define structured output prediction tasks for a wide variety of output modalities. In particular, they have been successfully used in the context of surrogate non-parametric regression, where the kernel trick is typically exploited in the input space as well. However, when inputs are images or texts, more expressive models such as deep neural networks seem more suited than non-parametric methods. In this work, we tackle the question of how to train neural networks to solve structured output prediction tasks, while still benefiting from the versatility and relevance of kernel-induced losses. We design a novel family of deep neural architectures, whose last layer predicts in a data-dependent finite-dimensional subspace of the infinite-dimensional output feature space deriving from the kernel-induced loss. This subspace is chosen as the span of the eigenfunctions of a randomly-approximated version of the empirical kernel covariance operator. Interestingly, this approach unlocks the use of gradient descent algorithms (and consequently of any neural architecture) for structured prediction. Experiments on synthetic tasks as well as real-world supervised graph prediction problems show the relevance of our method.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
Sketch In, Sketch Out: Accelerating both Learning and Inference for Structured Prediction with Kernels
Authors:
Tamim El Ahmad,
Luc Brogat-Motte,
Pierre Laforgue,
Florence d'Alché-Buc
Abstract:
Leveraging the kernel trick in both the input and output spaces, surrogate kernel methods are a flexible and theoretically grounded solution to structured output prediction. If they provide state-of-the-art performance on complex data sets of moderate size (e.g., in chemoinformatics), these approaches however fail to scale. We propose to equip surrogate kernel methods with sketching-based approxim…
▽ More
Leveraging the kernel trick in both the input and output spaces, surrogate kernel methods are a flexible and theoretically grounded solution to structured output prediction. If they provide state-of-the-art performance on complex data sets of moderate size (e.g., in chemoinformatics), these approaches however fail to scale. We propose to equip surrogate kernel methods with sketching-based approximations, applied to both the input and output feature maps. We prove excess risk bounds on the original structured prediction problem, showing how to attain close-to-optimal rates with a reduced sketch size that depends on the eigendecay of the input/output covariance operators. From a computational perspective, we show that the two approximations have distinct but complementary impacts: sketching the input kernel mostly reduces training time, while sketching the output kernel decreases the inference time. Empirically, our approach is shown to scale, achieving state-of-the-art performance on benchmark data sets where non-sketched methods are intractable.
△ Less
Submitted 6 May, 2024; v1 submitted 20 February, 2023;
originally announced February 2023.
-
FALSE: Fake News Automatic and Lightweight Solution
Authors:
Fatema Al Mukhaini,
Shaikhah Al Abdoulie,
Aisha Al Kharuosi,
Amal El Ahmad,
Monther Aldwairi
Abstract:
Fake news existed ever since there was news, from rumors to printed media then radio and television. Recently, the information age, with its communications and Internet breakthroughs, exacerbated the spread of fake news. Additionally, aside from e-Commerce, the current Internet economy is dependent on advertisements, views and clicks, which prompted many developers to bait the end users to click l…
▽ More
Fake news existed ever since there was news, from rumors to printed media then radio and television. Recently, the information age, with its communications and Internet breakthroughs, exacerbated the spread of fake news. Additionally, aside from e-Commerce, the current Internet economy is dependent on advertisements, views and clicks, which prompted many developers to bait the end users to click links or ads. Consequently, the wild spread of fake news through social media networks has impacted real world issues from elections to 5G adoption and the handling of the Covid- 19 pandemic. Efforts to detect and thwart fake news has been there since the advent of fake news, from fact checkers to artificial intelligence-based detectors. Solutions are still evolving as more sophisticated techniques are employed by fake news propagators. In this paper, R code have been used to study and visualize a modern fake news dataset. We use clustering, classification, correlation and various plots to analyze and present the data. The experiments show high efficiency of classifiers in telling apart real from fake news.
△ Less
Submitted 16 August, 2022;
originally announced August 2022.
-
Fast Kernel Methods for Generic Lipschitz Losses via $p$-Sparsified Sketches
Authors:
Tamim El Ahmad,
Pierre Laforgue,
Florence d'Alché-Buc
Abstract:
Kernel methods are learning algorithms that enjoy solid theoretical foundations while suffering from important computational limitations. Sketching, which consists in looking for solutions among a subspace of reduced dimension, is a well studied approach to alleviate these computational burdens. However, statistically-accurate sketches, such as the Gaussian one, usually contain few null entries, s…
▽ More
Kernel methods are learning algorithms that enjoy solid theoretical foundations while suffering from important computational limitations. Sketching, which consists in looking for solutions among a subspace of reduced dimension, is a well studied approach to alleviate these computational burdens. However, statistically-accurate sketches, such as the Gaussian one, usually contain few null entries, such that their application to kernel methods and their non-sparse Gram matrices remains slow in practice. In this paper, we show that sparsified Gaussian (and Rademacher) sketches still produce theoretically-valid approximations while allowing for important time and space savings thanks to an efficient \emph{decomposition trick}. To support our method, we derive excess risk bounds for both single and multiple output kernel problems, with generic Lipschitz losses, hereby providing new guarantees for a wide range of applications, from robust regression to multiple quantile regression. Our theoretical results are complemented with experiments showing the empirical superiority of our approach over SOTA sketching methods.
△ Less
Submitted 6 November, 2023; v1 submitted 8 June, 2022;
originally announced June 2022.
-
A transferable prediction model of molecular adsorption on metals based on adsorbate and substrate properties
Authors:
Paolo Restuccia,
Ehsan A. Ahmad,
Nicholas M. Harrison
Abstract:
Surface adsorption is one of the fundamental processes in numerous fields, including catalysis, environment, energy and medicine. The development of an adsorption model which provides an effective prediction of binding energy in minutes has been a long term goal in surface and interface science. The solution has been elusive as identifying the intrinsic determinants of the adsorption energy for va…
▽ More
Surface adsorption is one of the fundamental processes in numerous fields, including catalysis, environment, energy and medicine. The development of an adsorption model which provides an effective prediction of binding energy in minutes has been a long term goal in surface and interface science. The solution has been elusive as identifying the intrinsic determinants of the adsorption energy for various compositions, structures and environments is non-trivial. We introduce a new and flexible model for predicting adsorption energies to metal substrates. The model is based on easily computed, intrinsic properties of the substrate and adsorbate. It is parameterised using machine learning based on first-principles calculations of probe molecules (e.g., H$_2$O, CO$_2$, O$_2$, N$_2$) adsorbed to a range of pure metal substrates. The model predicts the computed dissociative adsorption energy to metal surfaces with a correlation coefficient of 0.93 and a mean absolute error of 0.77 eV for the large database of molecular adsorption energies provided by \textit{Catalysis-Hub.org} which have a range of 15 eV. As the model is based on pre-computed quantities it provides near-instantaneous estimates of adsorption energies and it is sufficiently accurate to eliminate around 90\% of candidates in screening study of new adsorbates. The model, therefore, significantly enhances current efforts to identify new molecular coatings in many applied research fields.
△ Less
Submitted 8 May, 2022; v1 submitted 20 August, 2021;
originally announced August 2021.
-
Semantic Segmentation of Human Thigh Quadriceps Muscle in Magnetic Resonance Images
Authors:
Ezak Ahmad,
Manu Goyal,
Jamie S. McPhee,
Hans Degens,
Moi Hoon Yap
Abstract:
This paper presents an end-to-end solution for MRI thigh quadriceps segmentation. This is the first attempt that deep learning methods are used for the MRI thigh segmentation task. We use the state-of-the-art Fully Convolutional Networks with transfer learning approach for the semantic segmentation of regions of interest in MRI thigh scans. To further improve the performance of the segmentation, w…
▽ More
This paper presents an end-to-end solution for MRI thigh quadriceps segmentation. This is the first attempt that deep learning methods are used for the MRI thigh segmentation task. We use the state-of-the-art Fully Convolutional Networks with transfer learning approach for the semantic segmentation of regions of interest in MRI thigh scans. To further improve the performance of the segmentation, we propose a post-processing technique using basic image processing methods. With our proposed method, we have established a new benchmark for MRI thigh quadriceps segmentation with mean Jaccard Similarity Index of 0.9502 and processing time of 0.117 second per image.
△ Less
Submitted 9 August, 2018; v1 submitted 1 January, 2018;
originally announced January 2018.
-
Manipulation of the magnetic configuration of (Ga,Mn)As nanostructures
Authors:
J. A. Haigh,
M. Wang,
A. W. Rushforth,
E. Ahmad,
K. W. Edmonds,
R. P. Campion,
C. T. Foxon,
B. L. Gallagher
Abstract:
We have studied the magnetic reversal of L-shaped nanostructures fabricated from (Ga,Mn)As. The strain relaxation due to the lithographic patterning results in each arm having a uniaxial magnetic anisotropy. Our analysis confirms that the magnetic reversal takes place via a combination of coherent rotation and domain wall propagation with the domain wall positioned at the corner of the device at…
▽ More
We have studied the magnetic reversal of L-shaped nanostructures fabricated from (Ga,Mn)As. The strain relaxation due to the lithographic patterning results in each arm having a uniaxial magnetic anisotropy. Our analysis confirms that the magnetic reversal takes place via a combination of coherent rotation and domain wall propagation with the domain wall positioned at the corner of the device at intermediate stages of the magnetic hysteresis loops. The domain wall energy can be extracted from our analysis. Such devices have found implementation in studies of current induced domain wall motion and have the potential for application as non-volatile memory elements.
△ Less
Submitted 11 August, 2009;
originally announced August 2009.
-
Voltage control of magnetocrystalline anisotropy in ferromagnetic - semiconductor/piezoelectric hybrid structures
Authors:
A. W. Rushforth,
E. De Ranieri,
J. Zemen,
J. Wunderlich,
K. W. Edmonds,
C. S. King,
E. Ahmad,
R. P. Campion,
C. T. Foxon,
B. L. Gallagher,
K. Vyborny,
J. Kucera,
T. Jungwirth
Abstract:
We demonstrate dynamic voltage control of the magnetic anisotropy of a (Ga,Mn)As device bonded to a piezoelectric transducer. The application of a uniaxial strain leads to a large reorientation of the magnetic easy axis which is detected by measuring longitudinal and transverse anisotropic magnetoresistance coefficients. Calculations based on the mean-field kinetic-exchange model of (Ga,Mn)As pr…
▽ More
We demonstrate dynamic voltage control of the magnetic anisotropy of a (Ga,Mn)As device bonded to a piezoelectric transducer. The application of a uniaxial strain leads to a large reorientation of the magnetic easy axis which is detected by measuring longitudinal and transverse anisotropic magnetoresistance coefficients. Calculations based on the mean-field kinetic-exchange model of (Ga,Mn)As provide microscopic understanding of the measured effect. Electrically induced magnetization switching and detection of unconventional crystalline components of the anisotropic magnetoresistance are presented, illustrating the generic utility of the piezo voltage control to provide new device functionalities and in the research of micromagnetic and magnetotransport phenomena in diluted magnetic semiconductors.
△ Less
Submitted 22 January, 2008; v1 submitted 6 January, 2008;
originally announced January 2008.
