-
MLXP: A Framework for Conducting Replicable Experiments in Python
Authors:
Michael Arbel,
Alexandre Zouaoui
Abstract:
Replicability in machine learning (ML) research is increasingly concerning due to the utilization of complex non-deterministic algorithms and the dependence on numerous hyper-parameter choices, such as model architecture and training datasets. Ensuring reproducible and replicable results is crucial for advancing the field, yet often requires significant technical effort to conduct systematic and w…
▽ More
Replicability in machine learning (ML) research is increasingly concerning due to the utilization of complex non-deterministic algorithms and the dependence on numerous hyper-parameter choices, such as model architecture and training datasets. Ensuring reproducible and replicable results is crucial for advancing the field, yet often requires significant technical effort to conduct systematic and well-organized experiments that yield robust conclusions. Several tools have been developed to facilitate experiment management and enhance reproducibility; however, they often introduce complexity that hinders adoption within the research community, despite being well-handled in industrial settings. To address the challenge of low adoption, we propose MLXP, an open-source, simple, and lightweight experiment management tool based on Python, available at https://github.com/inria-thoth/mlxp . MLXP streamlines the experimental process with minimal practitioner overhead while ensuring a high level of reproducibility.
△ Less
Submitted 17 June, 2024; v1 submitted 21 February, 2024;
originally announced February 2024.
-
Fast Semi-supervised Unmixing using Non-convex Optimization
Authors:
Behnood Rasti,
Alexandre Zouaoui,
Julien Mairal,
Jocelyn Chanussot
Abstract:
In this paper, we introduce a novel linear model tailored for semisupervised/library-based unmixing. Our model incorporates considerations for library mismatch while enabling the enforcement of the abundance sum-to-one constraint (ASC). Unlike conventional sparse unmixing methods, this model involves nonconvex optimization, presenting significant computational challenges. We demonstrate the effica…
▽ More
In this paper, we introduce a novel linear model tailored for semisupervised/library-based unmixing. Our model incorporates considerations for library mismatch while enabling the enforcement of the abundance sum-to-one constraint (ASC). Unlike conventional sparse unmixing methods, this model involves nonconvex optimization, presenting significant computational challenges. We demonstrate the efficacy of Alternating Methods of Multipliers (ADMM) in cyclically solving these intricate problems. We propose two semisupervised unmixing approaches, each relying on distinct priors applied to the new model in addition to the ASC: sparsity prior and convexity constraint. Our experimental results validate that enforcing the convexity constraint outperforms the sparsity prior for the endmember library. These results are corroborated across three simulated datasets (accounting for spectral variability and varying pixel purity levels) and the Cuprite dataset. Additionally, our comparison with conventional sparse unmixing methods showcases considerable advantages of our proposed model, which entails nonconvex optimization. Notably, our implementations of the proposed algorithms-fast semisupervised unmixing (FaSUn) and sparse unmixing using soft-shrinkage (SUnS)-prove considerably more efficient than traditional sparse unmixing methods. SUnS and FaSUn were implemented using PyTorch and provided in a dedicated Python package called Fast Semisupervised Unmixing (FUnmix), which is open-source and available at https://github.com/BehnoodRasti/FUnmix
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Image Processing and Machine Learning for Hyperspectral Unmixing: An Overview and the HySUPP Python Package
Authors:
Behnood Rasti,
Alexandre Zouaoui,
Julien Mairal,
Jocelyn Chanussot
Abstract:
Spectral pixels are often a mixture of the pure spectra of the materials, called endmembers, due to the low spatial resolution of hyperspectral sensors, double scattering, and intimate mixtures of materials in the scenes. Unmixing estimates the fractional abundances of the endmembers within the pixel. Depending on the prior knowledge of endmembers, linear unmixing can be divided into three main g…
▽ More
Spectral pixels are often a mixture of the pure spectra of the materials, called endmembers, due to the low spatial resolution of hyperspectral sensors, double scattering, and intimate mixtures of materials in the scenes. Unmixing estimates the fractional abundances of the endmembers within the pixel. Depending on the prior knowledge of endmembers, linear unmixing can be divided into three main groups: supervised, semi-supervised, and unsupervised (blind) linear unmixing. Advances in Image processing and machine learning substantially affected unmixing. This paper provides an overview of advanced and conventional unmixing approaches. Additionally, we draw a critical comparison between advanced and conventional techniques from the three categories. We compare the performance of the unmixing techniques on three simulated and two real datasets. The experimental results reveal the advantages of different unmixing categories for different unmixing scenarios. Moreover, we provide an open-source Python-based package available at https://github.com/BehnoodRasti/HySUPP to reproduce the results.
△ Less
Submitted 26 April, 2024; v1 submitted 18 August, 2023;
originally announced August 2023.
-
SUnAA: Sparse Unmixing using Archetypal Analysis
Authors:
Behnood Rasti,
Alexandre Zouaoui,
Julien Mairal,
Jocelyn Chanussot
Abstract:
This paper introduces a new sparse unmixing technique using archetypal analysis (SUnAA). First, we design a new model based on archetypal analysis. We assume that the endmembers of interest are a convex combination of endmembers provided by a spectral library and that the number of endmembers of interest is known. Then, we propose a minimization problem. Unlike most conventional sparse unmixing me…
▽ More
This paper introduces a new sparse unmixing technique using archetypal analysis (SUnAA). First, we design a new model based on archetypal analysis. We assume that the endmembers of interest are a convex combination of endmembers provided by a spectral library and that the number of endmembers of interest is known. Then, we propose a minimization problem. Unlike most conventional sparse unmixing methods, here the minimization problem is non-convex. We minimize the optimization objective iteratively using an active set algorithm. Our method is robust to the initialization and only requires the number of endmembers of interest. SUnAA is evaluated using two simulated datasets for which results confirm its better performance over other conventional and advanced techniques in terms of signal-to-reconstruction error. SUnAA is also applied to Cuprite dataset and the results are compared visually with the available geological map provided for this dataset. The qualitative assessment demonstrates the successful estimation of the minerals abundances and significantly improves the detection of dominant minerals compared to the conventional regression-based sparse unmixing methods. The Python implementation of SUnAA can be found at: https://github.com/BehnoodRasti/SUnAA.
△ Less
Submitted 9 August, 2023;
originally announced August 2023.
-
A Survey on Malware Detection with Graph Representation Learning
Authors:
Tristan Bilot,
Nour El Madhoun,
Khaldoun Al Agha,
Anis Zouaoui
Abstract:
Malware detection has become a major concern due to the increasing number and complexity of malware. Traditional detection methods based on signatures and heuristics are used for malware detection, but unfortunately, they suffer from poor generalization to unknown attacks and can be easily circumvented using obfuscation techniques. In recent years, Machine Learning (ML) and notably Deep Learning (…
▽ More
Malware detection has become a major concern due to the increasing number and complexity of malware. Traditional detection methods based on signatures and heuristics are used for malware detection, but unfortunately, they suffer from poor generalization to unknown attacks and can be easily circumvented using obfuscation techniques. In recent years, Machine Learning (ML) and notably Deep Learning (DL) achieved impressive results in malware detection by learning useful representations from data and have become a solution preferred over traditional methods. More recently, the application of such techniques on graph-structured data has achieved state-of-the-art performance in various domains and demonstrates promising results in learning more robust representations from malware. Yet, no literature review focusing on graph-based deep learning for malware detection exists. In this survey, we provide an in-depth literature review to summarize and unify existing works under the common approaches and architectures. We notably demonstrate that Graph Neural Networks (GNNs) reach competitive results in learning robust embeddings from malware represented as expressive graph structures, leading to an efficient detection by downstream classifiers. This paper also reviews adversarial attacks that are utilized to fool graph-based detection methods. Challenges and future research directions are discussed at the end of the paper.
△ Less
Submitted 17 August, 2023; v1 submitted 28 March, 2023;
originally announced March 2023.
-
Entropic Descent Archetypal Analysis for Blind Hyperspectral Unmixing
Authors:
Alexandre Zouaoui,
Gedeon Muhawenayo,
Behnood Rasti,
Jocelyn Chanussot,
Julien Mairal
Abstract:
In this paper, we introduce a new algorithm based on archetypal analysis for blind hyperspectral unmixing, assuming linear mixing of endmembers. Archetypal analysis is a natural formulation for this task. This method does not require the presence of pure pixels (i.e., pixels containing a single material) but instead represents endmembers as convex combinations of a few pixels present in the origin…
▽ More
In this paper, we introduce a new algorithm based on archetypal analysis for blind hyperspectral unmixing, assuming linear mixing of endmembers. Archetypal analysis is a natural formulation for this task. This method does not require the presence of pure pixels (i.e., pixels containing a single material) but instead represents endmembers as convex combinations of a few pixels present in the original hyperspectral image. Our approach leverages an entropic gradient descent strategy, which (i) provides better solutions for hyperspectral unmixing than traditional archetypal analysis algorithms, and (ii) leads to efficient GPU implementations. Since running a single instance of our algorithm is fast, we also propose an ensembling mechanism along with an appropriate model selection procedure that make our method robust to hyper-parameter choices while kee** the computational complexity reasonable. By using six standard real datasets, we show that our approach outperforms state-of-the-art matrix factorization and recent deep learning methods. We also provide an open-source PyTorch implementation: https://github.com/inria-thoth/EDAA.
△ Less
Submitted 26 September, 2022; v1 submitted 22 September, 2022;
originally announced September 2022.
-
A Trainable Spectral-Spatial Sparse Coding Model for Hyperspectral Image Restoration
Authors:
Théo Bodrito,
Alexandre Zouaoui,
Jocelyn Chanussot,
Julien Mairal
Abstract:
Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspec…
▽ More
Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspectral signals acquired on the spot makes restoration tasks challenging. In particular, training deep neural networks for restoration is difficult, in contrast to traditional RGB imaging problems where deep models tend to shine. In this paper, we advocate instead for a hybrid approach based on sparse coding principles that retains the interpretability of classical techniques encoding domain knowledge with handcrafted image priors, while allowing to train model parameters end-to-end without massive amounts of data. We show on various denoising benchmarks that our method is computationally efficient and significantly outperforms the state of the art.
△ Less
Submitted 18 November, 2021;
originally announced November 2021.
-
Singular elliptic equation involving the GJMS operator on the standard unit sphere
Authors:
Mohammed Benalili,
Ali Zouaoui
Abstract:
Given a Riemannian compact manifold (M,g) of dimension n>4, we have proven in [1] under some conditions that the equation : Pg(u) = Bu +Au2+Cu (1) where Pg is the GJMS-operator, n = dim(M) > 2k, A, B and C are smooth positive functions on M, p > 1 and 2] denotes the critical Sobolev admits twodistinct positive solutions. The proof of this result is essentially based on the given smooth function '…
▽ More
Given a Riemannian compact manifold (M,g) of dimension n>4, we have proven in [1] under some conditions that the equation : Pg(u) = Bu +Au2+Cu (1) where Pg is the GJMS-operator, n = dim(M) > 2k, A, B and C are smooth positive functions on M, p > 1 and 2] denotes the critical Sobolev admits twodistinct positive solutions. The proof of this result is essentially based on the given smooth function ' > 0 with norm k'kPg = 1 fulfilling some conditions ( see Theorem 3 in [1]). In this note we construct an example of such function on the unit standard sphere (Sn; h). Con- sequently the conditions of the Theorem are improved in the case of (Sn; h)
△ Less
Submitted 27 September, 2016;
originally announced September 2016.
-
Singular elliptic equation involving the GJMS operator on compact Riemannian manifold
Authors:
Mohammed Benalili,
Ali Zouaoui
Abstract:
In this paper we consider a singular elliptic equation involving the GJMS (Graham-Jenne-Mason-Sparling) operator of order k on n-dimensional compact Riemannian manifold with 2k<n. Mutiplicity and nonexistence results are established.
In this paper we consider a singular elliptic equation involving the GJMS (Graham-Jenne-Mason-Sparling) operator of order k on n-dimensional compact Riemannian manifold with 2k<n. Mutiplicity and nonexistence results are established.
△ Less
Submitted 30 April, 2017; v1 submitted 12 May, 2016;
originally announced May 2016.
