Showing 1–50 of 62 results for author: Oprea, A

SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agents

Authors: Ethan Rathbun, Christopher Amato, Alina Oprea

Abstract: Reinforcement learning (RL) is an actively growing field that is seeing increased usage in real-world, safety-critical applications -- making it paramount to ensure the robustness of RL algorithms against adversarial attacks. In this work we explore a particularly stealthy form of training-time attacks against RL -- backdoor poisoning. Here the adversary intercepts the training of an RL agent with… ▽ More Reinforcement learning (RL) is an actively growing field that is seeing increased usage in real-world, safety-critical applications -- making it paramount to ensure the robustness of RL algorithms against adversarial attacks. In this work we explore a particularly stealthy form of training-time attacks against RL -- backdoor poisoning. Here the adversary intercepts the training of an RL agent with the goal of reliably inducing a particular action when the agent observes a pre-determined trigger at inference time. We uncover theoretical limitations of prior work by proving their inability to generalize across domains and MDPs. Motivated by this, we formulate a novel poisoning attack framework which interlinks the adversary's objectives with those of finding an optimal policy -- guaranteeing attack success in the limit. Using insights from our theoretical analysis we develop ``SleeperNets'' as a universal backdoor attack which exploits a newly proposed threat model and leverages dynamic reward poisoning techniques. We evaluate our attack in 6 environments spanning multiple domains and demonstrate significant improvements in attack success over existing methods, while preserving benign episodic return. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: 23 pages, 14 figures, NeurIPS

Phantom: General Trigger Attacks on Retrieval Augmented Language Generation

Authors: Harsh Chaudhari, Giorgio Severi, John Abascal, Matthew Jagielski, Christopher A. Choquette-Choo, Milad Nasr, Cristina Nita-Rotaru, Alina Oprea

Abstract: Retrieval Augmented Generation (RAG) expands the capabilities of modern large language models (LLMs) in chatbot applications, enabling developers to adapt and personalize the LLM output without expensive training or fine-tuning. RAG systems use an external knowledge database to retrieve the most relevant documents for a given query, providing this context to the LLM generator. While RAG achieves i… ▽ More Retrieval Augmented Generation (RAG) expands the capabilities of modern large language models (LLMs) in chatbot applications, enabling developers to adapt and personalize the LLM output without expensive training or fine-tuning. RAG systems use an external knowledge database to retrieve the most relevant documents for a given query, providing this context to the LLM generator. While RAG achieves impressive utility in many applications, its adoption to enable personalized generative models introduces new security risks. In this work, we propose new attack surfaces for an adversary to compromise a victim's RAG system, by injecting a single malicious document in its knowledge database. We design Phantom, general two-step attack framework against RAG augmented LLMs. The first step involves crafting a poisoned document designed to be retrieved by the RAG system within the top-k results only when an adversarial trigger, a specific sequence of words acting as backdoor, is present in the victim's queries. In the second step, a specially crafted adversarial string within the poisoned document triggers various adversarial attacks in the LLM generator, including denial of service, reputation damage, privacy violations, and harmful behaviors. We demonstrate our attacks on multiple LLM architectures, including Gemma, Vicuna, and Llama. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Synthesizing Tight Privacy and Accuracy Bounds via Weighted Model Counting

Authors: Lisa Oakley, Steven Holtzen, Alina Oprea

Abstract: Programmatically generating tight differential privacy (DP) bounds is a hard problem. Two core challenges are (1) finding expressive, compact, and efficient encodings of the distributions of DP algorithms, and (2) state space explosion stemming from the multiple quantifiers and relational properties of the DP definition. We address the first challenge by develo** a method for tight privacy and… ▽ More Programmatically generating tight differential privacy (DP) bounds is a hard problem. Two core challenges are (1) finding expressive, compact, and efficient encodings of the distributions of DP algorithms, and (2) state space explosion stemming from the multiple quantifiers and relational properties of the DP definition. We address the first challenge by develo** a method for tight privacy and accuracy bound synthesis using weighted model counting on binary decision diagrams, a state of the art technique from the artificial intelligence and automated reasoning communities for exactly computing probability distributions. We address the second challenge by develo** a framework for leveraging inherent symmetries in DP algorithms. Our solution benefits from ongoing research in probabilistic programming languages, allowing us to succinctly and expressively represent different DP algorithms with approachable language syntax that can be used by non-experts. We provide a detailed case study of our solution on the binary randomized response algorithm. We also evaluate an implementation of our solution using the Dice probabilistic programming language for the randomized response and truncated geometric above threshold algorithms. We compare to prior work on exact DP verification using Markov chain probabilistic model checking. Very few existing works consider mechanized analysis of accuracy guarantees for DP algorithms. We additionally provide a detailed analysis using our technique for finding tight accuracy bounds for DP algorithms. △ Less

Submitted 29 February, 2024; v1 submitted 26 February, 2024; originally announced February 2024.

User Inference Attacks on Large Language Models

Authors: Nikhil Kandpal, Krishna Pillutla, Alina Oprea, Peter Kairouz, Christopher A. Choquette-Choo, Zheng Xu

Abstract: Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing u… ▽ More Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications. In this paper, we study the privacy implications of fine-tuning LLMs on user data. To this end, we consider a realistic threat model, called user inference, wherein an attacker infers whether or not a user's data was used for fine-tuning. We design attacks for performing user inference that require only black-box access to the fine-tuned LLM and a few samples from a user which need not be from the fine-tuning dataset. We find that LLMs are susceptible to user inference across a variety of fine-tuning datasets, at times with near perfect attack success rates. Further, we theoretically and empirically investigate the properties that make users vulnerable to user inference, finding that outlier users, users with identifiable shared features between examples, and users that contribute a large fraction of the fine-tuning data are most susceptible to attack. Based on these findings, we identify several methods for mitigating user inference including training with example-level differential privacy, removing within-user duplicate examples, and reducing a user's contribution to the training data. While these techniques provide partial mitigation of user inference, we highlight the need to develop methods to fully protect fine-tuned LLMs against this privacy risk. △ Less

Submitted 23 February, 2024; v1 submitted 13 October, 2023; originally announced October 2023.

Comments: v2 contains experiments on additional datasets and differential privacy

Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning

Authors: Harsh Chaudhari, Giorgio Severi, Alina Oprea, Jonathan Ullman

Abstract: The integration of machine learning (ML) in numerous critical applications introduces a range of privacy concerns for individuals who provide their datasets for model training. One such privacy risk is Membership Inference (MI), in which an attacker seeks to determine whether a particular data sample was included in the training dataset of a model. Current state-of-the-art MI attacks capitalize on… ▽ More The integration of machine learning (ML) in numerous critical applications introduces a range of privacy concerns for individuals who provide their datasets for model training. One such privacy risk is Membership Inference (MI), in which an attacker seeks to determine whether a particular data sample was included in the training dataset of a model. Current state-of-the-art MI attacks capitalize on access to the model's predicted confidence scores to successfully perform membership inference, and employ data poisoning to further enhance their effectiveness. In this work, we focus on the less explored and more realistic label-only setting, where the model provides only the predicted label on a queried sample. We show that existing label-only MI attacks are ineffective at inferring membership in the low False Positive Rate (FPR) regime. To address this challenge, we propose a new attack Chameleon that leverages a novel adaptive data poisoning strategy and an efficient query selection method to achieve significantly more accurate membership inference than existing label-only attacks, especially at low FPRs. △ Less

Submitted 16 January, 2024; v1 submitted 5 October, 2023; originally announced October 2023.

Comments: To appear at International Conference on Learning Representations (ICLR) 2024

Dropout Attacks

Authors: Andrew Yuan, Alina Oprea, Cheng Tan

Abstract: Dropout is a common operator in deep learning, aiming to prevent overfitting by randomly drop** neurons during training. This paper introduces a new family of poisoning attacks against neural networks named DROPOUTATTACK. DROPOUTATTACK attacks the dropout operator by manipulating the selection of neurons to drop instead of selecting them uniformly at random. We design, implement, and evaluate fo… ▽ More Dropout is a common operator in deep learning, aiming to prevent overfitting by randomly drop** neurons during training. This paper introduces a new family of poisoning attacks against neural networks named DROPOUTATTACK. DROPOUTATTACK attacks the dropout operator by manipulating the selection of neurons to drop instead of selecting them uniformly at random. We design, implement, and evaluate four DROPOUTATTACK variants that cover a broad range of scenarios. These attacks can slow or stop training, destroy prediction accuracy of target classes, and sabotage either precision or recall of a target class. In our experiments of training a VGG-16 model on CIFAR-100, our attack can reduce the precision of the victim class by 34.6% (from 81.7% to 47.1%) without incurring any degradation in model accuracy △ Less

Submitted 4 September, 2023; originally announced September 2023.

Poisoning Network Flow Classifiers

Authors: Giorgio Severi, Simona Boboila, Alina Oprea, John Holodnak, Kendra Kratkiewicz, Jason Matterer

Abstract: As machine learning (ML) classifiers increasingly oversee the automated monitoring of network traffic, studying their resilience against adversarial attacks becomes critical. This paper focuses on poisoning attacks, specifically backdoor attacks, against network traffic flow classifiers. We investigate the challenging scenario of clean-label poisoning where the adversary's capabilities are constra… ▽ More As machine learning (ML) classifiers increasingly oversee the automated monitoring of network traffic, studying their resilience against adversarial attacks becomes critical. This paper focuses on poisoning attacks, specifically backdoor attacks, against network traffic flow classifiers. We investigate the challenging scenario of clean-label poisoning where the adversary's capabilities are constrained to tampering only with the training data - without the ability to arbitrarily modify the training labels or any other component of the training process. We describe a trigger crafting strategy that leverages model interpretability techniques to generate trigger patterns that are effective even at very low poisoning rates. Finally, we design novel strategies to generate stealthy triggers, including an approach based on generative Bayesian network models, with the goal of minimizing the conspicuousness of the trigger, and thus making detection of an ongoing poisoning campaign more challenging. Our findings provide significant insights into the feasibility of poisoning attacks on network traffic classifiers used in multiple scenarios, including detecting malicious communication and application classification. △ Less

Submitted 2 June, 2023; originally announced June 2023.

Comments: 14 pages, 8 figures

TMI! Finetuned Models Leak Private Information from their Pretraining Data

Authors: John Abascal, Stanley Wu, Alina Oprea, Jonathan Ullman

Abstract: Transfer learning has become an increasingly popular technique in machine learning as a way to leverage a pretrained model trained for one task to assist with building a finetuned model for a related task. This paradigm has been especially popular for $\textit{privacy}$ in machine learning, where the pretrained model is considered public, and only the data for finetuning is considered sensitive. H… ▽ More Transfer learning has become an increasingly popular technique in machine learning as a way to leverage a pretrained model trained for one task to assist with building a finetuned model for a related task. This paradigm has been especially popular for $\textit{privacy}$ in machine learning, where the pretrained model is considered public, and only the data for finetuning is considered sensitive. However, there are reasons to believe that the data used for pretraining is still sensitive, making it essential to understand how much information the finetuned model leaks about the pretraining data. In this work we propose a new membership-inference threat model where the adversary only has access to the finetuned model and would like to infer the membership of the pretraining data. To realize this threat model, we implement a novel metaclassifier-based attack, $\textbf{TMI}$, that leverages the influence of memorized pretraining samples on predictions in the downstream task. We evaluate $\textbf{TMI}$ on both vision and natural language tasks across multiple transfer learning settings, including finetuning with differential privacy. Through our evaluation, we find that $\textbf{TMI}$ can successfully infer membership of pretraining examples using query access to the finetuned model. An open-source implementation of $\textbf{TMI}$ can be found $\href{https://github.com/johnmath/tmi-pets24}{\text{on GitHub}}$. △ Less

Submitted 21 March, 2024; v1 submitted 1 June, 2023; originally announced June 2023.

Unleashing the Power of Randomization in Auditing Differentially Private ML

Authors: Krishna Pillutla, Galen Andrew, Peter Kairouz, H. Brendan McMahan, Alina Oprea, Sewoong Oh

Abstract: We present a rigorous methodology for auditing differentially private machine learning algorithms by adding multiple carefully designed examples called canaries. We take a first principles approach based on three key components. First, we introduce Lifted Differential Privacy (LiDP) that expands the definition of differential privacy to handle randomized datasets. This gives us the freedom to desi… ▽ More We present a rigorous methodology for auditing differentially private machine learning algorithms by adding multiple carefully designed examples called canaries. We take a first principles approach based on three key components. First, we introduce Lifted Differential Privacy (LiDP) that expands the definition of differential privacy to handle randomized datasets. This gives us the freedom to design randomized canaries. Second, we audit LiDP by trying to distinguish between the model trained with $K$ canaries versus $K - 1$ canaries in the dataset, leaving one canary out. By drawing the canaries i.i.d., LiDP can leverage the symmetry in the design and reuse each privately trained model to run multiple statistical tests, one for each canary. Third, we introduce novel confidence intervals that take advantage of the multiple test statistics by adapting to the empirical higher-order correlations. Together, this new recipe demonstrates significant improvements in sample complexity, both theoretically and empirically, using synthetic and real data. Further, recent advances in designing stronger canaries can be readily incorporated into the new framework. △ Less

Submitted 28 May, 2023; originally announced May 2023.

One-shot Empirical Privacy Estimation for Federated Learning

Authors: Galen Andrew, Peter Kairouz, Sewoong Oh, Alina Oprea, H. Brendan McMahan, Vinith M. Suriyakumar

Abstract: Privacy estimation techniques for differentially private (DP) algorithms are useful for comparing against analytical bounds, or to empirically measure privacy loss in settings where known analytical bounds are not tight. However, existing privacy auditing techniques usually make strong assumptions on the adversary (e.g., knowledge of intermediate model iterates or the training data distribution),… ▽ More Privacy estimation techniques for differentially private (DP) algorithms are useful for comparing against analytical bounds, or to empirically measure privacy loss in settings where known analytical bounds are not tight. However, existing privacy auditing techniques usually make strong assumptions on the adversary (e.g., knowledge of intermediate model iterates or the training data distribution), are tailored to specific tasks, model architectures, or DP algorithm, and/or require retraining the model many times (typically on the order of thousands). These shortcomings make deploying such techniques at scale difficult in practice, especially in federated settings where model training can take days or weeks. In this work, we present a novel "one-shot" approach that can systematically address these challenges, allowing efficient auditing or estimation of the privacy loss of a model during the same, single training run used to fit model parameters, and without requiring any a priori knowledge about the model architecture, task, or DP training algorithm. We show that our method provides provably correct estimates for the privacy loss under the Gaussian mechanism, and we demonstrate its performance on well-established FL benchmark datasets under several adversarial threat models. △ Less

Submitted 18 April, 2024; v1 submitted 6 February, 2023; originally announced February 2023.

Comments: Final revision, oral presentation at ICLR 2024

Backdoor Attacks in Peer-to-Peer Federated Learning

Authors: Gokberk Yar, Simona Boboila, Cristina Nita-Rotaru, Alina Oprea

Abstract: Most machine learning applications rely on centralized learning processes, opening up the risk of exposure of their training datasets. While federated learning (FL) mitigates to some extent these privacy risks, it relies on a trusted aggregation server for training a shared global model. Recently, new distributed learning architectures based on Peer-to-Peer Federated Learning (P2PFL) offer advanta… ▽ More Most machine learning applications rely on centralized learning processes, opening up the risk of exposure of their training datasets. While federated learning (FL) mitigates to some extent these privacy risks, it relies on a trusted aggregation server for training a shared global model. Recently, new distributed learning architectures based on Peer-to-Peer Federated Learning (P2PFL) offer advantages in terms of both privacy and reliability. Still, their resilience to poisoning attacks during training has not been investigated. In this paper, we propose new backdoor attacks for P2PFL that leverage structural graph properties to select the malicious nodes, and achieve high attack success, while remaining stealthy. We evaluate our attacks under various realistic conditions, including multiple graph topologies, limited adversarial visibility of the network, and clients with non-IID data. Finally, we show the limitations of existing defenses adapted from FL and design a new defense that successfully mitigates the backdoor attacks, without an impact on model accuracy. △ Less

Submitted 25 June, 2023; v1 submitted 23 January, 2023; originally announced January 2023.

Measurement of the $^{14}$N(n,p)$^{14}$C cross section at the CERN n_TOF facility from sub-thermal energy to 800 keV

Authors: P. Torres-Sánchez, J. Praena, I. Porras, M. Sabaté-Gilarte, C. Lederer-Woods, O. Aberle, V. Alcayne, S. Amaducci, J. Andrzejewski, L. Audouin, V. Bécares, V. Babiano-Suarez, M. Bacak, M. Barbagallo, F. Bečvář, G. Bellia, E. Berthoumieux, J. Billowes, D. Bosnar, A. Brown, M. Busso, M. Caamaño, L. Caballero, F. Calviño, M. Calviani , et al. (107 additional authors not shown)

Abstract: Background: The $^{14}$N(n,p)$^{14}$C reaction is of interest in neutron capture therapy, where nitrogen-related dose is the main component due to low-energy neutrons, and in astrophysics, where 14N acts as a neutron poison in the s-process. Several discrepancies remain between the existing data obtained in partial energy ranges: thermal energy, keV region and resonance region. Purpose: Measuring… ▽ More Background: The $^{14}$N(n,p)$^{14}$C reaction is of interest in neutron capture therapy, where nitrogen-related dose is the main component due to low-energy neutrons, and in astrophysics, where 14N acts as a neutron poison in the s-process. Several discrepancies remain between the existing data obtained in partial energy ranges: thermal energy, keV region and resonance region. Purpose: Measuring the 14N(n,p)14C cross section from thermal to the resonance region in a single measurement for the first time, including characterization of the first resonances, and providing calculations of Maxwellian averaged cross sections (MACS). Method: Time-of-flight technique. Experimental Area 2 (EAR-2) of the neutron time-of-flight (n_TOF) facility at CERN. $^{10}$B(n,$α$)$^7$Li and $^{235}$U(n,f) reactions as references. Two detection systems running simultaneously, one on-beam and another off-beam. Description of the resonances with the R-matrix code sammy. Results: The cross section has been measured from sub-thermal energy to 800 keV resolving the two first resonances (at 492.7 and 644 keV). A thermal cross-section (1.809$\pm$0.045 b) lower than the two most recent measurements by slightly more than one standard deviation, but in line with the ENDF/B-VIII.0 and JEFF-3.3 evaluations has been obtained. A 1/v energy dependence of the cross section has been confirmed up to tens of keV neutron energy. The low energy tail of the first resonance at 492.7 keV is lower than suggested by evaluated values, while the overall resonance strength agrees with evaluations. Conclusions: Our measurement has allowed to determine the $^{14}$N(n,p) cross-section over a wide energy range for the first time. We have obtained cross-sections with high accuracy (2.5 %) from sub-thermal energy to 800 keV and used these data to calculate the MACS for kT = 5 to kT = 100 keV. △ Less

Submitted 9 December, 2022; originally announced December 2022.

Comments: 18 pages, 15 figures, 4 tables

Bad Citrus: Reducing Adversarial Costs with Model Distances

Authors: Giorgio Severi, Will Pearce, Alina Oprea

Abstract: Recent work by Jia et al., showed the possibility of effectively computing pairwise model distances in weight space, using a model explanation technique known as LIME. This method requires query-only access to the two models under examination. We argue this insight can be leveraged by an adversary to reduce the net cost (number of queries) of launching an evasion campaign against a deployed model.… ▽ More Recent work by Jia et al., showed the possibility of effectively computing pairwise model distances in weight space, using a model explanation technique known as LIME. This method requires query-only access to the two models under examination. We argue this insight can be leveraged by an adversary to reduce the net cost (number of queries) of launching an evasion campaign against a deployed model. We show that there is a strong negative correlation between the success rate of adversarial transfer and the distance between the victim model and the surrogate used to generate the evasive samples. Thus, we propose and evaluate a method to reduce adversarial costs by finding the closest surrogate model for adversarial transfer. △ Less

Submitted 6 October, 2022; originally announced October 2022.

Network-Level Adversaries in Federated Learning

Authors: Giorgio Severi, Matthew Jagielski, Gökberk Yar, Yuxuan Wang, Alina Oprea, Cristina Nita-Rotaru

Abstract: Federated learning is a popular strategy for training models on distributed, sensitive data, while preserving data privacy. Prior work identified a range of security threats on federated learning protocols that poison the data or the model. However, federated learning is a networked system where the communication between clients and server plays a critical role for the learning task performance. W… ▽ More Federated learning is a popular strategy for training models on distributed, sensitive data, while preserving data privacy. Prior work identified a range of security threats on federated learning protocols that poison the data or the model. However, federated learning is a networked system where the communication between clients and server plays a critical role for the learning task performance. We highlight how communication introduces another vulnerability surface in federated learning and study the impact of network-level adversaries on training federated learning models. We show that attackers drop** the network traffic from carefully selected clients can significantly decrease model accuracy on a target population. Moreover, we show that a coordinated poisoning campaign from a few clients can amplify the drop** attacks. Finally, we develop a server-side defense which mitigates the impact of our attacks by identifying and up-sampling clients likely to positively contribute towards target accuracy. We comprehensively evaluate our attacks and defenses on three datasets, assuming encrypted communication channels and attackers with partial visibility of the network. △ Less

Submitted 26 August, 2022; originally announced August 2022.

Comments: 12 pages. Appearing at IEEE CNS 2022

SNAP: Efficient Extraction of Private Properties with Poisoning

Authors: Harsh Chaudhari, John Abascal, Alina Oprea, Matthew Jagielski, Florian Tramèr, Jonathan Ullman

Abstract: Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model. Such attacks have privacy implications for data owners sharing their datasets to train machine learning models. Several existing approaches for property inference attacks against deep neural networks have been proposed, but they all rely on the attacker training a large… ▽ More Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model. Such attacks have privacy implications for data owners sharing their datasets to train machine learning models. Several existing approaches for property inference attacks against deep neural networks have been proposed, but they all rely on the attacker training a large number of shadow models, which induces a large computational overhead. In this paper, we consider the setting of property inference attacks in which the attacker can poison a subset of the training dataset and query the trained target model. Motivated by our theoretical analysis of model confidences under poisoning, we design an efficient property inference attack, SNAP, which obtains higher attack success and requires lower amounts of poisoning than the state-of-the-art poisoning-based property inference attack by Mahloujifar et al. For example, on the Census dataset, SNAP achieves 34% higher success rate than Mahloujifar et al. while being 56.5x faster. We also extend our attack to infer whether a certain property was present at all during training and estimate the exact proportion of a property of interest efficiently. We evaluate our attack on several properties of varying proportions from four datasets and demonstrate SNAP's generality and effectiveness. An open-source implementation of SNAP can be found at https://github.com/johnmath/snap-sp23. △ Less

Submitted 21 June, 2023; v1 submitted 25 August, 2022; originally announced August 2022.

Comments: 28 pages, 16 figures

Black-box Attacks Against Neural Binary Function Detection

Authors: Joshua Bundt, Michael Davinroy, Ioannis Agadakos, Alina Oprea, William Robertson

Abstract: Binary analyses based on deep neural networks (DNNs), or neural binary analyses (NBAs), have become a hotly researched topic in recent years. DNNs have been wildly successful at pushing the performance and accuracy envelopes in the natural language and image processing domains. Thus, DNNs are highly promising for solving binary analysis problems that are typically hard due to a lack of complete in… ▽ More Binary analyses based on deep neural networks (DNNs), or neural binary analyses (NBAs), have become a hotly researched topic in recent years. DNNs have been wildly successful at pushing the performance and accuracy envelopes in the natural language and image processing domains. Thus, DNNs are highly promising for solving binary analysis problems that are typically hard due to a lack of complete information resulting from the lossy compilation process. Despite this promise, it is unclear that the prevailing strategy of repurposing embeddings and model architectures originally developed for other problem domains is sound given the adversarial contexts under which binary analysis often operates. In this paper, we empirically demonstrate that the current state of the art in neural function boundary detection is vulnerable to both inadvertent and deliberate adversarial attacks. We proceed from the insight that current generation NBAs are built upon embeddings and model architectures intended to solve syntactic problems. We devise a simple, reproducible, and scalable black-box methodology for exploring the space of inadvertent attacks - instruction sequences that could be emitted by common compiler toolchains and configurations - that exploits this syntactic design focus. We then show that these inadvertent misclassifications can be exploited by an attacker, serving as the basis for a highly effective black-box adversarial example generation process. We evaluate this methodology against two state-of-the-art neural function boundary detectors: XDA and DeepDi. We conclude with an analysis of the evaluation data and recommendations for how future research might avoid succumbing to similar attacks. △ Less

Submitted 31 July, 2023; v1 submitted 24 August, 2022; originally announced August 2022.

Comments: 16 pages

Journal ref: The 26th International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2023), October 16-18, 2023

Modeling Self-Propagating Malware with Epidemiological Models

Authors: Alesia Chernikova, Nicolò Gozzi, Simona Boboila, Nicola Perra, Tina Eliassi-Rad, Alina Oprea

Abstract: Self-propagating malware (SPM) has recently resulted in large financial losses and high social impact, with well-known campaigns such as WannaCry and Colonial Pipeline being able to propagate rapidly on the Internet and cause service disruptions. To date, the propagation behavior of SPM is still not well understood, resulting in the difficulty of defending against these cyber threats. To address t… ▽ More Self-propagating malware (SPM) has recently resulted in large financial losses and high social impact, with well-known campaigns such as WannaCry and Colonial Pipeline being able to propagate rapidly on the Internet and cause service disruptions. To date, the propagation behavior of SPM is still not well understood, resulting in the difficulty of defending against these cyber threats. To address this gap, in this paper we perform a comprehensive analysis of a newly proposed epidemiological model for SPM propagation, Susceptible-Infected-Infected Dormant-Recovered (SIIDR). We perform a theoretical analysis of the stability of the SIIDR model and derive its basic reproduction number by representing it as a system of Ordinary Differential Equations with continuous time. We obtain access to 15 WananCry attack traces generated under various conditions, derive the model's transition rates, and show that SIIDR fits best the real data. We find that the SIIDR model outperforms more established compartmental models from epidemiology, such as SI, SIS, and SIR, at modeling SPM propagation. △ Less

Submitted 3 August, 2023; v1 submitted 5 August, 2022; originally announced August 2022.

Cyber Network Resilience against Self-Propagating Malware Attacks

Authors: Alesia Chernikova, Nicolò Gozzi, Simona Boboila, Priyanka Angadi, John Loughner, Matthew Wilden, Nicola Perra, Tina Eliassi-Rad, Alina Oprea

Abstract: Self-propagating malware (SPM) has led to huge financial losses, major data breaches, and widespread service disruptions in recent years. In this paper, we explore the problem of develo** cyber resilient systems capable of mitigating the spread of SPM attacks. We begin with an in-depth study of a well-known self-propagating malware, WannaCry, and present a compartmental model called SIIDR that a… ▽ More Self-propagating malware (SPM) has led to huge financial losses, major data breaches, and widespread service disruptions in recent years. In this paper, we explore the problem of develo** cyber resilient systems capable of mitigating the spread of SPM attacks. We begin with an in-depth study of a well-known self-propagating malware, WannaCry, and present a compartmental model called SIIDR that accurately captures the behavior observed in real-world attack traces. Next, we investigate ten cyber defense techniques, including existing edge and node hardening strategies, as well as newly developed methods based on reconfiguring network communication (NodeSplit) and isolating communities. We evaluate all defense strategies in detail using six real-world communication graphs collected from a large retail network and compare their performance across a wide range of attacks and network topologies. We show that several of these defenses are able to efficiently reduce the spread of SPM attacks modeled with SIIDR. For instance, given a strong attack that infects 97% of nodes when no defense is employed, strategically securing a small number of nodes (0.08%) reduces the infection footprint in one of the networks down to 1%. △ Less

Submitted 8 October, 2022; v1 submitted 27 June, 2022; originally announced June 2022.

Comments: 20 pages

CELEST: Federated Learning for Globally Coordinated Threat Detection

Authors: Talha Ongun, Simona Boboila, Alina Oprea, Tina Eliassi-Rad, Jason Hiser, Jack Davidson

Abstract: The cyber-threat landscape has evolved tremendously in recent years, with new threat variants emerging daily, and large-scale coordinated campaigns becoming more prevalent. In this study, we propose CELEST (CollaborativE LEarning for Scalable Threat detection, a federated machine learning framework for global threat detection over HTTP, which is one of the most commonly used protocols for malware… ▽ More The cyber-threat landscape has evolved tremendously in recent years, with new threat variants emerging daily, and large-scale coordinated campaigns becoming more prevalent. In this study, we propose CELEST (CollaborativE LEarning for Scalable Threat detection, a federated machine learning framework for global threat detection over HTTP, which is one of the most commonly used protocols for malware dissemination and communication. CELEST leverages federated learning in order to collaboratively train a global model across multiple clients who keep their data locally, thus providing increased privacy and confidentiality assurances. Through a novel active learning component integrated with the federated learning technique, our system continuously discovers and learns the behavior of new, evolving, and globally-coordinated cyber threats. We show that CELEST is able to expose attacks that are largely invisible to individual organizations. For instance, in one challenging attack scenario with data exfiltration malware, the global model achieves a three-fold increase in Precision-Recall AUC compared to the local model. We also design a poisoning detection and mitigation method, DTrust, specifically designed for federated learning in the collaborative threat detection domain. DTrust successfully detects poisoning clients using the feedback from participating clients to investigate and remove them from the training process. We deploy CELEST on two university networks and show that it is able to detect the malicious HTTP communication with high precision and low false positive rates. Furthermore, during its deployment, CELEST detected a set of previously unknown 42 malicious URLs and 20 malicious domains in one day, which were confirmed to be malicious by VirusTotal. △ Less

Submitted 16 March, 2023; v1 submitted 23 May, 2022; originally announced May 2022.

SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning

Authors: Harsh Chaudhari, Matthew Jagielski, Alina Oprea

Abstract: Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, by design, MPC protocols faithfully compute the training functionality, which the adversarial ML community has shown to leak private information and can be tampered with in poisoning attacks. In this work, we argue t… ▽ More Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, by design, MPC protocols faithfully compute the training functionality, which the adversarial ML community has shown to leak private information and can be tampered with in poisoning attacks. In this work, we argue that model ensembles, implemented in our framework called SafeNet, are a highly MPC-amenable way to avoid many adversarial ML attacks. The natural partitioning of data amongst owners in MPC training allows this approach to be highly scalable at training time, provide provable protection from poisoning attacks, and provably defense against a number of privacy attacks. We demonstrate SafeNet's efficiency, accuracy, and resilience to poisoning on several machine learning datasets and models trained in end-to-end and transfer learning scenarios. For instance, SafeNet reduces backdoor attack success significantly, while achieving $39\times$ faster training and $36 \times$ less communication than the four-party MPC framework of Dalskov et al. Our experiments show that ensembling retains these benefits even in many non-iid settings. The simplicity, cheap setup, and robustness properties of ensembling make it a strong first choice for training ML models privately in MPC. △ Less

Submitted 8 September, 2022; v1 submitted 20 May, 2022; originally announced May 2022.

How to Combine Membership-Inference Attacks on Multiple Updated Models

Authors: Matthew Jagielski, Stanley Wu, Alina Oprea, Jonathan Ullman, Roxana Geambasu

Abstract: A large body of research has shown that machine learning models are vulnerable to membership inference (MI) attacks that violate the privacy of the participants in the training data. Most MI research focuses on the case of a single standalone model, while production machine-learning platforms often update models over time, on data that often shifts in distribution, giving the attacker more informa… ▽ More A large body of research has shown that machine learning models are vulnerable to membership inference (MI) attacks that violate the privacy of the participants in the training data. Most MI research focuses on the case of a single standalone model, while production machine-learning platforms often update models over time, on data that often shifts in distribution, giving the attacker more information. This paper proposes new attacks that take advantage of one or more model updates to improve MI. A key part of our approach is to leverage rich information from standalone MI attacks mounted separately against the original and updated models, and to combine this information in specific ways to improve attack effectiveness. We propose a set of combination functions and tuning methods for each, and present both analytical and quantitative justification for various options. Our results on four public datasets show that our attacks are effective at using update information to give the adversary a significant advantage over attacks on standalone models, but also compared to a prior MI attack that takes advantage of model updates in a related machine-unlearning setting. We perform the first measurements of the impact of distribution shift on MI attacks with model updates, and show that a more drastic distribution shift results in significantly higher MI risk than a gradual shift. Our code is available at https://www.github.com/stanleykywu/model-updates. △ Less

Submitted 12 May, 2022; originally announced May 2022.

Comments: 31 pages, 9 figures

Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

Authors: Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A. Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, Fabio Roli

Abstract: The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to com… ▽ More The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model's performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the last 15 years. We start by categorizing the current threat models and attacks, and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning, and shed light on the current limitations and open research questions in this research field. △ Less

Submitted 9 March, 2023; v1 submitted 4 May, 2022; originally announced May 2022.

Comments: 35 pages, Accepted at ACM Computing Surveys

High accuracy, high resolution 235U(n,f) cross section from n_TOF (CERN) in the thermal to 10 keV energy range

Authors: n_TOF collaboration, :, M. Mastromarco, S. Amaducci, N. Colonna, P. Finocchiaro, L. Cosentino, O. Aberle, J. Andrzejewski, L. Audouin, M. Bacak, J. Balibrea, M. Barbagallo, F. Bečvář, E. Berthoumieux, J. Billowes, D. Bosnar, A. Brown, M. Caamaño, F. Calviño, M. Calviani, D. Cano-Ott, R. Cardella, A. Casanovas, F. Cerutti , et al. (98 additional authors not shown)

Abstract: The 235U(n,f) cross section was measured in a wide energy range (25 meV - 170 keV) at the n_TOF facility at CERN, relative to 6Li(n,t) and 10B(n,alpha) standard reactions, with high resolution and accuracy, with a setup based on a stack of six samples and six silicon detectors placed in the neutron beam. In this paper we report on the results in the region between thermal and 10 keV neutron energy… ▽ More The 235U(n,f) cross section was measured in a wide energy range (25 meV - 170 keV) at the n_TOF facility at CERN, relative to 6Li(n,t) and 10B(n,alpha) standard reactions, with high resolution and accuracy, with a setup based on a stack of six samples and six silicon detectors placed in the neutron beam. In this paper we report on the results in the region between thermal and 10 keV neutron energy. A resonance analysis has been performed up to 200 eV, with the code SAMMY. The resulting fission kernels are compared with the ones extracted on the basis of the resonance parameters of the most recent major evaluated data libraries. A comparison of the n_TOF data with the evaluated cross sections is also performed from thermal to 10 keV neutron energy for the energy-averaged cross section in energy groups of suitably chosen width. A good agreement is found in average between the new results and the latest evaluated data files ENDF-B/VIII and JEFF-3.3, as well as with respect to the IAEA reference files. However, some discrepancies are still present in some specific energy regions. The new dataset here presented, characterized by unprecedented resolution and accuracy, can help improving the evaluations in the Resolved Resonance Region and up to 10 keV, and reduce the uncertainties that affect this region. △ Less

Submitted 2 February, 2022; originally announced February 2022.

PORTFILER: Port-Level Network Profiling for Self-Propagating Malware Detection

Authors: Talha Ongun, Oliver Spohngellert, Benjamin Miller, Simona Boboila, Alina Oprea, Tina Eliassi-Rad, Jason Hiser, Alastair Nottingham, Jack Davidson, Malathi Veeraraghavan

Abstract: Recent self-propagating malware (SPM) campaigns compromised hundred of thousands of victim machines on the Internet. It is challenging to detect these attacks in their early stages, as adversaries utilize common network services, use novel techniques, and can evade existing detection mechanisms. We propose PORTFILER (PORT-Level Network Traffic ProFILER), a new machine learning system applied to ne… ▽ More Recent self-propagating malware (SPM) campaigns compromised hundred of thousands of victim machines on the Internet. It is challenging to detect these attacks in their early stages, as adversaries utilize common network services, use novel techniques, and can evade existing detection mechanisms. We propose PORTFILER (PORT-Level Network Traffic ProFILER), a new machine learning system applied to network traffic for detecting SPM attacks. PORTFILER extracts port-level features from the Zeek connection logs collected at a border of a monitored network, applies anomaly detection techniques to identify suspicious events, and ranks the alerts across ports for investigation by the Security Operations Center (SOC). We propose a novel ensemble methodology for aggregating individual models in PORTFILER that increases resilience against several evasion strategies compared to standard ML baselines. We extensively evaluate PORTFILER on traffic collected from two university networks, and show that it can detect SPM attacks with different patterns, such as WannaCry and Mirai, and performs well under evasion. Ranking across ports achieves precision over 0.94 with low false positive rates in the top ranked alerts. When deployed on the university networks, PORTFILER detected anomalous SPM-like activity on one of the campus networks, confirmed by the university SOC as malicious. PORTFILER also detected a Mirai attack recreated on the two university networks with higher precision and recall than deep-learning-based autoencoder methods. △ Less

Submitted 24 May, 2022; v1 submitted 27 December, 2021; originally announced December 2021.

Comments: An earlier version is accepted to be published in IEEE Conference on Communications and Network Security (CNS) 2021

Living-Off-The-Land Command Detection Using Active Learning

Authors: Talha Ongun, Jack W. Stokes, Jonathan Bar Or, Ke Tian, Farid Tajaddodianfar, Joshua Neil, Christian Seifert, Alina Oprea, John C. Platt

Abstract: In recent years, enterprises have been targeted by advanced adversaries who leverage creative ways to infiltrate their systems and move laterally to gain access to critical data. One increasingly common evasive method is to hide the malicious activity behind a benign program by using tools that are already installed on user computers. These programs are usually part of the operating system distrib… ▽ More In recent years, enterprises have been targeted by advanced adversaries who leverage creative ways to infiltrate their systems and move laterally to gain access to critical data. One increasingly common evasive method is to hide the malicious activity behind a benign program by using tools that are already installed on user computers. These programs are usually part of the operating system distribution or another user-installed binary, therefore this type of attack is called "Living-Off-The-Land". Detecting these attacks is challenging, as adversaries may not create malicious files on the victim computers and anti-virus scans fail to detect them. We propose the design of an Active Learning framework called LOLAL for detecting Living-Off-the-Land attacks that iteratively selects a set of uncertain and anomalous samples for labeling by a human analyst. LOLAL is specifically designed to work well when a limited number of labeled samples are available for training machine learning models to detect attacks. We investigate methods to represent command-line text using word-embedding techniques, and design ensemble boosting classifiers to distinguish malicious and benign samples based on the embedding representation. We leverage a large, anonymized dataset collected by an endpoint security product and demonstrate that our ensemble classifiers achieve an average F1 score of 0.96 at classifying different attack classes. We show that our active learning method consistently improves the classifier performance, as more training data is labeled, and converges in less than 30 iterations when starting with a small number of labeled instances. △ Less

Submitted 29 November, 2021; originally announced November 2021.

Comments: 14 pages, published in RAID 2021

Adversarial Robustness Verification and Attack Synthesis in Stochastic Systems

Authors: Lisa Oakley, Alina Oprea, Stavros Tripakis

Abstract: Probabilistic model checking is a useful technique for specifying and verifying properties of stochastic systems including randomized protocols and reinforcement learning models. Existing methods rely on the assumed structure and probabilities of certain system transitions. These assumptions may be incorrect, and may even be violated by an adversary who gains control of system components. In thi… ▽ More Probabilistic model checking is a useful technique for specifying and verifying properties of stochastic systems including randomized protocols and reinforcement learning models. Existing methods rely on the assumed structure and probabilities of certain system transitions. These assumptions may be incorrect, and may even be violated by an adversary who gains control of system components. In this paper, we develop a formal framework for adversarial robustness in systems modeled as discrete time Markov chains (DTMCs). We base our framework on existing methods for verifying probabilistic temporal logic properties and extend it to include deterministic, memoryless policies acting in Markov decision processes (MDPs). Our framework includes a flexible approach for specifying structure-preserving and non structure-preserving adversarial models. We outline a class of threat models under which adversaries can perturb system transitions, constrained by an $\varepsilon$ ball around the original transition probabilities. We define three main DTMC adversarial robustness problems: adversarial robustness verification, maximal $δ$ synthesis, and worst case attack synthesis. We present two optimization-based solutions to these three problems, leveraging traditional and parametric probabilistic model checking techniques. We then evaluate our solutions on two stochastic protocols and a collection of Grid World case studies, which model an agent acting in an environment described as an MDP. We find that the parametric solution results in fast computation for small parameter spaces. In the case of less restrictive (stronger) adversaries, the number of parameters increases, and directly computing property satisfaction probabilities is more scalable. We demonstrate the usefulness of our definitions and solutions by comparing system outcomes over various properties, threat models, and case studies. △ Less

Submitted 31 July, 2022; v1 submitted 5 October, 2021; originally announced October 2021.

Comments: To Appear, 35th IEEE Computer Security Foundations Symposium (2022)

Collaborative Information Sharing for ML-Based Threat Detection

Authors: Talha Ongun, Simona Boboila, Alina Oprea, Tina Eliassi-Rad, Alastair Nottingham, Jason Hiser, Jack Davidson

Abstract: Recently, coordinated attack campaigns started to become more widespread on the Internet. In May 2017, WannaCry infected more than 300,000 machines in 150 countries in a few days and had a large impact on critical infrastructure. Existing threat sharing platforms cannot easily adapt to emerging attack patterns. At the same time, enterprises started to adopt machine learning-based threat detection… ▽ More Recently, coordinated attack campaigns started to become more widespread on the Internet. In May 2017, WannaCry infected more than 300,000 machines in 150 countries in a few days and had a large impact on critical infrastructure. Existing threat sharing platforms cannot easily adapt to emerging attack patterns. At the same time, enterprises started to adopt machine learning-based threat detection tools in their local networks. In this paper, we pose the question: \emph{What information can defenders share across multiple networks to help machine learning-based threat detection adapt to new coordinated attacks?} We propose three information sharing methods across two networks, and show how the shared information can be used in a machine-learning network-traffic model to significantly improve its ability of detecting evasive self-propagating malware. △ Less

Submitted 23 April, 2021; originally announced April 2021.

Comments: 6 pages, 5 figures. To be published in AI4CS-SDM2021

On Generating and Labeling Network Traffic with Realistic, Self-Propagating Malware

Authors: Molly Buchanan, Jeffrey W. Collyer, Jack W. Davidson, Saikat Dey, Mark Gardner, Jason D. Hiser, Jeffry Lang, Alastair Nottingham, Alina Oprea

Abstract: Research and development of techniques which detect or remediate malicious network activity require access to diverse, realistic, contemporary data sets containing labeled malicious connections. In the absence of such data, said techniques cannot be meaningfully trained, tested, and evaluated. Synthetically produced data containing fabricated or merged network traffic is of limited value as it is… ▽ More Research and development of techniques which detect or remediate malicious network activity require access to diverse, realistic, contemporary data sets containing labeled malicious connections. In the absence of such data, said techniques cannot be meaningfully trained, tested, and evaluated. Synthetically produced data containing fabricated or merged network traffic is of limited value as it is easily distinguishable from real traffic by even simple machine learning (ML) algorithms. Real network data is preferable, but while ubiquitous is broadly both sensitive and lacking in ground truth labels, limiting its utility for ML research. This paper presents a multi-faceted approach to generating a data set of labeled malicious connections embedded within anonymized network traffic collected from large production networks. Real-world malware is defanged and introduced to simulated, secured nodes within those networks to generate realistic traffic while maintaining sufficient isolation to protect real data and infrastructure. Network sensor data, including this embedded malware traffic, is collected at a network edge and anonymized for research use. Network traffic was collected and produced in accordance with the aforementioned methods at two major educational institutions. The result is a highly realistic, long term, multi-institution data set with embedded data labels spanning over 1.5 trillion connections and over a petabyte of sensor log data. The usability of this data set is demonstrated by its utility to our artificial intelligence and machine learning (AI/ML) research program. △ Less

Submitted 27 May, 2022; v1 submitted 20 April, 2021; originally announced April 2021.

Comments: 4+2 pages, 3 figures, 1 table, for AI4CS-SDM21

Imaging neutron capture cross sections: i-TED proof-of-concept and future prospects based on Machine-Learning techniques

Authors: V. Babiano-Suárez, J. Lerendegui-Marco, J. Balibrea-Correa, L. Caballero, D. Calvo, I. Ladarescu, C. Domingo-Pardo, F. Calviño, A. Casanovas, A. Tarifeño-Saldivia, V. Alcayne, C. Guerrero, M. A. Millán-Callado, M. T. Rodríguez González, M. Barbagallo, O. Aberle, S. Amaducci, J. Andrzejewski, L. Audouin, M. Bacak, S. Bennett, E. Berthoumieux, J. Billowes, D. Bosnar, A. Brown , et al. (110 additional authors not shown)

Abstract: i-TED is an innovative detection system which exploits Compton imaging techniques to achieve a superior signal-to-background ratio in ($n,γ$) cross-section measurements using time-of-flight technique. This work presents the first experimental validation of the i-TED apparatus for high-resolution time-of-flight experiments and demonstrates for the first time the concept proposed for background reje… ▽ More i-TED is an innovative detection system which exploits Compton imaging techniques to achieve a superior signal-to-background ratio in ($n,γ$) cross-section measurements using time-of-flight technique. This work presents the first experimental validation of the i-TED apparatus for high-resolution time-of-flight experiments and demonstrates for the first time the concept proposed for background rejection. To this aim both $^{197}$Au($n,γ$) and $^{56}$Fe($n, γ$) reactions were measured at CERN n\_TOF using an i-TED demonstrator based on only three position-sensitive detectors. Two \cds detectors were also used to benchmark the performance of i-TED. The i-TED prototype built for this study shows a factor of $\sim$3 higher detection sensitivity than state-of-the-art \cds detectors in the $\sim$10~keV neutron energy range of astrophysical interest. This paper explores also the perspectives of further enhancement in performance attainable with the final i-TED array consisting of twenty position-sensitive detectors and new analysis methodologies based on Machine-Learning techniques. △ Less

Submitted 18 December, 2020; originally announced December 2020.

Comments: 16 pages, 16 figures

Extracting Training Data from Large Language Models

Authors: Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel

Abstract: It has become common to publish large (billion parameter) language models that have been trained on private datasets. This paper demonstrates that in such settings, an adversary can perform a training data extraction attack to recover individual training examples by querying the language model. We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and ar… ▽ More It has become common to publish large (billion parameter) language models that have been trained on private datasets. This paper demonstrates that in such settings, an adversary can perform a training data extraction attack to recover individual training examples by querying the language model. We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model's training data. These extracted examples include (public) personally identifiable information (names, phone numbers, and email addresses), IRC conversations, code, and 128-bit UUIDs. Our attack is possible even though each of the above sequences are included in just one document in the training data. We comprehensively evaluate our extraction attack to understand the factors that contribute to its success. Worryingly, we find that larger models are more vulnerable than smaller models. We conclude by drawing lessons and discussing possible safeguards for training large language models. △ Less

Submitted 15 June, 2021; v1 submitted 14 December, 2020; originally announced December 2020.

Conductance Model for Single-Crystalline/Compact Metal Oxide Gas Sensing Layers in the Non-Degenerate Limit: Example of Epitaxial SnO$_2$(101)

Authors: Cristian Simion, Federico Schipani, Alexandra Papadogianni, Adelina Stanoiu, Melanie Budde, Alexandru Oprea, Udo Weimar, Oliver Bierwagen, Nicolae Barsan

Abstract: Semiconducting metal oxide (SMOX)-based gas sensors are indispensable for safety and health applications, e.g. explosive, toxic gas alarms, controls for intake into car cabins and monitor for industrial processes. In the past, the sensor community has been studying polycrystalline materials as sensors where the porous and random microstructure of the SMOX does not allow a separation of the phenome… ▽ More Semiconducting metal oxide (SMOX)-based gas sensors are indispensable for safety and health applications, e.g. explosive, toxic gas alarms, controls for intake into car cabins and monitor for industrial processes. In the past, the sensor community has been studying polycrystalline materials as sensors where the porous and random microstructure of the SMOX does not allow a separation of the phenomena involved in the sensing process. This lead to conduction models that can model and predict the behavior of the overall response, but they were not capable of giving fundamental information regarding the basic mechanisms taking place. The study of epitaxial layers is the definite prove to clarify the different aspects and contributions of the sensing mechanisms that are not possible to do by studying a polycrystalline sample. A detailed analytical model for n and p-type single-crystalline/compact metal oxide gas sensors was developed that directly relates the conductance of the sample with changes in the surface electrostatic potential. Combined DC resistance and work function measurements were used in a compact SnO2 (101) layer in operando conditions that allowed us to check the validity of our model in the region where Boltzmann approximation holds to determine surface and bulk properties of the material. △ Less

Submitted 6 October, 2020; originally announced October 2020.

Shape Coexistence at Zero Spin in 64Ni Driven by the Monopole Tensor Interaction

Authors: N. Mărginean, D. Little, Y. Tsunoda, S. Leoni, R. V. F. Janssens, B. Fornal, T. Otsuka, C. Michelagnoli, L. Stan, F. C. L. Crespi, C. Costache, R. Lica, M. Sferrazza, A. Turturica, A. D. Ayangeakaa, K. Auranen, M. Barani, P. C. Bender, S. Bottoni, M. Boromiza, A. Bracco, S. Călinescu, C. M. Campbell, M. P. Carpenter, P. Chowdhury , et al. (53 additional authors not shown)

Abstract: The low-spin structure of the semimagic 64Ni nucleus has been considerably expanded: combining four experiments, several 0+ and 2+ excited states were identified below 4.5 MeV, and their properties established. The Monte Carlo shell model accounts for the results and unveils an unexpectedly complex landscape of coexisting shapes: a prolate 0+ excitation is located at a surprisingly high energy (34… ▽ More The low-spin structure of the semimagic 64Ni nucleus has been considerably expanded: combining four experiments, several 0+ and 2+ excited states were identified below 4.5 MeV, and their properties established. The Monte Carlo shell model accounts for the results and unveils an unexpectedly complex landscape of coexisting shapes: a prolate 0+ excitation is located at a surprisingly high energy (3463 keV), with a collective 2+ state 286 keV above it, the first such observation in Ni isotopes. The evolution in excitation energy of the prolate minimum across the neutron N = 40 subshell gap highlights the impact of the monopole interaction and its variation in strength with N. △ Less

Submitted 11 August, 2020; originally announced August 2020.

Journal ref: Physical Review Letters 125, 102502 (2020)

Subpopulation Data Poisoning Attacks

Authors: Matthew Jagielski, Giorgio Severi, Niklas Pousette Harger, Alina Oprea

Abstract: Machine learning systems are deployed in critical settings, but they might fail in unexpected ways, impacting the accuracy of their predictions. Poisoning attacks against machine learning induce adversarial modification of data used by a machine learning algorithm to selectively change its output when it is deployed. In this work, we introduce a novel data poisoning attack called a \emph{subpopula… ▽ More Machine learning systems are deployed in critical settings, but they might fail in unexpected ways, impacting the accuracy of their predictions. Poisoning attacks against machine learning induce adversarial modification of data used by a machine learning algorithm to selectively change its output when it is deployed. In this work, we introduce a novel data poisoning attack called a \emph{subpopulation attack}, which is particularly relevant when datasets are large and diverse. We design a modular framework for subpopulation attacks, instantiate it with different building blocks, and show that the attacks are effective for a variety of datasets and machine learning models. We further optimize the attacks in continuous domains using influence functions and gradient optimization methods. Compared to existing backdoor poisoning attacks, subpopulation attacks have the advantage of inducing misclassification in naturally distributed data points at inference time, making the attacks extremely stealthy. We also show that our attack strategy can be used to improve upon existing targeted attacks. We prove that, under some assumptions, subpopulation attacks are impossible to defend against, and empirically demonstrate the limitations of existing defenses against our attacks, highlighting the difficulty of protecting machine learning against this threat. △ Less

Submitted 12 May, 2021; v1 submitted 24 June, 2020; originally announced June 2020.

Comments: May12 update: add sever + backdoor defenses, comparison to witches' brew attack, better comparison to related work, transferability of representations for cmatch

With Great Dispersion Comes Greater Resilience: Efficient Poisoning Attacks and Defenses for Linear Regression Models

Authors: Jialin Wen, Benjamin Zi Hao Zhao, Minhui Xue, Alina Oprea, Haifeng Qian

Abstract: With the rise of third parties in the machine learning pipeline, the service provider in "Machine Learning as a Service" (MLaaS), or external data contributors in online learning, or the retraining of existing models, the need to ensure the security of the resulting machine learning models has become an increasingly important topic. The security community has demonstrated that without transparency… ▽ More With the rise of third parties in the machine learning pipeline, the service provider in "Machine Learning as a Service" (MLaaS), or external data contributors in online learning, or the retraining of existing models, the need to ensure the security of the resulting machine learning models has become an increasingly important topic. The security community has demonstrated that without transparency of the data and the resulting model, there exist many potential security risks, with new risks constantly being discovered. In this paper, we focus on one of these security risks -- poisoning attacks. Specifically, we analyze how attackers may interfere with the results of regression learning by poisoning the training datasets. To this end, we analyze and develop a new poisoning attack algorithm. Our attack, termed Nopt, in contrast with previous poisoning attack algorithms, can produce larger errors with the same proportion of poisoning data-points. Furthermore, we also significantly improve the state-of-the-art defense algorithm, termed TRIM, proposed by Jagielsk et al. (IEEE S&P 2018), by incorporating the concept of probability estimation of clean data-points into the algorithm. Our new defense algorithm, termed Proda, demonstrates an increased effectiveness in reducing errors arising from the poisoning dataset through optimizing ensemble models. We highlight that the time complexity of TRIM had not been estimated; however, we deduce from their work that TRIM can take exponential time complexity in the worst-case scenario, in excess of Proda's logarithmic time. The performance of both our proposed attack and defense algorithms is extensively evaluated on four real-world datasets of housing prices, loans, health care, and bike sharing services. We hope that our work will inspire future research to develop more robust learning algorithms immune to poisoning attacks. △ Less

Submitted 19 May, 2021; v1 submitted 21 June, 2020; originally announced June 2020.

Comments: Accepted to IEEE Transactions on Information Forensics and Security (TIFS) 2021

Auditing Differentially Private Machine Learning: How Private is Private SGD?

Authors: Matthew Jagielski, Jonathan Ullman, Alina Oprea

Abstract: We investigate whether Differentially Private SGD offers better privacy in practice than what is guaranteed by its state-of-the-art analysis. We do so via novel data poisoning attacks, which we show correspond to realistic privacy attacks. While previous work (Ma et al., arXiv 2019) proposed this connection between differential privacy and data poisoning as a defense against data poisoning, our us… ▽ More We investigate whether Differentially Private SGD offers better privacy in practice than what is guaranteed by its state-of-the-art analysis. We do so via novel data poisoning attacks, which we show correspond to realistic privacy attacks. While previous work (Ma et al., arXiv 2019) proposed this connection between differential privacy and data poisoning as a defense against data poisoning, our use as a tool for understanding the privacy of a specific mechanism is new. More generally, our work takes a quantitative, empirical approach to understanding the privacy afforded by specific implementations of differentially private algorithms that we believe has the potential to complement and influence analytical work on differential privacy. △ Less

Submitted 13 June, 2020; originally announced June 2020.

Explanation-Guided Backdoor Poisoning Attacks Against Malware Classifiers

Authors: Giorgio Severi, Jim Meyer, Scott Coull, Alina Oprea

Abstract: Training pipelines for machine learning (ML) based malware classification often rely on crowdsourced threat feeds, exposing a natural attack injection point. In this paper, we study the susceptibility of feature-based ML malware classifiers to backdoor poisoning attacks, specifically focusing on challenging "clean label" attacks where attackers do not control the sample labeling process. We propos… ▽ More Training pipelines for machine learning (ML) based malware classification often rely on crowdsourced threat feeds, exposing a natural attack injection point. In this paper, we study the susceptibility of feature-based ML malware classifiers to backdoor poisoning attacks, specifically focusing on challenging "clean label" attacks where attackers do not control the sample labeling process. We propose the use of techniques from explainable machine learning to guide the selection of relevant features and values to create effective backdoor triggers in a model-agnostic fashion. Using multiple reference datasets for malware classification, including Windows PE files, PDFs, and Android applications, we demonstrate effective attacks against a diverse set of machine learning models and evaluate the effect of various constraints imposed on the attacker. To demonstrate the feasibility of our backdoor attacks in practice, we create a watermarking utility for Windows PE files that preserves the binary's functionality, and we leverage similar behavior-preserving alteration methodologies for Android and PDF files. Finally, we experiment with potential defensive strategies and show the difficulties of completely defending against these attacks, especially when the attacks blend in with the legitimate sample distribution. △ Less

Submitted 10 January, 2021; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: 18 pages, 5 figures. To appear in USENIX Security 2021

Structural, optical, and electrical properties of unintentionally doped NiO layers grown on MgO by plasma-assisted molecular beam epitaxy

Authors: Melanie Budde, Carsten Tschammer, Philipp Franz, Johannes Feldl, Manfred Ramsteiner, Rüdiger Goldhahn, Martin Feneberg, Nicolae Barsan, Alexandru Oprea, Oliver Bierwagen

Abstract: NiO layers were grown on MgO(100), MgO(110), and MgO(111) substrates by plasma-assisted molecular beam epitaxy under Ni-flux limited growth conditions. Single crystalline growth with a cube-on-cube epitaxial relationship was confirmed by X-ray diffraction measurements for all used growth conditions and substrates except MgO(111). A detailed growth series on MgO(100) was prepared using substrate te… ▽ More NiO layers were grown on MgO(100), MgO(110), and MgO(111) substrates by plasma-assisted molecular beam epitaxy under Ni-flux limited growth conditions. Single crystalline growth with a cube-on-cube epitaxial relationship was confirmed by X-ray diffraction measurements for all used growth conditions and substrates except MgO(111). A detailed growth series on MgO(100) was prepared using substrate temperatures ranging from 20 °C to 900 °C to investigate the influence on the layer characteristics. Energy-dispersive X-ray spectroscopy indicated close-to-stoichiometric layers with an oxygen content of ~47 at. % and ~50 at. % grown under low and high O-flux, respectively. All NiO layers had a root-mean-square surface roughness below 1 nm, measured by atomic force microscopy, except for rougher layers grown at 900 °C or using molecular oxygen. Growth at 900 °C led to a significant diffusion of Mg from the substrate into the film. The relative intensity of the quasi-forbidden one-phonon Raman peak is introduced as a gauge of the crystal quality, indicating the highest layer quality for growth at low oxygen flux and high growth temperature, likely due to the resulting high adatom diffusion length during growth. The optical and electrical properties were investigated by spectroscopic ellipsometry and resistance measurements, respectively. All NiO layers were transparent with an optical bandgap around 3.6 eV and semi-insulating at room temperature. However, changes upon exposure to reducing or oxidizing gases of the resistance of a representative layer at elevated temperature were able to confirm p-type conductivity, highlighting their suitability as a model system for research on oxide-based gas sensing. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Journal ref: Journal of Applied Physics 123, 195301 (2018)

Review and new concepts for neutron-capture measurements of astrophysical interest

Authors: C. Domingo-Pardo, V. Babiano-Suarez, J. Balibrea-Correa, L. Caballero, I. Ladarescu, J. Lerendegui-Marco, J. L. Tain, F. Calviño, A. Casanovas, A. Segarra, A. E. Tarifeño-Saldivia, C. Guerrero, M. A. Millán-Callado, J. M. Quesada, M. T. Rodríguez-González, O. Aberle, V. Alcayne, S. Amaducci, J. Andrzejewski, L. Audouin, M. Bacak, M. Barbagallo, S. Bennett, E. Berthoumieux, D. Bosnar , et al. (106 additional authors not shown)

Abstract: The idea of slow-neutron capture nucleosynthesis formulated in 1957 triggered a tremendous experimental effort in different laboratories worldwide to measure the relevant nuclear physics input quantities, namely ($n,γ$) cross sections over the stellar temperature range (from few eV up to several hundred keV) for most of the isotopes involved from Fe up to Bi. A brief historical review focused on t… ▽ More The idea of slow-neutron capture nucleosynthesis formulated in 1957 triggered a tremendous experimental effort in different laboratories worldwide to measure the relevant nuclear physics input quantities, namely ($n,γ$) cross sections over the stellar temperature range (from few eV up to several hundred keV) for most of the isotopes involved from Fe up to Bi. A brief historical review focused on total energy detectors will be presented to illustrate how, advances in instrumentation have led, over the years, to the assessment and discovery of many new aspects of $s$-process nucleosynthesis and to the progressive refinement of theoretical models of stellar evolution. A summary will be presented on current efforts to develop new detection concepts, such as the Total-Energy Detector with $γ$-ray imaging capability (i-TED). The latter is based on the simultaneous combination of Compton imaging with neutron time-of-flight (TOF) techniques, in order to achieve a superior level of sensitivity and selectivity in the measurement of stellar neutron capture rates. △ Less

Submitted 16 November, 2019; originally announced November 2019.

Comments: Nuclear Physics in Astrophysics IX - Conference Proceedings

FENCE: Feasible Evasion Attacks on Neural Networks in Constrained Environments

Authors: Alesia Chernikova, Alina Oprea

Abstract: As advances in Deep Neural Networks (DNNs) demonstrate unprecedented levels of performance in many critical applications, their vulnerability to attacks is still an open question. We consider evasion attacks at testing time against Deep Learning in constrained environments, in which dependencies between features need to be satisfied. These situations may arise naturally in tabular data or may be t… ▽ More As advances in Deep Neural Networks (DNNs) demonstrate unprecedented levels of performance in many critical applications, their vulnerability to attacks is still an open question. We consider evasion attacks at testing time against Deep Learning in constrained environments, in which dependencies between features need to be satisfied. These situations may arise naturally in tabular data or may be the result of feature engineering in specific application domains, such as threat detection in cyber security. We propose a general iterative gradient-based framework called FENCE for crafting evasion attacks that take into consideration the specifics of constrained domains and application requirements. We apply it against Feed-Forward Neural Networks trained for two cyber security applications: network traffic botnet classification and malicious domain classification, to generate feasible adversarial examples. We extensively evaluate the success rate and performance of our attacks, compare their improvement over several baselines, and analyze factors that impact the attack success rate, including the optimization objective and the data imbalance. We show that with minimal effort (e.g., generating 12 additional network connections), an attacker can change the model's prediction from the Malicious class to Benign and evade the classifier. We show that models trained on datasets with higher imbalance are more vulnerable to our FENCE attacks. Finally, we demonstrate the potential of performing adversarial training in constrained domains to increase the model resilience against these evasion attacks. △ Less

Submitted 14 June, 2022; v1 submitted 23 September, 2019; originally announced September 2019.

Comments: 35 pages

AppMine: Behavioral Analytics for Web Application Vulnerability Detection

Authors: Indranil Jana, Alina Oprea

Abstract: Web applications in widespread use have always been the target of large-scale attacks, leading to massive disruption of services and financial loss, as in the Equifax data breach. It has become common practice to deploy web application in containers like Docker for better portability and ease of deployment. We design a system called AppMine for lightweight monitoring of web applications running in… ▽ More Web applications in widespread use have always been the target of large-scale attacks, leading to massive disruption of services and financial loss, as in the Equifax data breach. It has become common practice to deploy web application in containers like Docker for better portability and ease of deployment. We design a system called AppMine for lightweight monitoring of web applications running in Docker containers and detection of unknown web vulnerabilities. AppMine is an unsupervised learning system, trained only on legitimate workloads of web application, to detect anomalies based on either traditional models (PCA and one-class SVM), or more advanced neural-network architectures (LSTM). In our evaluation, we demonstrate that the neural network model outperforms more traditional methods on a range of web applications and recreated exploits. For instance, AppMine achieves average AUC scores as high as 0.97 for the Apache Struts application (with the CVE-2017-5638 exploit used in the Equifax breach), while the AUC scores for PCA and one-class SVM are 0.81 and 0.83, respectively. △ Less

Submitted 5 August, 2019; originally announced August 2019.

The House That Knows You: User Authentication Based on IoT Data

Authors: Talha Ongun, Oliver Spohngellert, Alina Oprea, Cristina Nita-Rotaru, Mihai Christodorescu, Negin Salajegheh

Abstract: Home-based Internet of Things (IoT) devices have gained in popularity and many households have become 'smart' by using devices such as smart sensors, locks, and voice-based assistants. Traditional authentication methods such as passwords, biometrics or multi-factor (using SMS or email) are either not applicable in the smart home setting, or they are inconvenient as they break the natural flow of i… ▽ More Home-based Internet of Things (IoT) devices have gained in popularity and many households have become 'smart' by using devices such as smart sensors, locks, and voice-based assistants. Traditional authentication methods such as passwords, biometrics or multi-factor (using SMS or email) are either not applicable in the smart home setting, or they are inconvenient as they break the natural flow of interaction with these devices. Voice-based biometrics are limited due to safety and privacy concerns. Given the limitations of existing authentication techniques, we explore new opportunities for user authentication in smart home environments. Specifically, we design a novel authentication method based on behavioral features extracted from user interactions with IoT devices. We perform an IRB-approved user study in the IoT lab at our university over a period of three weeks. We collect network traffic from multiple users interacting with 15 IoT devices in our lab and extract a large number of features to capture user activity. We experiment with multiple classification algorithms and also design an ensemble classifier with two models using disjoint set of features. We demonstrate that our ensemble model can classify five users with 0.97 accuracy. The behavioral authentication modules could help address the new challenges emerging with smart home ecosystems and they open up the possibility of creating flexible policies for authorization and access control. △ Less

Submitted 27 December, 2021; v1 submitted 1 August, 2019; originally announced August 2019.

Comments: 11 pages, 5 figures

On Designing Machine Learning Models for Malicious Network Traffic Classification

Authors: Talha Ongun, Timothy Sakharaov, Simona Boboila, Alina Oprea, Tina Eliassi-Rad

Abstract: Machine learning (ML) started to become widely deployed in cyber security settings for shortening the detection cycle of cyber attacks. To date, most ML-based systems are either proprietary or make specific choices of feature representations and machine learning models. The success of these techniques is difficult to assess as public benchmark datasets are currently unavailable. In this paper, we… ▽ More Machine learning (ML) started to become widely deployed in cyber security settings for shortening the detection cycle of cyber attacks. To date, most ML-based systems are either proprietary or make specific choices of feature representations and machine learning models. The success of these techniques is difficult to assess as public benchmark datasets are currently unavailable. In this paper, we provide concrete guidelines and recommendations for using supervised ML in cyber security. As a case study, we consider the problem of botnet detection from network traffic data. Among our findings we highlight that: (1) feature representations should take into consideration attack characteristics; (2) ensemble models are well-suited to handle class imbalance; (3) the granularity of ground truth plays an important role in the success of these methods. △ Less

Submitted 10 July, 2019; originally announced July 2019.

QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game

Authors: Lisa Oakley, Alina Oprea

Abstract: A rise in Advanced Persistent Threats (APTs) has introduced a need for robustness against long-running, stealthy attacks which circumvent existing cryptographic security guarantees. FlipIt is a security game that models attacker-defender interactions in advanced scenarios such as APTs. Previous work analyzed extensively non-adaptive strategies in FlipIt, but adaptive strategies rise naturally in p… ▽ More A rise in Advanced Persistent Threats (APTs) has introduced a need for robustness against long-running, stealthy attacks which circumvent existing cryptographic security guarantees. FlipIt is a security game that models attacker-defender interactions in advanced scenarios such as APTs. Previous work analyzed extensively non-adaptive strategies in FlipIt, but adaptive strategies rise naturally in practical interactions as players receive feedback during the game. We model the FlipIt game as a Markov Decision Process and introduce QFlip, an adaptive strategy for FlipIt based on temporal difference reinforcement learning. We prove theoretical results on the convergence of our new strategy against an opponent playing with a Periodic strategy. We confirm our analysis experimentally by extensive evaluation of QFlip against specific opponents. QFlip converges to the optimal adaptive strategy for Periodic and Exponential opponents using associated state spaces. Finally, we introduce a generalized QFlip strategy with composite state space that outperforms a Greedy strategy for several distributions including Periodic and Uniform, without prior knowledge of the opponent's strategy. We also release an OpenAI Gym environment for FlipIt to facilitate future research. △ Less

Submitted 20 December, 2019; v1 submitted 27 June, 2019; originally announced June 2019.

Comments: Outstanding Student Paper award

Journal ref: Decision and Game Theory for Security. GameSec 2019. Lecture Notes in Computer Science, vol 11836. Springer, Cham. pp 364-384

Are Self-Driving Cars Secure? Evasion Attacks against Deep Neural Networks for Steering Angle Prediction

Authors: Alesia Chernikova, Alina Oprea, Cristina Nita-Rotaru, BaekGyu Kim

Abstract: Deep Neural Networks (DNNs) have tremendous potential in advancing the vision for self-driving cars. However, the security of DNN models in this context leads to major safety implications and needs to be better understood. We consider the case study of steering angle prediction from camera images, using the dataset from the 2014 Udacity challenge. We demonstrate for the first time adversarial test… ▽ More Deep Neural Networks (DNNs) have tremendous potential in advancing the vision for self-driving cars. However, the security of DNN models in this context leads to major safety implications and needs to be better understood. We consider the case study of steering angle prediction from camera images, using the dataset from the 2014 Udacity challenge. We demonstrate for the first time adversarial testing-time attacks for this application for both classification and regression settings. We show that minor modifications to the camera image (an L2 distance of 0.82 for one of the considered models) result in mis-classification of an image to any class of attacker's choice. Furthermore, our regression attack results in a significant increase in Mean Square Error (MSE) by a factor of 69 in the worst case. △ Less

Submitted 15 April, 2019; originally announced April 2019.

Comments: Preprint of the work accepted for publication at the IEEE Workshop on the Internet of Safe Things, San Francisco, CA, USA, May 23, 2019

Private Hierarchical Clustering and Efficient Approximation

Authors: Xianrui Meng, Dimitrios Papadopoulos, Alina Oprea, Nikos Triandopoulos

Abstract: In collaborative learning, multiple parties contribute their datasets to jointly deduce global machine learning models for numerous predictive tasks. Despite its efficacy, this learning paradigm fails to encompass critical application domains that involve highly sensitive data, such as healthcare and security analytics, where privacy risks limit entities to individually train models using only the… ▽ More In collaborative learning, multiple parties contribute their datasets to jointly deduce global machine learning models for numerous predictive tasks. Despite its efficacy, this learning paradigm fails to encompass critical application domains that involve highly sensitive data, such as healthcare and security analytics, where privacy risks limit entities to individually train models using only their own datasets. In this work, we target privacy-preserving collaborative hierarchical clustering. We introduce a formal security definition that aims to achieve the balance between utility and privacy and present a two-party protocol that provably satisfies it. We then extend our protocol with: (i) an optimized version for the single-linkage clustering, and (ii) scalable approximation variants. We implement all our schemes and experimentally evaluate their performance and accuracy on synthetic and real datasets, obtaining very encouraging results. For example, end-to-end execution of our secure approximate protocol for over 1M 10-dimensional data samples requires 35sec of computation and achieves 97.09% accuracy. △ Less

Submitted 1 October, 2021; v1 submitted 9 April, 2019; originally announced April 2019.

Journal ref: CCSW 2021

Measurement of the 235U(n,f) cross section relative to the 6Li(n,t) and 10B(n,alpha) standards from thermal to 170 keV neutron energy range at n_TOF

Authors: S. Amaducci, L. Cosentino, M. Barbagallo, N. Colonna, A. Mengoni, C. Massimi, S. Lo Meo, P. Finocchiaro, O. Aberle, J. Andrzejewski, L. Audouin, M. Bacak, J. Balibrea, F. Bečvář, E. Berthoumieux, J. Billowes, D. Bosnar, A. Brown, M. Caamaño, F. Calviño, M. Calviani, D. Cano-Ott, R. Cardella, A. Casanovas, F. Cerutti , et al. (96 additional authors not shown)

Abstract: The 235U(n,f) cross section was measured in a wide energy range at n_TOF relative to 6Li(n,t) and 10B(n,alpha), with high resolution and in a wide energy range, with a setup based on a stack of six samples and six silicon detectors placed in the neutron beam. This allowed us to make a direct comparison of the reaction yields under the same experimental conditions, and taking into account the forwa… ▽ More The 235U(n,f) cross section was measured in a wide energy range at n_TOF relative to 6Li(n,t) and 10B(n,alpha), with high resolution and in a wide energy range, with a setup based on a stack of six samples and six silicon detectors placed in the neutron beam. This allowed us to make a direct comparison of the reaction yields under the same experimental conditions, and taking into account the forward/backward emission asymmetry. A hint of an anomaly in the 10÷30 keV neutron energy range had been previously observed in other experiments, indicating a cross section systematically lower by several percent relative to major evaluations. The present results indicate that the evaluated cross section in the 9÷18 keV neutron energy range is indeed overestimated, both in the recent updates of ENDF/B-VIII.0 and of the IAEA reference data. Furthermore, these new high-resolution data confirm the existence of resonance-like structures in the keV neutron energy region. The new, high accuracy results here reported may lead to a reduction of the uncertainty in the 1÷100 keV neutron energy region. Finally, the present data provide additional confidence on the recently re-evaluated cross section integral between 7.8 and 11 eV. △ Less

Submitted 4 March, 2019; v1 submitted 27 February, 2019; originally announced February 2019.

Comments: 22 pages, 20 figures, submitted to PRC

Differentially Private Fair Learning

Authors: Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Malvajerdi, Jonathan Ullman

Abstract: Motivated by settings in which predictive models may be required to be non-discriminatory with respect to certain attributes (such as race), but even collecting the sensitive attribute may be forbidden or restricted, we initiate the study of fair learning under the constraint of differential privacy. We design two learning algorithms that simultaneously promise differential privacy and equalized o… ▽ More Motivated by settings in which predictive models may be required to be non-discriminatory with respect to certain attributes (such as race), but even collecting the sensitive attribute may be forbidden or restricted, we initiate the study of fair learning under the constraint of differential privacy. We design two learning algorithms that simultaneously promise differential privacy and equalized odds, a 'fairness' condition that corresponds to equalizing false positive and negative rates across protected groups. Our first algorithm is a private implementation of the equalized odds post-processing approach of [Hardt et al., 2016]. This algorithm is appealingly simple, but must be able to use protected group membership explicitly at test time, which can be viewed as a form of 'disparate treatment'. Our second algorithm is a differentially private version of the oracle-efficient in-processing approach of [Agarwal et al., 2018] that can be used to find the optimal fair classifier, given access to a subroutine that can solve the original (not necessarily fair) learning problem. This algorithm is more complex but need not have access to protected group membership at test time. We identify new tradeoffs between fairness, accuracy, and privacy that emerge only when requiring all three properties, and show that these tradeoffs can be milder if group membership may be used at test time. We conclude with a brief experimental evaluation. △ Less

Submitted 31 May, 2019; v1 submitted 6 December, 2018; originally announced December 2018.

Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks

Authors: Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, Fabio Roli

Abstract: Transferability captures the ability of an attack against a machine-learning model to be effective against a different, potentially unknown, model. Empirical evidence for transferability has been shown in previous work, but the underlying reasons why an attack transfers or not are not yet well understood. In this paper, we present a comprehensive analysis aimed to investigate the transferability o… ▽ More Transferability captures the ability of an attack against a machine-learning model to be effective against a different, potentially unknown, model. Empirical evidence for transferability has been shown in previous work, but the underlying reasons why an attack transfers or not are not yet well understood. In this paper, we present a comprehensive analysis aimed to investigate the transferability of both test-time evasion and training-time poisoning attacks. We provide a unifying optimization framework for evasion and poisoning attacks, and a formal definition of transferability of such attacks. We highlight two main factors contributing to attack transferability: the intrinsic adversarial vulnerability of the target model, and the complexity of the surrogate model used to optimize the attack. Based on these insights, we define three metrics that impact an attack's transferability. Interestingly, our results derived from theoretical analysis hold for both evasion and poisoning attacks, and are confirmed experimentally using a wide range of linear and non-linear classifiers and datasets. △ Less

Submitted 13 June, 2019; v1 submitted 8 September, 2018; originally announced September 2018.

MSC Class: 68T10; 68T45

The $^{7}$Be($\boldsymbol{n,p}$)$^{7}$Li reaction and the Cosmological Lithium Problem: measurement of the cross section in a wide energy range at n_TOF (CERN)

Authors: L. Damone, M. Barbagallo, M. Mastromarco, A. Mengoni, L. Cosentino, E. Maugeri, S. Heinitz, D. Schumann, R. Dressler, F. Käppeler, N. Colonna, P. Finocchiaro, J. Andrzejewski, J. Perkowski, A. Gawlik, O. Aberle, S. Altstadt, M. Ayranov, L. Audouin, M. Bacak, J. Balibrea-Correa, J. Ballof, V. Bécares, F. Bečvář, C. Beinrucker , et al. (133 additional authors not shown)

Abstract: We report on the measurement of the $^{7}$Be($n, p$)$^{7}$Li cross section from thermal to approximately 325 keV neutron energy, performed in the high-flux experimental area (EAR2) of the n\_TOF facility at CERN. This reaction plays a key role in the lithium yield of the Big Bang Nucleosynthesis (BBN) for standard cosmology. The only two previous time-of-flight measurements performed on this react… ▽ More We report on the measurement of the $^{7}$Be($n, p$)$^{7}$Li cross section from thermal to approximately 325 keV neutron energy, performed in the high-flux experimental area (EAR2) of the n\_TOF facility at CERN. This reaction plays a key role in the lithium yield of the Big Bang Nucleosynthesis (BBN) for standard cosmology. The only two previous time-of-flight measurements performed on this reaction did not cover the energy window of interest for BBN, and showed a large discrepancy between each other. The measurement was performed with a Si-telescope, and a high-purity sample produced by implantation of a $^{7}$Be ion beam at the ISOLDE facility at CERN. While a significantly higher cross section is found at low-energy, relative to current evaluations, in the region of BBN interest the present results are consistent with the values inferred from the time-reversal $^{7}$Li($p, n$)$^{7}$Be reaction, thus yielding only a relatively minor improvement on the so-called Cosmological Lithium Problem (CLiP). The relevance of these results on the near-threshold neutron production in the p+$^{7}$Li reaction is also discussed. △ Less

Submitted 8 June, 2018; originally announced June 2018.

Journal ref: Phys. Rev. Lett. 121, 042701 (2018)

Cross section measurements of $^{155,157}$Gd(n,$γ$) induced by thermal and epithermal neutrons

Authors: M. Mastromarco, A. Manna, O. Aberle, S. Amaducci, J. Andrzejewski, L. Audouin, M. Bacak, J. Balibrea, M. Barbagallo, F. Becvar, E. Berthoumieux, J. Billowes, D. Bosnar, A. Brown, M. Caamano, F. Calvino, M. Calviani, D. Cano-Ott, R. Cardella, A. Casanovas, D. M. Castelluccio, F. Cerutti, Y. H. Chen, E. Chiaveri, G. Clai , et al. (99 additional authors not shown)

Abstract: Neutron capture measurements on $^{155}$Gd and $^{157}$Gd were performed using the time-of-flight technique at the n\_TOF facility at CERN. Four samples in form of self-sustaining metallic discs isotopically enriched in $^{155}$Gd and $^{157}$Gd were used. The measurements were carried out at the experimental area (EAR1) at 185 m from the neutron source, with an array of 4 C$_6$D$_6$ liquid scinti… ▽ More Neutron capture measurements on $^{155}$Gd and $^{157}$Gd were performed using the time-of-flight technique at the n\_TOF facility at CERN. Four samples in form of self-sustaining metallic discs isotopically enriched in $^{155}$Gd and $^{157}$Gd were used. The measurements were carried out at the experimental area (EAR1) at 185 m from the neutron source, with an array of 4 C$_6$D$_6$ liquid scintillation detectors. The capture cross sections of $^{155}$Gd and $^{157}$Gd at neutron kinetic energy of 0.0253 eV have been estimated to be 62.2(2.2) kb and 239.8(9.3) kb, respectively, thus up to 6\% different relative to the ones reported in the nuclear data libraries. A resonance shape analysis has been performed in the resolved resonance region up to 180 eV and 300 eV, respectively, in average resonance parameters have been found in good agreement with evaluations. Above these energies the observed resonance-like structures in the cross section have been tentatively characterised in terms of resonance energy and area up to 1 keV. △ Less

Submitted 10 May, 2018; originally announced May 2018.

Journal ref: Eur. Phys. J. A (2019) 55: 9