-
Learning with distributional inverters
Authors:
Eric Binnendyk,
Marco Carmosino,
Antonina Kolokolova,
Ramyaa Ramyaa,
Manuel Sabin
Abstract:
We generalize the "indirect learning" technique of Furst et. al., 1991 to reduce from learning a concept class over a samplable distribution $μ$ to learning the same concept class over the uniform distribution. The reduction succeeds when the sampler for $μ$ is both contained in the target concept class and efficiently invertible in the sense of Impagliazzo & Luby, 1989. We give two applications.…
▽ More
We generalize the "indirect learning" technique of Furst et. al., 1991 to reduce from learning a concept class over a samplable distribution $μ$ to learning the same concept class over the uniform distribution. The reduction succeeds when the sampler for $μ$ is both contained in the target concept class and efficiently invertible in the sense of Impagliazzo & Luby, 1989. We give two applications.
- We show that AC0[q] is learnable over any succinctly-described product distribution. AC0[q] is the class of constant-depth Boolean circuits of polynomial size with AND, OR, NOT, and counting modulo $q$ gates of unbounded fanins. Our algorithm runs in randomized quasi-polynomial time and uses membership queries.
- If there is a strongly useful natural property in the sense of Razborov & Rudich 1997 -- an efficient algorithm that can distinguish between random strings and strings of non-trivial circuit complexity -- then general polynomial-sized Boolean circuits are learnable over any efficiently samplable distribution in randomized polynomial time, given membership queries to the target function
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
-
Mind the Gap: On Bridging the Semantic Gap between Machine Learning and Information Security
Authors:
Michael R. Smith,
Nicholas T. Johnson,
Joe B. Ingram,
Armida J. Carbajal,
Ramyaa Ramyaa,
Evelyn Domschot,
Christopher C. Lamb,
Stephen J. Verzi,
W. Philip Kegelmeyer
Abstract:
Despite the potential of Machine learning (ML) to learn the behavior of malware, detect novel malware samples, and significantly improve information security (InfoSec) we see few, if any, high-impact ML techniques in deployed systems, notwithstanding multiple reported successes in open literature. We hypothesize that the failure of ML in making high-impacts in InfoSec are rooted in a disconnect be…
▽ More
Despite the potential of Machine learning (ML) to learn the behavior of malware, detect novel malware samples, and significantly improve information security (InfoSec) we see few, if any, high-impact ML techniques in deployed systems, notwithstanding multiple reported successes in open literature. We hypothesize that the failure of ML in making high-impacts in InfoSec are rooted in a disconnect between the two communities as evidenced by a semantic gap---a difference in how executables are described (e.g. the data and features extracted from the data). Specifically, current datasets and representations used by ML are not suitable for learning the behaviors of an executable and differ significantly from those used by the InfoSec community. In this paper, we survey existing datasets used for classifying malware by ML algorithms and the features that are extracted from the data. We observe that: 1) the current set of extracted features are primarily syntactic, not behavioral, 2) datasets generally contain extreme exemplars producing a dataset in which it is easy to discriminate classes, and 3) the datasets provide significantly different representations of the data encountered in real-world systems. For ML to make more of an impact in the InfoSec community requires a change in the data (including the features and labels) that is used to bridge the current semantic gap. As a first step in enabling more behavioral analyses, we label existing malware datasets with behavioral features using open-source threat reports associated with malware families. This behavioral labeling alters the analysis from identifying intent (e.g. good vs bad) or malware family membership to an analysis of which behaviors are exhibited by an executable. We offer the annotations with the hope of inspiring future improvements in the data that will further bridge the semantic gap between the ML and InfoSec communities.
△ Less
Submitted 4 May, 2020;
originally announced May 2020.
-
Biologically inspired sleep algorithm for artificial neural networks
Authors:
Giri P Krishnan,
Timothy Tadros,
Ramyaa Ramyaa,
Maxim Bazhenov
Abstract:
Sleep plays an important role in incremental learning and consolidation of memories in biological systems. Motivated by the processes that are known to be involved in sleep generation in biological networks, we developed an algorithm that implements a sleep-like phase in artificial neural networks (ANNs). After initial training phase, we convert the ANN to a spiking neural network (SNN) and simula…
▽ More
Sleep plays an important role in incremental learning and consolidation of memories in biological systems. Motivated by the processes that are known to be involved in sleep generation in biological networks, we developed an algorithm that implements a sleep-like phase in artificial neural networks (ANNs). After initial training phase, we convert the ANN to a spiking neural network (SNN) and simulate an offline sleep-like phase using spike-timing dependent plasticity rules to modify synaptic weights. The SNN is then converted back to the ANN and evaluated or trained on new inputs. We demonstrate several performance improvements after applying this processing to ANNs trained on MNIST, CUB200 and a motivating toy dataset. First, in an incremental learning framework, sleep is able to recover older tasks that were otherwise forgotten in the ANN without sleep phase due to catastrophic forgetting. Second, sleep results in forward transfer learning of unseen tasks. Finally, sleep improves generalization ability of the ANNs to classify images with various types of noise. We provide a theoretical basis for the beneficial role of the brain-inspired sleep-like phase for the ANNs and present an algorithmic way for future implementations of the various features of sleep in deep learning ANNs. Overall, these results suggest that biological sleep can help mitigate a number of problems ANNs suffer from, such as poor generalization and catastrophic forgetting for incremental learning.
△ Less
Submitted 1 August, 2019;
originally announced August 2019.
-
Denotational cost semantics for functional languages with inductive types
Authors:
Norman Danner,
Daniel R. Licata,
Ramyaa Ramyaa
Abstract:
A central method for analyzing the asymptotic complexity of a functional program is to extract and then solve a recurrence that expresses evaluation cost in terms of input size. The relevant notion of input size is often specific to a datatype, with measures including the length of a list, the maximum element in a list, and the height of a tree. In this work, we give a formal account of the extrac…
▽ More
A central method for analyzing the asymptotic complexity of a functional program is to extract and then solve a recurrence that expresses evaluation cost in terms of input size. The relevant notion of input size is often specific to a datatype, with measures including the length of a list, the maximum element in a list, and the height of a tree. In this work, we give a formal account of the extraction of cost and size recurrences from higher-order functional programs over inductive datatypes. Our approach allows a wide range of programmer-specified notions of size, and ensures that the extracted recurrences correctly predict evaluation cost. To extract a recurrence from a program, we first make costs explicit by applying a monadic translation from the source language to a complexity language, and then abstract datatype values as sizes. Size abstraction can be done semantically, working in models of the complexity language, or syntactically, by adding rules to a preorder judgement. We give several different models of the complexity language, which support different notions of size. Additionally, we prove by a logical relations argument that recurrences extracted by this process are upper bounds for evaluation cost; the proof is entirely syntactic and therefore applies to all of the models we consider.
△ Less
Submitted 5 June, 2015;
originally announced June 2015.
-
Power of Nondetreministic JAGs on Cayley graphs
Authors:
Martin Hofmann,
Ramyaa Ramyaa
Abstract:
The Immerman-Szelepcsenyi Theorem uses an algorithm for co-st- connectivity based on inductive counting to prove that NLOGSPACE is closed un- der complementation. We want to investigate whether counting is necessary for this theorem to hold. Concretely, we show that Nondeterministic Jum** Graph Autmata (ND-JAGs) (pebble automata on graphs), on several families of Cayley graphs, are equal in powe…
▽ More
The Immerman-Szelepcsenyi Theorem uses an algorithm for co-st- connectivity based on inductive counting to prove that NLOGSPACE is closed un- der complementation. We want to investigate whether counting is necessary for this theorem to hold. Concretely, we show that Nondeterministic Jum** Graph Autmata (ND-JAGs) (pebble automata on graphs), on several families of Cayley graphs, are equal in power to nondeterministic logspace Turing machines that are given such graphs as a linear encoding. In particular, it follows that ND-JAGs can solve co-st-connectivity on those graphs. This came as a surprise since Cook and Rackoff showed that deterministic JAGs cannot solve st-connectivity on many Cayley graphs due to their high self-similarity (every neighbourhood looks the same). Thus, our results show that on these graphs, nondeterminism provably adds computational power. The families of Cayley graphs we consider include Cayley graphs of abelian groups and of all finite simple groups irrespective of how they are presented and graphs corresponding to groups generated by various product constructions, in- cluding iterated ones. We remark that assessing the precise power of nondeterministic JAGs and in par- ticular whether they can solve co-st-connectivity on arbitrary graphs is left as an open problem by Edmonds, Poon and Achlioptas. Our results suggest a positive answer to this question and in particular considerably limit the search space for a potential counterexample.
△ Less
Submitted 30 October, 2013;
originally announced October 2013.
-
Implicit complexity for coinductive data: a characterization of corecurrence
Authors:
Daniel Leivant,
Ramyaa Ramyaa
Abstract:
We propose a framework for reasoning about programs that manipulate coinductive data as well as inductive data. Our approach is based on using equational programs, which support a seamless combination of computation and reasoning, and using productivity (fairness) as the fundamental assertion, rather than bi-simulation. The latter is expressible in terms of the former. As an application to this…
▽ More
We propose a framework for reasoning about programs that manipulate coinductive data as well as inductive data. Our approach is based on using equational programs, which support a seamless combination of computation and reasoning, and using productivity (fairness) as the fundamental assertion, rather than bi-simulation. The latter is expressible in terms of the former. As an application to this framework, we give an implicit characterization of corecurrence: a function is definable using corecurrence iff its productivity is provable using coinduction for formulas in which data-predicates do not occur negatively. This is an analog, albeit in weaker form, of a characterization of recurrence (i.e. primitive recursion) in [Leivant, Unipolar induction, TCS 318, 2004].
△ Less
Submitted 5 January, 2012;
originally announced January 2012.
-
Outage Probability of Multiple-Input Single-Output (MISO) Systems with Delayed Feedback
Authors:
Venkata Sreekanta Annpureddy,
Devdutt V. Marathe,
T. R. Ramya,
Srikrishna Bhashyam
Abstract:
We investigate the effect of feedback delay on the outage probability of multiple-input single-output (MISO) fading channels. Channel state information at the transmitter (CSIT) is a delayed version of the channel state information available at the receiver (CSIR). We consider two cases of CSIR: (a) perfect CSIR and (b) CSI estimated at the receiver using training symbols. With perfect CSIR, und…
▽ More
We investigate the effect of feedback delay on the outage probability of multiple-input single-output (MISO) fading channels. Channel state information at the transmitter (CSIT) is a delayed version of the channel state information available at the receiver (CSIR). We consider two cases of CSIR: (a) perfect CSIR and (b) CSI estimated at the receiver using training symbols. With perfect CSIR, under a short-term power constraint, we determine: (a) the outage probability for beamforming with imperfect CSIT (BF-IC) analytically, and (b) the optimal spatial power allocation (OSPA) scheme that minimizes outage numerically. Results show that, for delayed CSIT, BF-IC is close to optimal for low SNR and uniform spatial power allocation (USPA) is close to optimal at high SNR. Similarly, under a long-term power constraint, we show that BF-IC is close to optimal for low SNR and USPA is close to optimal at high SNR. With imperfect CSIR, we obtain an upper bound on the outage probability with USPA and BF-IC. Results show that the loss in performance due to imperfection in CSIR is not significant, if the training power is chosen appropriately.
△ Less
Submitted 5 November, 2007; v1 submitted 3 March, 2007;
originally announced March 2007.
