-
Mitigating Algorithmic Bias on Facial Expression Recognition
Authors:
Glauco Amigo,
Pablo Rivas Perea,
Robert J. Marks
Abstract:
Biased datasets are ubiquitous and present a challenge for machine learning. For a number of categories on a dataset that are equally important but some are sparse and others are common, the learning algorithms will favor the ones with more presence. The problem of biased datasets is especially sensitive when dealing with minority people groups. How can we, from biased data, generate algorithms th…
▽ More
Biased datasets are ubiquitous and present a challenge for machine learning. For a number of categories on a dataset that are equally important but some are sparse and others are common, the learning algorithms will favor the ones with more presence. The problem of biased datasets is especially sensitive when dealing with minority people groups. How can we, from biased data, generate algorithms that treat every person equally? This work explores one way to mitigate bias using a debiasing variational autoencoder with experiments on facial expression recognition.
△ Less
Submitted 23 December, 2023;
originally announced December 2023.
-
Algorithmic Information Forecastability
Authors:
Glauco Amigo,
Daniel Andrés Díaz-Pachón,
Robert J. Marks,
Charles Baylis
Abstract:
The outcome of all time series cannot be forecast, e.g. the flip** of a fair coin. Others, like the repeated {01} sequence {010101...} can be forecast exactly. Algorithmic information theory can provide a measure of forecastability that lies between these extremes. The degree of forecastability is a function of only the data. For prediction (or classification) of labeled data, we propose three c…
▽ More
The outcome of all time series cannot be forecast, e.g. the flip** of a fair coin. Others, like the repeated {01} sequence {010101...} can be forecast exactly. Algorithmic information theory can provide a measure of forecastability that lies between these extremes. The degree of forecastability is a function of only the data. For prediction (or classification) of labeled data, we propose three categories for forecastability: oracle forecastability for predictions that are always exact, precise forecastability for errors up to a bound, and probabilistic forecastability for any other predictions. Examples are given in each case.
△ Less
Submitted 1 December, 2023; v1 submitted 21 April, 2023;
originally announced April 2023.
-
Changepoint Detection for Real-Time Spectrum Sharing Radar
Authors:
Samuel Haug,
Austin Egbert,
Robert J. Marks II,
Charles Baylis,
Anthony Martone
Abstract:
Radar must adapt to changing environments, and we propose changepoint detection as a method to do so. In the world of increasingly congested radio frequencies, radars must adapt to avoid interference. Many radar systems employ the prediction action cycle to proactively determine transmission mode while spectrum sharing. This method constructs and implements a model of the environment to predict un…
▽ More
Radar must adapt to changing environments, and we propose changepoint detection as a method to do so. In the world of increasingly congested radio frequencies, radars must adapt to avoid interference. Many radar systems employ the prediction action cycle to proactively determine transmission mode while spectrum sharing. This method constructs and implements a model of the environment to predict unused frequencies, and then transmits in this predicted availability. For these selection strategies, performance is directly reliant on the quality of the underlying environmental models. In order to keep up with a changing environment, these models can employ changepoint detection. Changepoint detection is the identification of sudden changes, or changepoints, in the distribution from which data is drawn. This information allows the models to discard "garbage" data from a previous distribution, which has no relation to the current state of the environment. In this work, bayesian online changepoint detection (BOCD) is applied to the sense and predict algorithm to increase the accuracy of its models and improve its performance. In the context of spectrum sharing, these changepoints represent interferers leaving and entering the spectral environment. The addition of changepoint detection allows for dynamic and robust spectrum sharing even as interference patterns change dramatically. BOCD is especially advantageous because it enables online changepoint detection, allowing models to be updated continuously as data are collected. This strategy can also be applied to many other predictive algorithms that create models in a changing environment.
△ Less
Submitted 30 June, 2022;
originally announced July 2022.
-
Dilated POCS: Minimax Convex Optimization
Authors:
Albert R. Yu,
Robert J. Marks II,
Keith E. Schubert,
Charles Baylis,
Austin Egbert,
Adam Goad,
Sam Haug
Abstract:
Alternating projection onto convex sets (POCS) provides an iterative procedure to find a signal that satisfies two or more convex constraints when the sets intersect. For nonintersecting constraints, the method of simultaneous projections produces a minimum mean square error (MMSE) solution. In certain cases, a minimax solution is more desirable. Generating a minimax solution is possible using dil…
▽ More
Alternating projection onto convex sets (POCS) provides an iterative procedure to find a signal that satisfies two or more convex constraints when the sets intersect. For nonintersecting constraints, the method of simultaneous projections produces a minimum mean square error (MMSE) solution. In certain cases, a minimax solution is more desirable. Generating a minimax solution is possible using dilated POCS. The minimax solution uses morphological dilation of nonintersecting signal convex constraints. The sets are progressively dilated to the point where there is intersection at a minimax solution. Examples are given contrasting the MMSE and minimax solutions in problems of tomographic reconstruction of images. Dilated POCS adds a new imaging modality for image synthesis. Lastly, morphological erosion of signal sets is suggested as a method to shrink the overlap when sets intersect at more than one point.
△ Less
Submitted 27 January, 2023; v1 submitted 9 June, 2022;
originally announced June 2022.
-
In-Situ Assessment of Array Antenna Currents for Real-Time Impedance Tuning
Authors:
Charles Baylis,
Adam Goad,
Trevor Van Hoosier,
Austin Egbert,
Robert J. Marks II
Abstract:
Impedance tuning has shown promise to maximize output power, gain, or efficiency for transmitter amplifier devices in changing frequency or array environments. Using impedance tuning in phased-array elements, however, has a significant and dynamically changing impact on the transmitted array pattern. To assess the impact of impedance tuning operations on the array pattern, this paper presents a si…
▽ More
Impedance tuning has shown promise to maximize output power, gain, or efficiency for transmitter amplifier devices in changing frequency or array environments. Using impedance tuning in phased-array elements, however, has a significant and dynamically changing impact on the transmitted array pattern. To assess the impact of impedance tuning operations on the array pattern, this paper presents a simple method to monitor the antenna input current in real time, allowing optimizations that attempt to preserve the array pattern while achieving goals of increased power, gain, or efficiency. The real-time monitoring of current provides many advantages over traditional array calibration methods in situations where the array element chains can change significantly in magnitude and/or phase during operations.
△ Less
Submitted 7 June, 2022;
originally announced June 2022.
-
Sometimes size does not matter
Authors:
Daniel Andrés Díaz-Pachón,
Ola Hössjer,
Robert J. Marks II
Abstract:
Cosmological fine-tuning has traditionally been associated with the narrowness of the intervals in which the parameters of the physical models must be located to make life possible. A more thorough approach focuses on the probability of the interval, not on its size. Most attempts to measure the probability of the life-permitting interval for a given parameter rely on a Bayesian statistical approa…
▽ More
Cosmological fine-tuning has traditionally been associated with the narrowness of the intervals in which the parameters of the physical models must be located to make life possible. A more thorough approach focuses on the probability of the interval, not on its size. Most attempts to measure the probability of the life-permitting interval for a given parameter rely on a Bayesian statistical approach for which the prior distribution of the parameter is uniform. However, the parameters in these models often take values in spaces of infinite size, so that a uniformity assumption is not possible. This is known as the normalization problem. This paper explains a framework to measure tuning that, among others, deals with normalization, assuming that the prior distribution belongs to a class of maximum entropy (maxent) distributions. By analyzing an upper bound of the tuning probability for this class of distributions the method solves the so-called weak anthropic principle, and offer a solution, at least in this context, to the well-known lack of invariance of maxent distributions. The implication of this approach is that, since all mathematical models need parameters, tuning is not only a question of natural science, but also a problem of mathematical modeling. Cosmological tuning is thus a particular instantiation of a more general scenario. Therefore, whenever a mathematical model is used to describe nature, not only in physics but in all of science, tuning is present. And the question of whether the tuning is fine or coarse for a given parameter -- if the interval in which the parameter is located has low or high probability, respectively -- depends crucially not only on the interval but also on the assumed class of prior distributions. Novel upper bounds for tuning probabilities are presented.
△ Less
Submitted 25 April, 2022;
originally announced April 2022.
-
Active information requirements for fixation on the Wright-Fisher model of population genetics
Authors:
Daniel Andrés Díaz-Pachón,
Robert J. Marks II
Abstract:
In the context of population genetics, active information can be extended to measure the change of information of a given event (e.g., fixation of an allele) from a neutral model in which only genetic drift is taken into account to a non-neutral model that includes other sources of frequency variation (e.g., selection and mutation). In this paper we illustrate active information in population gene…
▽ More
In the context of population genetics, active information can be extended to measure the change of information of a given event (e.g., fixation of an allele) from a neutral model in which only genetic drift is taken into account to a non-neutral model that includes other sources of frequency variation (e.g., selection and mutation). In this paper we illustrate active information in population genetics through the Wright-Fisher model.
△ Less
Submitted 12 November, 2021;
originally announced November 2021.
-
Generalized active information: extensions to unbounded domains
Authors:
Daniel Andrés Díaz-Pachón,
Robert J. Marks II
Abstract:
In the last three decades, several measures of complexity have been proposed. Up to this point, most of such measures have only been developed for finite spaces. In these scenarios the baseline distribution is uniform. This makes sense because, among other things, the uniform distribution is the measure of maximum entropy over the relevant space. Active information traditionally assumes a finite i…
▽ More
In the last three decades, several measures of complexity have been proposed. Up to this point, most of such measures have only been developed for finite spaces. In these scenarios the baseline distribution is uniform. This makes sense because, among other things, the uniform distribution is the measure of maximum entropy over the relevant space. Active information traditionally assumes a finite interval universe of discourse but can be extended to other cases where maximum entropy is defined. Illustrating this is the purpose of this paper. Disequilibrium from maximum entropy, measured as active information, can be evaluated from baselines with unbounded support.
△ Less
Submitted 12 November, 2021;
originally announced November 2021.
-
Generatively Augmented Neural Network Watchdog for Image Classification Networks
Authors:
Justin M. Bui,
Glauco A. Amigo,
Robert J. Marks II
Abstract:
The identification of out-of-distribution data is vital to the deployment of classification networks. For example, a generic neural network that has been trained to differentiate between images of dogs and cats can only classify an input as either a dog or a cat. If a picture of a car or a kumquat were to be supplied to this classifier, the result would still be either a dog or a cat. In order to…
▽ More
The identification of out-of-distribution data is vital to the deployment of classification networks. For example, a generic neural network that has been trained to differentiate between images of dogs and cats can only classify an input as either a dog or a cat. If a picture of a car or a kumquat were to be supplied to this classifier, the result would still be either a dog or a cat. In order to mitigate this, techniques such as the neural network watchdog have been developed. The compression of the image input into the latent layer of the autoencoder defines the region of in-distribution in the image space. This in-distribution set of input data has a corresponding boundary in the image space. The watchdog assesses whether inputs are in inside or outside this boundary. This paper demonstrates how to sharpen this boundary using generative network training data augmentation thereby bettering the discrimination and overall performance of the watchdog.
△ Less
Submitted 7 September, 2021;
originally announced September 2021.
-
Cascade Watchdog: A Multi-tiered Adversarial Guard for Outlier Detection
Authors:
Glauco Amigo,
Justin M. Bui,
Charles Baylis,
Robert J. Marks
Abstract:
The identification of out-of-distribution content is critical to the successful implementation of neural networks. Watchdog techniques have been developed to support the detection of these inputs, but the performance can be limited by the amount of available data. Generative adversarial networks have displayed numerous capabilities, including the ability to generate facsimiles with excellent accur…
▽ More
The identification of out-of-distribution content is critical to the successful implementation of neural networks. Watchdog techniques have been developed to support the detection of these inputs, but the performance can be limited by the amount of available data. Generative adversarial networks have displayed numerous capabilities, including the ability to generate facsimiles with excellent accuracy. This paper presents and empirically evaluates a multi-tiered watchdog, which is developed using GAN generated data, for improved out-of-distribution detection. The cascade watchdog uses adversarial training to increase the amount of available data similar to the out-of-distribution elements that are more difficult to detect. Then, a specialized second guard is added in sequential order. The results show a solid and significant improvement on the detection of the most challenging out-of-distribution inputs while preserving an extremely low false positive rate.
△ Less
Submitted 14 June, 2022; v1 submitted 20 August, 2021;
originally announced August 2021.
-
Classification of Common Waveforms Including a Watchdog for Unknown Signals
Authors:
C. Tanner Fredieu,
Justin Bui,
Anthony Martone,
Robert J. Marks II,
Charles Baylis,
R. Michael Buehrer
Abstract:
In this paper, we examine the use of a deep multi-layer perceptron model architecture to classify received signal samples as coming from one of four common waveforms, Single Carrier (SC), Single-Carrier Frequency Division Multiple Access (SC-FDMA), Orthogonal Frequency Division Multiplexing (OFDM), and Linear Frequency Modulation (LFM), used in communication and radar networks. Synchronization of…
▽ More
In this paper, we examine the use of a deep multi-layer perceptron model architecture to classify received signal samples as coming from one of four common waveforms, Single Carrier (SC), Single-Carrier Frequency Division Multiple Access (SC-FDMA), Orthogonal Frequency Division Multiplexing (OFDM), and Linear Frequency Modulation (LFM), used in communication and radar networks. Synchronization of the signals is not needed as we assume there is an unknown and uncompensated time and frequency offset. An autoencoder with a deep CNN architecture is also examined to create a new fifth classification category of an unknown waveform type. This is accomplished by calculating a minimum and maximum threshold values from the root mean square error (RMSE) of the radar and communication waveforms. The classifier and autoencoder work together to monitor a spectrum area to identify the common waveforms inside the area of operation along with detecting unknown waveforms. Results from testing showed the classifier had 100\% classification rate above 0 dB with accuracy of 83.2\% and 94.7\% at -10 dB and -5 dB, respectively, with signal impairments present. Results for the anomaly detector showed 85.3\% accuracy at 0 dB with 100\% at SNR greater than 0 dB with signal impairments present when using a high-value Fast Fourier Transform (FFT) size. Accurate detection rates decline as additional noise is introduced to the signals, with 78.1\% at -5 dB and 56.5\% at -10 dB. However, these low rates seen can be potentially mitigated by using even higher FFT sizes also shown in our results.
△ Less
Submitted 16 August, 2021;
originally announced August 2021.
-
Is Cosmological Tuning Fine or Coarse?
Authors:
Daniel Andrés Díaz-Pachón,
Ola Hössjer,
Robert J. Marks II
Abstract:
The fine-tuning of the universe for life, the idea that the constants of nature (or ratios between them) must belong to very small intervals in order for life to exist, has been debated by scientists for several decades. Several criticisms have emerged concerning probabilistic measurement of life-permitting intervals. Herein, a Bayesian statistical approach is used to assign an upper bound for the…
▽ More
The fine-tuning of the universe for life, the idea that the constants of nature (or ratios between them) must belong to very small intervals in order for life to exist, has been debated by scientists for several decades. Several criticisms have emerged concerning probabilistic measurement of life-permitting intervals. Herein, a Bayesian statistical approach is used to assign an upper bound for the probability of tuning, which is invariant with respect to change of physical units, and under certain assumptions it is small whenever the life-permitting interval is small on a relative scale. The computation of the upper bound of the tuning probability is achieved by first assuming that the prior is chosen by the principle of maximum entropy (MaxEnt). The unknown parameters of this MaxEnt distribution are then handled in such a way that the weak anthropic principle is not violated. The MaxEnt assumption is "maximally noncommittal with regard to missing information." This approach is sufficiently general to be applied to constants of current cosmological models, or to other constants possibly under different models. Application of the MaxEnt model reveals, for example, that the ratio of the universal gravitational constant to the square of the Hubble constant is finely tuned in some cases, whereas the amplitude of primordial fluctuations is not.
△ Less
Submitted 13 July, 2021; v1 submitted 5 March, 2021;
originally announced April 2021.
-
Symbiotic Hybrid Neural Network Watchdog For Outlier Detection
Authors:
Justin Bui,
Robert J. Marks II
Abstract:
Neural networks are largely black boxes. A neural network trained to classify fruit may classify a picture of a giraffe as a banana. A neural network watchdog's job is to identify such inputs, allowing a classifier to disregard such data. We investigate whether the watchdog should be separate from the neural network or symbiotically attached. We present empirical evidence that the symbiotic watchd…
▽ More
Neural networks are largely black boxes. A neural network trained to classify fruit may classify a picture of a giraffe as a banana. A neural network watchdog's job is to identify such inputs, allowing a classifier to disregard such data. We investigate whether the watchdog should be separate from the neural network or symbiotically attached. We present empirical evidence that the symbiotic watchdog performs better than when the neural networks are disjoint.
△ Less
Submitted 21 April, 2021; v1 submitted 28 February, 2021;
originally announced March 2021.
-
Autoencoder Watchdog Outlier Detection for Classifiers
Authors:
Justin Bui,
Robert J Marks II
Abstract:
Neural networks have often been described as black boxes. A generic neural network trained to differentiate between kittens and puppies will classify a picture of a kumquat as a kitten or a puppy. An autoencoder watch dog screens trained classifier/regression machine input candidates before processing, e.g. to first test whether the neural network input is a puppy or a kitten. Preliminary results…
▽ More
Neural networks have often been described as black boxes. A generic neural network trained to differentiate between kittens and puppies will classify a picture of a kumquat as a kitten or a puppy. An autoencoder watch dog screens trained classifier/regression machine input candidates before processing, e.g. to first test whether the neural network input is a puppy or a kitten. Preliminary results are presented using convolutional neural networks and convolutional autoencoder watchdogs using MNIST images.
△ Less
Submitted 10 February, 2021; v1 submitted 23 October, 2020;
originally announced October 2020.
-
Range Improvement in Single-Beam Phased Array Radars by Amplifier Impedance Tuning
Authors:
Pedro Rodriguez-Garcia,
Jack Sifri,
Caleb Calabrese,
Charles Baylis,
Robert J. Marks II
Abstract:
Significant radar range degradation can be incurred due to variations in antenna impedance from changing array scan angle. Element-wise reconfigurable impedance tuners can be used to optimally match the power amplifier device; however, the impedance matching can also affect the array pattern. In this paper, the effects of element-wise impedance tuning on the transmitted power and on the array patt…
▽ More
Significant radar range degradation can be incurred due to variations in antenna impedance from changing array scan angle. Element-wise reconfigurable impedance tuners can be used to optimally match the power amplifier device; however, the impedance matching can also affect the array pattern. In this paper, the effects of element-wise impedance tuning on the transmitted power and on the array pattern are studied, and an approach is recommended for the element-wise implementation of impedance tuners. Examples of impact on array pattern and transmitted power are given using simulation of a designed switched-stub impedance tuner. As a result of these experiments, guidelines are developed for the creation of real-time circuit optimization techniques in the array elements.
△ Less
Submitted 29 January, 2020;
originally announced January 2020.
-
Linear State Feedback Stabilization on Time Scales
Authors:
Billy J. Jackson,
John M. Davis,
Ian A. Gravagne,
Robert J. Marks II
Abstract:
For a general class of dynamical systems (of which the canonical continuous and uniform discrete versions are but special cases), we prove that there is a state feedback gain such that the resulting closed-loop system is uniformly exponentially stable with a prescribed rate. The methods here generalize and extend Gramian-based linear state feedback control to much more general time domains, e.g.…
▽ More
For a general class of dynamical systems (of which the canonical continuous and uniform discrete versions are but special cases), we prove that there is a state feedback gain such that the resulting closed-loop system is uniformly exponentially stable with a prescribed rate. The methods here generalize and extend Gramian-based linear state feedback control to much more general time domains, e.g. nonuniform discrete or a combination of continuous and discrete time. In conclusion, we discuss an experimental implementation of this theory.
△ Less
Submitted 15 October, 2009;
originally announced October 2009.
-
Algebraic and Dynamic Lyapunov Equations on Time Scales
Authors:
John M. Davis,
Ian A. Gravagne,
Robert J. Marks II,
Alice A. Ramos
Abstract:
We revisit the canonical continuous-time and discrete-time matrix algebraic and matrix differential equations that play a central role in Lyapunov based stability arguments. The goal is to generalize and extend these types of equations and subsequent analysis to dynamical systems on domains other than $\R$ or $\Z$, e.g. nonuniform discrete domains or domains consisting of a mixture of discrete a…
▽ More
We revisit the canonical continuous-time and discrete-time matrix algebraic and matrix differential equations that play a central role in Lyapunov based stability arguments. The goal is to generalize and extend these types of equations and subsequent analysis to dynamical systems on domains other than $\R$ or $\Z$, e.g. nonuniform discrete domains or domains consisting of a mixture of discrete and continuous components. We compare and contrast the standard theory with the theory in this general case.
△ Less
Submitted 10 October, 2009;
originally announced October 2009.
-
Controllability, Observability, Realizability, and Stability of Dynamic Linear Systems
Authors:
John M. Davis,
Ian A. Gravagne,
Billy J. Jackson,
Robert J. Marks II
Abstract:
We develop a linear systems theory that coincides with the existing theories for continuous and discrete dynamical systems, but that also extends to linear systems defined on nonuniform time domains. The approach here is based on generalized Laplace transform methods (e.g. shifts and convolution) from our recent work \cite{DaGrJaMaRa}. We study controllability in terms of the controllability Gra…
▽ More
We develop a linear systems theory that coincides with the existing theories for continuous and discrete dynamical systems, but that also extends to linear systems defined on nonuniform time domains. The approach here is based on generalized Laplace transform methods (e.g. shifts and convolution) from our recent work \cite{DaGrJaMaRa}. We study controllability in terms of the controllability Gramian and various rank conditions (including Kalman's) in both the time invariant and time varying settings and compare the results. We also explore observability in terms of both Gramian and rank conditions as well as realizability results. We conclude by applying this systems theory to connect exponential and BIBO stability problems in this general setting. Numerous examples are included to show the utility of these results.
△ Less
Submitted 3 March, 2009; v1 submitted 23 January, 2009;
originally announced January 2009.
