-
Evaluating Proposed Fairness Models for Face Recognition Algorithms
Authors:
John J. Howard,
Eli J. Laird,
Yevgeniy B. Sirotin,
Rebecca E. Rubin,
Jerry L. Tipton,
Arun R. Vemury
Abstract:
The development of face recognition algorithms by academic and commercial organizations is growing rapidly due to the onset of deep learning and the widespread availability of training data. Though tests of face recognition algorithm performance indicate yearly performance gains, error rates for many of these systems differ based on the demographic composition of the test set. These "demographic d…
▽ More
The development of face recognition algorithms by academic and commercial organizations is growing rapidly due to the onset of deep learning and the widespread availability of training data. Though tests of face recognition algorithm performance indicate yearly performance gains, error rates for many of these systems differ based on the demographic composition of the test set. These "demographic differentials" in algorithm performance can contribute to unequal or unfair outcomes for certain groups of people, raising concerns with increased worldwide adoption of face recognition systems. Consequently, regulatory bodies in both the United States and Europe have proposed new rules requiring audits of biometric systems for "discriminatory impacts" (European Union Artificial Intelligence Act) and "fairness" (U.S. Federal Trade Commission). However, no standard for measuring fairness in biometric systems yet exists. This paper characterizes two proposed measures of face recognition algorithm fairness (fairness measures) from scientists in the U.S. and Europe. We find that both proposed methods are challenging to interpret when applied to disaggregated face recognition error rates as they are commonly experienced in practice. To address this, we propose a set of interpretability criteria, termed the Functional Fairness Measure Criteria (FFMC), that outlines a set of properties desirable in a face recognition algorithm fairness measure. We further develop a new fairness measure, the Gini Aggregation Rate for Biometric Equitability (GARBE), and show how, in conjunction with the Pareto optimization, this measure can be used to select among alternative algorithms based on the accuracy/fairness trade-space. Finally, we have open-sourced our dataset of machine-readable, demographically disaggregated error rates. We believe this is currently the largest open-source dataset of its kind.
△ Less
Submitted 9 March, 2022;
originally announced March 2022.
-
Modeling Needs for High Power Target
Authors:
Charlotte Barbier,
Sujit Bidhar,
Marco Calviani,
Jeff Dooling,
Jian Gao,
Aaron Jacques,
Wei Lu,
Roberto Li Voti,
Frederique Pellemoine,
Justin Mach,
David Senor,
Fernando Sordo,
Izabela Szlufarska,
Joseph Tipton,
Dan Wilcox,
Drew Winder
Abstract:
The next generation of high power targets will use more complex geometries, novel materials, and new concepts (like flowing granular materials); however, the current numerical approaches will not be sufficient to converge towards a reliable target design that satisfies the physical requirements. We will discuss what can be improved in the next 10 years in target modeling to support high power (MW…
▽ More
The next generation of high power targets will use more complex geometries, novel materials, and new concepts (like flowing granular materials); however, the current numerical approaches will not be sufficient to converge towards a reliable target design that satisfies the physical requirements. We will discuss what can be improved in the next 10 years in target modeling to support high power (MW class) targets.
△ Less
Submitted 9 March, 2022;
originally announced March 2022.
-
A Unified Approach to Computing the Zeros of Classical Orthogonal Polynomials
Authors:
Ridha Moussa,
James Tipton
Abstract:
The authors present a unified method for calculating the zeros of the classical orthogonal polynomials based upon the electrostatic interpretation and its connection to the energy minimization problem. Examples are given with error estimates for three cases of the Jacobi polynomials, three cases of the Laguerre polynomials, and the Hermite polynomials. In the case of the Chebyshev polynomials, exa…
▽ More
The authors present a unified method for calculating the zeros of the classical orthogonal polynomials based upon the electrostatic interpretation and its connection to the energy minimization problem. Examples are given with error estimates for three cases of the Jacobi polynomials, three cases of the Laguerre polynomials, and the Hermite polynomials. In the case of the Chebyshev polynomials, exact errors are given.
△ Less
Submitted 19 September, 2021;
originally announced September 2021.
-
Reliability and Validity of Image-Based and Self-Reported Skin Phenotype Metrics
Authors:
John J. Howard,
Yevgeniy B. Sirotin,
Jerry L. Tipton,
Arun R. Vemury
Abstract:
With increasing adoption of face recognition systems, it is important to ensure adequate performance of these technologies across demographic groups. Recently, phenotypes such as skin-tone, have been proposed as superior alternatives to traditional race categories when exploring performance differentials. However, there is little consensus regarding how to appropriately measure skin-tone in evalua…
▽ More
With increasing adoption of face recognition systems, it is important to ensure adequate performance of these technologies across demographic groups. Recently, phenotypes such as skin-tone, have been proposed as superior alternatives to traditional race categories when exploring performance differentials. However, there is little consensus regarding how to appropriately measure skin-tone in evaluations of biometric performance or in AI more broadly. In this study, we explore the relationship between face-area-lightness-measures (FALMs) estimated from images and ground-truth skin readings collected using a device designed to measure human skin. FALMs estimated from different images of the same individual varied significantly relative to ground-truth FALM. This variation was only reduced by greater control of acquisition (camera, background, and environment). Next, we compare ground-truth FALM to Fitzpatrick Skin Types (FST) categories obtained using the standard, in-person, medical survey and show FST is poorly predictive of skin-tone. Finally, we show how noisy estimation of FALM leads to errors selecting explanatory factors for demographic differentials. These results demonstrate that measures of skin-tone for biometric performance evaluations must come from objective, characterized, and controlled sources. Further, despite this being a currently practiced approach, estimating FST categories and FALMs from uncontrolled imagery does not provide an appropriate measure of skin-tone.
△ Less
Submitted 18 June, 2021;
originally announced June 2021.
-
Quantifying the Extent to Which Race and Gender Features Determine Identity in Commercial Face Recognition Algorithms
Authors:
John J. Howard,
Yevgeniy B. Sirotin,
Jerry L. Tipton,
Arun R. Vemury
Abstract:
Human face features can be used to determine individual identity as well as demographic information like gender and race. However, the extent to which black-box commercial face recognition algorithms (CFRAs) use gender and race features to determine identity is poorly understood despite increasing deployments by government and industry. In this study, we quantified the degree to which gender and r…
▽ More
Human face features can be used to determine individual identity as well as demographic information like gender and race. However, the extent to which black-box commercial face recognition algorithms (CFRAs) use gender and race features to determine identity is poorly understood despite increasing deployments by government and industry. In this study, we quantified the degree to which gender and race features influenced face recognition similarity scores between different people, i.e. non-mated scores. We ran this study using five different CFRAs and a sample of 333 diverse test subjects. As a control, we compared the behavior of these non-mated distributions to a commercial iris recognition algorithm (CIRA). Confirming prior work, all CFRAs produced higher similarity scores for people of the same gender and race, an effect known as "broad homogeneity". No such effect was observed for the CIRA. Next, we applied principal components analysis (PCA) to similarity score matrices. We show that some principal components (PCs) of CFRAs cluster people by gender and race, but the majority do not. Demographic clustering in the PCs accounted for only 10 % of the total CFRA score variance. No clustering was observed for the CIRA. This demonstrates that, although CFRAs use some gender and race features to establish identity, most features utilized by current CFRAs are unrelated to gender and race, similar to the iris texture patterns utilized by the CIRA. Finally, reconstruction of similarity score matrices using only PCs that showed no demographic clustering reduced broad homogeneity effects, but also decreased the separation between mated and non-mated scores. This suggests it's possible for CFRAs to operate on features unrelated to gender and race, albeit with somewhat lower recognition accuracy, but that this is not the current commercial practice.
△ Less
Submitted 15 October, 2020;
originally announced October 2020.
-
Identifying latent classes with ordered categorical indicators
Authors:
R. Noah Padgett,
Rebecca J. Tipton
Abstract:
A Monte Carlo simulation was used to determine which assumptions for ordered categorical data, continuity vs. discrete categories, most frequently identifies the underlying factor structure when a response variable has five ordered categories. The impact of infrequently endorsed response categories was also examined, a condition that has not been fully explored. The typical method for overcoming i…
▽ More
A Monte Carlo simulation was used to determine which assumptions for ordered categorical data, continuity vs. discrete categories, most frequently identifies the underlying factor structure when a response variable has five ordered categories. The impact of infrequently endorsed response categories was also examined, a condition that has not been fully explored. The typical method for overcoming infrequently endorsed categories in applied research is to collapse response options with adjacent categories resulting in less response categories that are endorsed more frequently, but this approach may not necessarily provide useful information. Response category endorsement issues have been studied in Item Response Theory, but this issue has not been addressed in classification analyses nor has fit measure performance been examined under these conditions. We found that the performance of commonly used fit statistics to identify the true number of latent class depends on the whether continuity is assumed, sample size, and convergence. Fit statistics performed best when the five response options are assumed to be categorical. However, in situations with lower sample sizes and when convergence is an issue, assuming continuity and using the adjusted Lo-Mendell-Rubin likelihood ratio test may be useful.
△ Less
Submitted 15 September, 2020;
originally announced September 2020.
-
Predicting paleoclimate from compositional data using multivariate Gaussian process inverse prediction
Authors:
John R. Tipton,
Mevin B. Hooten,
Connor Nolan,
Robert K. Booth,
Jason McLachlan
Abstract:
Multivariate compositional count data arise in many applications including ecology, microbiology, genetics, and paleoclimate. A frequent question in the analysis of multivariate compositional count data is what values of a covariate(s) give rise to the observed composition. Learning the relationship between covariates and the compositional count allows for inverse prediction of unobserved covariat…
▽ More
Multivariate compositional count data arise in many applications including ecology, microbiology, genetics, and paleoclimate. A frequent question in the analysis of multivariate compositional count data is what values of a covariate(s) give rise to the observed composition. Learning the relationship between covariates and the compositional count allows for inverse prediction of unobserved covariates given compositional count observations. Gaussian processes provide a flexible framework for modeling functional responses with respect to a covariate without assuming a functional form. Many scientific disciplines use Gaussian process approximations to improve prediction and make inference on latent processes and parameters. When prediction is desired on unobserved covariates given realizations of the response variable, this is called inverse prediction. Because inverse prediction is mathematically and computationally challenging, predicting unobserved covariates often requires fitting models that are different from the hypothesized generative model. We present a novel computational framework that allows for efficient inverse prediction using a Gaussian process approximation to generative models. Our framework enables scientific learning about how the latent processes co-vary with respect to covariates while simultaneously providing predictions of missing covariates. The proposed framework is capable of efficiently exploring the high dimensional, multi-modal latent spaces that arise in the inverse problem. To demonstrate flexibility, we apply our method in a generalized linear model framework to predict latent climate states given multivariate count data. Based on cross-validation, our model has predictive skill competitive with current methods while simultaneously providing formal, statistical inference on the underlying community dynamics of the biological system previously not available.
△ Less
Submitted 12 March, 2019;
originally announced March 2019.
-
Partial Classification of Polynomials and an Orthonormal Basis Construction on the Associated Basin of Attraction
Authors:
James Tipton
Abstract:
In the paper "Infinite product representations for kernels and iterations of functions", the authors associate certain Fatou subsets with reproducing kernel Hilbert spaces. They also present a method for constructing an orthonormal basis for said Hilbert space, but the method depends on the polynomial of the given Fatou set. We provide a partial classification of those polynomials the method appli…
▽ More
In the paper "Infinite product representations for kernels and iterations of functions", the authors associate certain Fatou subsets with reproducing kernel Hilbert spaces. They also present a method for constructing an orthonormal basis for said Hilbert space, but the method depends on the polynomial of the given Fatou set. We provide a partial classification of those polynomials the method applies to.
△ Less
Submitted 23 December, 2018;
originally announced December 2018.
-
The basis function approach for modeling autocorrelation in ecological data
Authors:
Trevor J. Hefley,
Kristin M. Broms,
Brian M. Brost,
Frances E. Buderman,
Shannon L. Kay,
Henry R. Scharf,
John R. Tipton,
Perry J. Williams,
Mevin B. Hooten
Abstract:
Analyzing ecological data often requires modeling the autocorrelation created by spatial and temporal processes. Many of the statistical methods used to account for autocorrelation can be viewed as regression models that include basis functions. Understanding the concept of basis functions enables ecologists to modify commonly used ecological models to account for autocorrelation, which can improv…
▽ More
Analyzing ecological data often requires modeling the autocorrelation created by spatial and temporal processes. Many of the statistical methods used to account for autocorrelation can be viewed as regression models that include basis functions. Understanding the concept of basis functions enables ecologists to modify commonly used ecological models to account for autocorrelation, which can improve inference and predictive accuracy. Understanding the properties of basis functions is essential for evaluating the fit of spatial or time-series models, detecting a hidden form of multicollinearity, and analyzing large data sets. We present important concepts and properties related to basis functions and illustrate several tools and techniques ecologists can use when modeling autocorrelation in ecological data.
△ Less
Submitted 17 June, 2016;
originally announced June 2016.
-
Dynamics of Orthonormal Bases Associated to Basins of Attraction
Authors:
James Tipton
Abstract:
In the paper "Infinite Product Represenations for Kernels and Iterations of Functions", a technique was developed which allows for the construction of a reproducing kernel Hilbert space on basins of attraction containing $0$. When the right conditions are met, an explicit orthonormal basis can be constructed using a particular class of operators. It is natural then to consider how the orthonormal…
▽ More
In the paper "Infinite Product Represenations for Kernels and Iterations of Functions", a technique was developed which allows for the construction of a reproducing kernel Hilbert space on basins of attraction containing $0$. When the right conditions are met, an explicit orthonormal basis can be constructed using a particular class of operators. It is natural then to consider how the orthonormal basis changes as we let the basin of attraction vary. We will consider this question for the basins of attraction containing $0$ of the family of polynomials $\mathcal{F} = \{az^{2^{n+2}}-2az^{2^{n+1}}:a\neq0\}$, where $n\in\mathbb{N}$.
△ Less
Submitted 5 January, 2017; v1 submitted 20 November, 2015;
originally announced November 2015.
-
Multiplier Sequences for Simple Sets of Polynomials
Authors:
Tamás Forgács,
James Tipton,
Benjamin Wright
Abstract:
In this paper we give a new characterization of simple sets of polynomials B with the property that the set of B-multiplier sequences contains all Q-multiplier sequence for every simple set Q. We characterize sequences of real numbers which are multiplier sequences for every simple set Q, and obtain some results toward the partitioning of the set of classical multiplier sequences.
In this paper we give a new characterization of simple sets of polynomials B with the property that the set of B-multiplier sequences contains all Q-multiplier sequence for every simple set Q. We characterize sequences of real numbers which are multiplier sequences for every simple set Q, and obtain some results toward the partitioning of the set of classical multiplier sequences.
△ Less
Submitted 29 August, 2011;
originally announced August 2011.
-
The Kauffman polynomial and trivalent graphs
Authors:
Carmen Caprau,
James Tipton
Abstract:
We construct a state model for the two-variable Kauffman polynomial using planar trivalent graphs. We also use this model to obtain a polynomial invariant for a certain type of trivalent graphs embedded in three-dimensional space.
We construct a state model for the two-variable Kauffman polynomial using planar trivalent graphs. We also use this model to obtain a polynomial invariant for a certain type of trivalent graphs embedded in three-dimensional space.
△ Less
Submitted 24 May, 2012; v1 submitted 6 July, 2011;
originally announced July 2011.
-
Two-electron quantum dots as scalable qubits
Authors:
J. H. Jefferson,
M. Fearn,
D. L. J. Tipton,
T. P. Spiller
Abstract:
We show that two electrons confined in a square semiconductor quantum dot have two isolated low-lying energy eigenstates, which have the potential to form the basis of scalable computing elements (qubits). Initialisation, one-qubit and two-qubit universal gates, and readout are performed using electrostatic gates and magnetic fields. Two-qubit transformations are performed via the Coulomb intera…
▽ More
We show that two electrons confined in a square semiconductor quantum dot have two isolated low-lying energy eigenstates, which have the potential to form the basis of scalable computing elements (qubits). Initialisation, one-qubit and two-qubit universal gates, and readout are performed using electrostatic gates and magnetic fields. Two-qubit transformations are performed via the Coulomb interaction between electrons on adjacent dots. Choice of initial states and subsequent asymmetric tuning of the tunnelling energy parameters on adjacent dots control the effect of this interaction.
△ Less
Submitted 4 September, 2002; v1 submitted 12 June, 2002;
originally announced June 2002.
