-
Zero-failure testing of binary classifiers
Authors:
Ioannis Ivrissimtzis,
Matthew Houliston,
Shauna Concannon,
Graham Roberts
Abstract:
We propose using performance metrics derived from zero-failure testing to assess binary classifiers. The principal characteristic of the proposed approach is the asymmetric treatment of the two types of error. In particular, we construct a test set consisting of positive and negative samples, set the operating point of the binary classifier at the lowest value that will result to correct classific…
▽ More
We propose using performance metrics derived from zero-failure testing to assess binary classifiers. The principal characteristic of the proposed approach is the asymmetric treatment of the two types of error. In particular, we construct a test set consisting of positive and negative samples, set the operating point of the binary classifier at the lowest value that will result to correct classifications of all positive samples, and use the algorithm's success rate on the negative samples as a performance measure. A property of the proposed approach, setting it apart from other commonly used testing methods, is that it allows the construction of a series of tests of increasing difficulty, corresponding to a nested sequence of positive sample test sets. We illustrate the proposed method on the problem of age estimation for determining whether a subject is above a legal age threshold, a problem that exemplifies the asymmetry of the two types of error. Indeed, misclassifying an under-aged subject is a legal and regulatory issue, while misclassifications of people above the legal age is an efficiency issue primarily concerning the commercial user of the age estimation system.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
The Impact of 2D and 3D Gamified VR on Learning American Sign Language
Authors:
**di Wang,
Ioannis Ivrissimtzis,
Zhaoxing Li,
Lei Shi
Abstract:
Sign language has been extensively studied as a means of facilitating effective communication between hearing individuals and the deaf community. With the continuous advancements in virtual reality (VR) and gamification technologies, an increasing number of studies have begun to explore the application of these emerging technologies in sign language learning. This paper describes a user study that…
▽ More
Sign language has been extensively studied as a means of facilitating effective communication between hearing individuals and the deaf community. With the continuous advancements in virtual reality (VR) and gamification technologies, an increasing number of studies have begun to explore the application of these emerging technologies in sign language learning. This paper describes a user study that compares the impact of 2D and 3D games on the user experience in ASL learning. Empirical evidence gathered through questionnaires supports the positive impact of 3D game environments on user engagement and overall experience, particularly in relation to attractiveness, usability, and efficiency. Moreover, initial findings demonstrate a similar behaviour of 2D and 3D games in terms of enhancing user experience. Finally, the study identifies areas where improvements can be made to enhance the dependability and clarity of 3D game environments. These findings contribute to the understanding of how game-based approaches, and specifically the utilisation of 3D environments, can positively influence the learning experience of ASL.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Race Bias Analysis of Bona Fide Errors in face anti-spoofing
Authors:
Latifah Abduh,
Ioannis Ivrissimtzis
Abstract:
The study of bias in Machine Learning is receiving a lot of attention in recent years, however, few only papers deal explicitly with the problem of race bias in face anti-spoofing. In this paper, we present a systematic study of race bias in face anti-spoofing with three key characteristics: the focus is on analysing potential bias in the bona fide errors, where significant ethical and legal issue…
▽ More
The study of bias in Machine Learning is receiving a lot of attention in recent years, however, few only papers deal explicitly with the problem of race bias in face anti-spoofing. In this paper, we present a systematic study of race bias in face anti-spoofing with three key characteristics: the focus is on analysing potential bias in the bona fide errors, where significant ethical and legal issues lie; the analysis is not restricted to the final binary outcomes of the classifier, but also covers the classifier's scalar responses and its latent space; the threshold determining the operating point of the classifier is considered a variable. We demonstrate the proposed bias analysis process on a VQ-VAE based face anti-spoofing algorithm, trained on the Replay Attack and the Spoof in the Wild (SiW) databases, and analysed for bias on the SiW and Racial Faces in the Wild (RFW), databases. The results demonstrate that race bias is not necessarily the result of different mean response values among the various populations. Instead, it can be better understood as the combined effect of several possible characteristics of the response distributions: different means; different variances; bimodal behaviour; existence of outliers.
△ Less
Submitted 11 October, 2022;
originally announced October 2022.
-
Use of in-the-wild images for anomaly detection in face anti-spoofing
Authors:
Latifah Abduh,
Ioannis Ivrissimtzis
Abstract:
The traditional approach to face anti-spoofing sees it as a binary classification problem, and binary classifiers are trained and validated on specialized anti-spoofing databases. One of the drawbacks of this approach is that, due to the variability of face spoofing attacks, environmental factors, and the typically small sample size, such classifiers do not generalize well to previously unseen dat…
▽ More
The traditional approach to face anti-spoofing sees it as a binary classification problem, and binary classifiers are trained and validated on specialized anti-spoofing databases. One of the drawbacks of this approach is that, due to the variability of face spoofing attacks, environmental factors, and the typically small sample size, such classifiers do not generalize well to previously unseen databases. Anomaly detection, which approaches face anti-spoofing as a one-class classification problem, is emerging as an increasingly popular alternative approach. Nevertheless, in all existing work on anomaly detection for face anti-spoofing, the proposed training protocols utilize images from specialized anti-spoofing databases only, even though only common images of real faces are needed. Here, we explore the use of in-the-wild images, and images from non-specialized face databases, to train one-class classifiers for face anti-spoofing. Employing a well-established technique, we train a convolutional autoencoder on real faces and compare the reconstruction error of the input against a threshold to classify a face image accordingly as either client or imposter.
Our results show that the inclusion in the training set of in-the-wild images increases the discriminating power of the classifier significantly on an unseen database, as evidenced by a large increase in the value of the Area Under the Curve. In a limitation of our approach, we note that the problem of finding a suitable operating point on the unseen database remains a challenge, as evidenced by the values of the Half Total Error Rate.
△ Less
Submitted 18 June, 2020;
originally announced June 2020.
-
A study of the effect of the illumination model on the generation of synthetic training datasets
Authors:
Xin Zhang,
Ning Jia,
Ioannis Ivrissimtzis
Abstract:
The use of computer generated images to train Deep Neural Networks is a viable alternative to real images when the latter are scarce or expensive. In this paper, we study how the illumination model used by the rendering software affects the quality of the generated images. We created eight training sets, each one with a different illumination model, and tested them on three different network archi…
▽ More
The use of computer generated images to train Deep Neural Networks is a viable alternative to real images when the latter are scarce or expensive. In this paper, we study how the illumination model used by the rendering software affects the quality of the generated images. We created eight training sets, each one with a different illumination model, and tested them on three different network architectures, ResNet, U-Net and a combined architecture developed by us. The test set consisted of photos of 3D printed objects produced from the same CAD models used to generate the training set. The effect of the other parameters of the rendering process, such as textures and camera position, was randomized.
Our results show that the effect of the illumination model is important, comparable in significance to the network architecture. We also show that both light probes capturing natural environmental light, and modelled lighting environments, can give good results. In the case of light probes, we identified as two significant factors affecting performance the similarity between the light probe and the test environment, as well as the light probe's resolution. Regarding modelled lighting environment, similarity with the test environment was again identified as a significant factor.
△ Less
Submitted 15 June, 2020;
originally announced June 2020.
-
Using theoretical ROC curves for analysing machine learning binary classifiers
Authors:
Luma Omar,
Ioannis Ivrissimtzis
Abstract:
Most binary classifiers work by processing the input to produce a scalar response and comparing it to a threshold value. The various measures of classifier performance assume, explicitly or implicitly, probability distributions $P_s$ and $P_n$ of the response belonging to either class, probability distributions for the cost of each type of misclassification, and compute a performance score from th…
▽ More
Most binary classifiers work by processing the input to produce a scalar response and comparing it to a threshold value. The various measures of classifier performance assume, explicitly or implicitly, probability distributions $P_s$ and $P_n$ of the response belonging to either class, probability distributions for the cost of each type of misclassification, and compute a performance score from the expected cost.
In machine learning, classifier responses are obtained experimentally and performance scores are computed directly from them, without any assumptions on $P_s$ and $P_n$. Here, we argue that the omitted step of estimating theoretical distributions for $P_s$ and $P_n$ can be useful. In a biometric security example, we fit beta distributions to the responses of two classifiers, one based on logistic regression and one on ANNs, and use them to establish a categorisation into a small number of classes with different extremal behaviours at the ends of the ROC curves.
△ Less
Submitted 21 September, 2019;
originally announced September 2019.
-
From Farey fractions to the Klein quartic and beyond
Authors:
Ioannis Ivrissimtzis,
David Singerman,
James Strudwick
Abstract:
In his 1878/79 paper "Ueber die transformation siebenter ordnung der elliptischen functionen", Klein produced his famous 14-sided polygon representing the Klein quartic, his Riemann surface of genus 3 which has PSL(2,7) as its automorphism group. The construction and method of side pairings are fairly complicated. By considering the Farey map modulo 7 we show how to obtain a fundamental polygon fo…
▽ More
In his 1878/79 paper "Ueber die transformation siebenter ordnung der elliptischen functionen", Klein produced his famous 14-sided polygon representing the Klein quartic, his Riemann surface of genus 3 which has PSL(2,7) as its automorphism group. The construction and method of side pairings are fairly complicated. By considering the Farey map modulo 7 we show how to obtain a fundamental polygon for Klein's surface using arithmetic. Now the side pairings are immediate and essentially the same as in Klein's paper. We also extend this idea from 7 to 11 as Klein attempted to do in his follow up paper "Ueber die transformation elfter ordnung der elliptischen functionen", in 1879.
△ Less
Submitted 18 September, 2019;
originally announced September 2019.
-
Watermark retrieval from 3D printed objects via synthetic data training
Authors:
Xin Zhang,
Ning Jia,
Ioannis Ivrissimtzis
Abstract:
We present a deep neural network based method for the retrieval of watermarks from images of 3D printed objects. To deal with the variability of all possible 3D printing and image acquisition settings we train the network with synthetic data. The main simulator parameters such as texture, illumination and camera position are dynamically randomized in non-realistic ways, forcing the neural network…
▽ More
We present a deep neural network based method for the retrieval of watermarks from images of 3D printed objects. To deal with the variability of all possible 3D printing and image acquisition settings we train the network with synthetic data. The main simulator parameters such as texture, illumination and camera position are dynamically randomized in non-realistic ways, forcing the neural network to learn the intrinsic features of the 3D printed watermarks. At the end of the pipeline, the watermark, in the form of a two-dimensional bit array, is retrieved through a series of simple image processing and statistical operations applied on the confidence map generated by the neural network. The results demonstrate that the inclusion of synthetic DR data in the training set increases the generalization power of the network, which performs better on images from previously unseen 3D printed objects. We conclude that in our application domain of information retrieval from 3D printed objects, where access to the exact CAD files of the printed objects can be assumed, one can use inexpensive synthetic data to enhance neural network training, reducing the need for the labour intensive process of creating large amounts of hand labelled real data or the need to generate photorealistic synthetic data.
△ Less
Submitted 23 May, 2019;
originally announced May 2019.
-
Watermark Retrieval from 3D Printed Objects via Convolutional Neural Networks
Authors:
Xin Zhang,
Qian Wang,
Toby Breckon,
Ioannis Ivrissimtzis
Abstract:
We present a method for reading digital data embedded in planar 3D printed surfaces. The data are organised in binary arrays and embedded as surface textures in a way inspired by QR codes. At the core of the retrieval method lies a Convolutional Neural Network, outputting a confidence map of the location of the surface textures encoding value 1 bits. Subsequently, the bit array is retrieved throug…
▽ More
We present a method for reading digital data embedded in planar 3D printed surfaces. The data are organised in binary arrays and embedded as surface textures in a way inspired by QR codes. At the core of the retrieval method lies a Convolutional Neural Network, outputting a confidence map of the location of the surface textures encoding value 1 bits. Subsequently, the bit array is retrieved through a series of simple image processing and statistical operations applied on the confidence map. Extensive experimentation with images captured from various camera views, under various illumination conditions and from objects printed with various material colours, shows that the proposed method generalizes well and achieves the level of accuracy required in practical applications.
△ Less
Submitted 19 November, 2018;
originally announced November 2018.
-
Trivalent expanders, $(Δ-Y)$-transformation, and hyperbolic surfaces
Authors:
Ioannis Ivrissimtzis,
Norbert Peyerimhoff,
Alina Vdovina
Abstract:
We construct a new family of trivalent expanders tessellating hyperbolic surfaces with large isometry groups. These graphs are obtained from a family of Cayley graphs of nilpotent groups via $(Δ-Y)$-transformations. We compare this family with Platonic graphs and their associated hyperbolic surfaces and see that they are generally very different with only one hyperbolic surface in the intersection…
▽ More
We construct a new family of trivalent expanders tessellating hyperbolic surfaces with large isometry groups. These graphs are obtained from a family of Cayley graphs of nilpotent groups via $(Δ-Y)$-transformations. We compare this family with Platonic graphs and their associated hyperbolic surfaces and see that they are generally very different with only one hyperbolic surface in the intersection. Moreover, we study combinatorial, topological and spectral properties of our trivalent graphs and their associated hyperbolic surfaces.
△ Less
Submitted 11 October, 2018;
originally announced October 2018.
-
Simplicial Complex Entropy
Authors:
Stefan Dantchev,
Ioannis Ivrissimtzis
Abstract:
We propose an entropy function for simplicial complices. Its value gives the expected cost of the optimal encoding of sequences of vertices of the complex, when any two vertices belonging to the same simplex are indistinguishable. We show that the proposed entropy function can be computed efficiently. By computing the entropy of several complices consisting of hundreds of simplices, we show that t…
▽ More
We propose an entropy function for simplicial complices. Its value gives the expected cost of the optimal encoding of sequences of vertices of the complex, when any two vertices belonging to the same simplex are indistinguishable. We show that the proposed entropy function can be computed efficiently. By computing the entropy of several complices consisting of hundreds of simplices, we show that the proposed entropy function can be used in the analysis of the large sequences of simplicial complices that often appear in computational topology applications.
△ Less
Submitted 23 March, 2016;
originally announced March 2016.
-
Trivalent expanders and hyperbolic surfaces
Authors:
Ioannis Ivrissimtzis,
Norbert Peyerimhoff,
Alina Vdovina
Abstract:
We introduce a family of trivalent expanders which tessellate compact hyperbolic surfaces with large isometry groups. We compare this family with Platonic graphs and modifications of them and prove topological and spectral properties of these families.
We introduce a family of trivalent expanders which tessellate compact hyperbolic surfaces with large isometry groups. We compare this family with Platonic graphs and modifications of them and prove topological and spectral properties of these families.
△ Less
Submitted 10 February, 2012;
originally announced February 2012.
-
Spectral representations of vertex transitive graphs, Archimedean solids and finite Coxeter groups
Authors:
Ioannis Ivrissimtzis,
Norbert Peyerimhoff
Abstract:
In this article, we study eigenvalue functions of varying transition probability matrices on finite, vertex transitive graphs. We prove that the eigenvalue function of an eigenvalue of fixed higher multiplicity has a critical point if and only if the corresponding spectral representation is equilateral. We also show how the geometric realisation of a finite Coxeter group as a reflection group can…
▽ More
In this article, we study eigenvalue functions of varying transition probability matrices on finite, vertex transitive graphs. We prove that the eigenvalue function of an eigenvalue of fixed higher multiplicity has a critical point if and only if the corresponding spectral representation is equilateral. We also show how the geometric realisation of a finite Coxeter group as a reflection group can be used to obtain an explicit orthogonal system of eigenfunctions. Combining both results, we describe the behaviour of the spectral representations of the second highest eigenvalue function under the change of the transition probabilities in the case of Archimedean solids.
△ Less
Submitted 17 October, 2014; v1 submitted 13 June, 2011;
originally announced June 2011.
