Showing 1–12 of 12 results for author: Barney, E

Data Bias According to Bipol: Men are Naturally Right and It is the Role of Women to Follow Their Lead

Authors: Irene Pagliai, Goya van Boven, Tosin Adewumi, Lama Alkhaled, Namrata Gurung, Isabella Södergren, Elisa Barney

Abstract: We introduce new large labeled datasets on bias in 3 languages and show in experiments that bias exists in all 10 datasets of 5 languages evaluated, including benchmark datasets on the English GLUE/SuperGLUE leaderboards. The 3 new languages give a total of almost 6 million labeled samples and we benchmark on these datasets using SotA multilingual pretrained models: mT5 and mBERT. The challenge of… ▽ More We introduce new large labeled datasets on bias in 3 languages and show in experiments that bias exists in all 10 datasets of 5 languages evaluated, including benchmark datasets on the English GLUE/SuperGLUE leaderboards. The 3 new languages give a total of almost 6 million labeled samples and we benchmark on these datasets using SotA multilingual pretrained models: mT5 and mBERT. The challenge of social bias, based on prejudice, is ubiquitous, as recent events with AI and large language models (LLMs) have shown. Motivated by this challenge, we set out to estimate bias in multiple datasets. We compare some recent bias metrics and use bipol, which has explainability in the metric. We also confirm the unverified assumption that bias exists in toxic comments by randomly sampling 200 samples from a toxic dataset population using the confidence level of 95% and error margin of 7%. Thirty gold samples were randomly distributed in the 200 samples to secure the quality of the annotation. Our findings confirm that many of the datasets have male bias (prejudice against women), besides other types of bias. We publicly release our new datasets, lexica, models, and codes. △ Less

Submitted 7 April, 2024; originally announced April 2024.

Comments: 11 pages, 6 figures

On the Limitations of Large Language Models (LLMs): False Attribution

Authors: Tosin Adewumi, Nudrat Habib, Lama Alkhaled, Elisa Barney

Abstract: In this work, we provide insight into one important limitation of large language models (LLMs), i.e. false attribution, and introduce a new hallucination metric - Simple Hallucination Index (SHI). The task of automatic author attribution for relatively small chunks of text is an important NLP task but can be challenging. We empirically evaluate the power of 3 open SotA LLMs in zero-shot setting (L… ▽ More In this work, we provide insight into one important limitation of large language models (LLMs), i.e. false attribution, and introduce a new hallucination metric - Simple Hallucination Index (SHI). The task of automatic author attribution for relatively small chunks of text is an important NLP task but can be challenging. We empirically evaluate the power of 3 open SotA LLMs in zero-shot setting (LLaMA-2-13B, Mixtral 8x7B, and Gemma-7B), especially as human annotation can be costly. We collected the top 10 most popular books, according to Project Gutenberg, divided each one into equal chunks of 400 words, and asked each LLM to predict the author. We then randomly sampled 162 chunks for human evaluation from each of the annotated books, based on the error margin of 7% and a confidence level of 95% for the book with the most chunks (Great Expectations by Charles Dickens, having 922 chunks). The average results show that Mixtral 8x7B has the highest prediction accuracy, the lowest SHI, and a Pearson's correlation (r) of 0.737, 0.249, and -0.9996, respectively, followed by LLaMA-2-13B and Gemma-7B. However, Mixtral 8x7B suffers from high hallucinations for 3 books, rising as high as an SHI of 0.87 (in the range 0-1, where 1 is the worst). The strong negative correlation of accuracy and SHI, given by r, demonstrates the fidelity of the new hallucination metric, which is generalizable to other tasks. We publicly release the annotated chunks of data and our codes to aid the reproducibility and evaluation of other models. △ Less

Submitted 6 April, 2024; originally announced April 2024.

Comments: 8 pages, 5 figures

Instruction Makes a Difference

Authors: Tosin Adewumi, Nudrat Habib, Lama Alkhaled, Elisa Barney

Abstract: We introduce Instruction Document Visual Question Answering (iDocVQA) dataset and Large Language Document (LLaDoc) model, for training Language-Vision (LV) models for document analysis and predictions on document images, respectively. Usually, deep neural networks for the DocVQA task are trained on datasets lacking instructions. We show that using instruction-following datasets improves performanc… ▽ More We introduce Instruction Document Visual Question Answering (iDocVQA) dataset and Large Language Document (LLaDoc) model, for training Language-Vision (LV) models for document analysis and predictions on document images, respectively. Usually, deep neural networks for the DocVQA task are trained on datasets lacking instructions. We show that using instruction-following datasets improves performance. We compare performance across document-related datasets using the recent state-of-the-art (SotA) Large Language and Vision Assistant (LLaVA)1.5 as the base model. We also evaluate the performance of the derived models for object hallucination using the Polling-based Object Probing Evaluation (POPE) dataset. The results show that instruction-tuning performance ranges from 11X to 32X of zero-shot performance and from 0.1% to 4.2% over non-instruction (traditional task) finetuning. Despite the gains, these still fall short of human performance (94.36%), implying there's much room for improvement. △ Less

Submitted 13 June, 2024; v1 submitted 1 February, 2024; originally announced February 2024.

Comments: Accepted at the 16th IAPR International Workshop On Document Analysis Systems (DAS)

Imaging High Jitter, Very Fast Phenomena: A Remedy for Shutter Lag

Authors: Noah Hoppis, Kathryn M. Sturge, Jonathan E. Barney, Brian L. Beaudoin, Ariana M. Bussio, Ashley E. Hammell, Samuel L. Henderson, James E. Krutzler, Joseph P. Lichthardt, Alexander H. Mueller, Karl Smith, Bryce C. Tappan, Timothy W. Koeth

Abstract: Dielectric breakdown is an example of a natural phenomenon that occurs on very short time scales, making it incredibly difficult to capture optical images of the process. Event initiation jitter is one of the primary challenges, as even a microsecond of jitter time can cause the imaging attempt to fail. Initial attempts to capture images of dielectric breakdown with a gigahertz frame rate camera a… ▽ More Dielectric breakdown is an example of a natural phenomenon that occurs on very short time scales, making it incredibly difficult to capture optical images of the process. Event initiation jitter is one of the primary challenges, as even a microsecond of jitter time can cause the imaging attempt to fail. Initial attempts to capture images of dielectric breakdown with a gigahertz frame rate camera and an exploding bridge wire initiation were stymied by high initiation jitter. Subsequently, a novel optical delay line apparatus was developed in order to effectively circumvent the jitter and reliably image dielectric breakdown. The design and performance of the optical delay line apparatus are presented. The optical delay line increased the image capture success rate from 25% to 94% while also permitting enhanced temporal resolution and has applications for use in imaging other high-jitter, extremely fast phenomena. △ Less

Submitted 21 December, 2023; originally announced December 2023.

Comments: 11 pages, 6 figures, accepted manuscript

Journal ref: Rev. Sci. Instrum. 1 December 2023; 94 (12): 125109

ProCoT: Stimulating Critical Thinking and Writing of Students through Engagement with Large Language Models (LLMs)

Authors: Tosin Adewumi, Lama Alkhaled, Claudia Buck, Sergio Hernandez, Saga Brilioth, Mkpe Kekung, Yelvin Ragimov, Elisa Barney

Abstract: We introduce a novel writing method called Probing Chain-of-Thought (ProCoT), which potentially prevents students from cheating using a Large Language Model (LLM), such as ChatGPT, while enhancing their active learning. LLMs have disrupted education and many other fields. For fear of students cheating, many have resorted to banning their use. These LLMs are also known for hallucinations. We conduc… ▽ More We introduce a novel writing method called Probing Chain-of-Thought (ProCoT), which potentially prevents students from cheating using a Large Language Model (LLM), such as ChatGPT, while enhancing their active learning. LLMs have disrupted education and many other fields. For fear of students cheating, many have resorted to banning their use. These LLMs are also known for hallucinations. We conduct studies with ProCoT in two different courses with 65 students. The students in each course were asked to prompt an LLM of their choice with one question from a set of four and required to affirm or refute statements in the LLM output by using peer-reviewed references. The results show two things: (1) ProCoT stimulates creative/critical thinking and writing of students through engagement with LLMs when we compare the LLM-only output to ProCoT output and (2) ProCoT can prevent cheating because of clear limitations in existing LLMs, particularly ChatGPT, when we compare students' ProCoT output to LLM ProCoT output. We also discover that most students prefer to give answers in fewer words than LLMs, which are typically verbose. The average word counts for students in the first course, ChatGPT (v3.5), and Phind (v8) are 208, 391 and 383, respectively. △ Less

Submitted 1 May, 2024; v1 submitted 15 December, 2023; originally announced December 2023.

Comments: 8 pages, 4 figures

Learning Oculomotor Behaviors from Scanpath

Authors: Beibin Li, Nicholas Nuechterlein, Erin Barney, Claire Foster, Minah Kim, Monique Mahony, Adham Atyabi, Li Feng, Quan Wang, Pamela Ventola, Linda Shapiro, Frederick Shic

Abstract: Identifying oculomotor behaviors relevant for eye-tracking applications is a critical but often challenging task. Aiming to automatically learn and extract knowledge from existing eye-tracking data, we develop a novel method that creates rich representations of oculomotor scanpaths to facilitate the learning of downstream tasks. The proposed stimulus-agnostic Oculomotor Behavior Framework (OBF) mo… ▽ More Identifying oculomotor behaviors relevant for eye-tracking applications is a critical but often challenging task. Aiming to automatically learn and extract knowledge from existing eye-tracking data, we develop a novel method that creates rich representations of oculomotor scanpaths to facilitate the learning of downstream tasks. The proposed stimulus-agnostic Oculomotor Behavior Framework (OBF) model learns human oculomotor behaviors from unsupervised and semi-supervised tasks, including reconstruction, predictive coding, fixation identification, and contrastive learning tasks. The resultant pre-trained OBF model can be used in a variety of applications. Our pre-trained model outperforms baseline approaches and traditional scanpath methods in autism spectrum disorder and viewed-stimulus classification tasks. Ablation experiments further show our proposed method could achieve even better results with larger model sizes and more diverse eye-tracking training datasets, supporting the model's potential for future eye-tracking applications. Open source code: http://github.com/BeibinLi/OBF. △ Less

Submitted 11 August, 2021; originally announced August 2021.

Comments: Accepted ACM ICMI 2021

A study of MIR photoluminescence from Pr$^{3+}$ doped chalcogenide fibers pumped at near-infrared wavelengths

Authors: S. Sujecki, L. Sojka, E. Beres-Pawlik, R. Piramidowicz, H. Sakr, Z. Tang, E. Barney, D. Furniss, T. M. Benson, A. B. Seddon

Abstract: We perform a numerical analysis of mid-infrared photoluminescence emitted by praseodymium (III) doped chalcogenide selenide glass pumped at near-infrared wavelengths. The results obtained show that an effective inversion of level populations can be achieved using both 1480 nm and 1595 nm laser diodes. The rate of the spontaneous emission achieved when pum** at 1480 nm and 1595 nm is comparable t… ▽ More We perform a numerical analysis of mid-infrared photoluminescence emitted by praseodymium (III) doped chalcogenide selenide glass pumped at near-infrared wavelengths. The results obtained show that an effective inversion of level populations can be achieved using both 1480 nm and 1595 nm laser diodes. The rate of the spontaneous emission achieved when pum** at 1480 nm and 1595 nm is comparable to this achieved using the standard pum** wavelength of 2040 nm. △ Less

Submitted 27 April, 2021; originally announced April 2021.

Comments: 7 pages, 8 figures

MSC Class: 00A79 ACM Class: J.2

Ultra-broadband mid-infrared emission from Pr$^{3+}$/Dy$^{3+}$ co-doped selenide-chalcogenide glass fiber spectrally shaped by varying the pum** arrangement

Authors: Lukasz Sojka, Zhuoqi Tang, Dinuka Jayasuriya, Meili Shen, David Furniss, Emma Barney, Trevor M. Benson, Angela B. Seddon, Slawomir Sujecki

Abstract: In this contribution, a comprehensive experimental study of photoluminescence from Pr3+/Dy3+ co-doped selenide-chalcogenide multimode fiber samples is discussed. The selenide-chalcogenide multimode fiber samples co-doped with 500 ppm of Pr3+ ions and 500 ppm of Dy3+ ions are prepared using conventional melt-quenching. The main objective of the study is the analysis of the pum** wavelength select… ▽ More In this contribution, a comprehensive experimental study of photoluminescence from Pr3+/Dy3+ co-doped selenide-chalcogenide multimode fiber samples is discussed. The selenide-chalcogenide multimode fiber samples co-doped with 500 ppm of Pr3+ ions and 500 ppm of Dy3+ ions are prepared using conventional melt-quenching. The main objective of the study is the analysis of the pum** wavelength selection on the shape of the output spectrum. For this purpose, the Pr3+/Dy3+ co-doped selenide-chalcogenide multimode fiber samples are illuminated at one end using pump lasers operating at the wavelengths of 1320 nm , 1511 nm and 1700 nm. The results obtained show that the Pr3+/Dy3+ ion co-doped selenide-chalcogenide multimode fiber emits photoluminescence spanning from 2000 nm to 6000 nm. Also it is demonstrated that, by varying the output power and wavelength of the pump sources, the spectral shape of the emitted luminescence can be modified to either reduce or enhance the contribution of radiation within a particular wavelength band. The presented results confirm that Pr3+/Dy3+ co-doped selenide-chalcogenide multimode fiber is a good candidate for the realization of broadband spontaneous emission fiber sources with shaped output spectrum for the mid-infrared wavelength region. △ Less

Submitted 28 April, 2021; originally announced April 2021.

Comments: 17 pages, 14 figures

MSC Class: 00A79 ACM Class: J.2

Spatiotemporal modeling of mid-infrared photoluminescence from terbium (iii) ion doped chalcogenide-selenide multimode fibers

Authors: Slawomir Sujecki, Lukasz Sojka, Zhuoqi Tang, Dinuka Jayasuriya, David Furniss, Emma Barney, Trevor Benson, Angela Seddon

Abstract: In this contribution a numerical model is developed to study the time dynamics of photoluminescence emitted by Tb3+ doped multimode chalcogenide-selenide glass fibers pumped by laser light at approximately 2 microns. The model consists of a set of partial differential equations (PDEs), which describe the temporal and spatial evolution of the photon density and level populations within the fiber. I… ▽ More In this contribution a numerical model is developed to study the time dynamics of photoluminescence emitted by Tb3+ doped multimode chalcogenide-selenide glass fibers pumped by laser light at approximately 2 microns. The model consists of a set of partial differential equations (PDEs), which describe the temporal and spatial evolution of the photon density and level populations within the fiber. In order to solve numerically the PDEs a Method of Lines is applied. The modeling parameters are extracted from measurements and from data available in the literature. The numerical results obtained support experimental observations. In particular, the developed model reproduces the discrepancies that are observed between the photoluminescence decay curves obtained from different points along the fiber. The numerical analysis is also used to explain the source of these discrepancies. △ Less

Submitted 27 April, 2021; originally announced April 2021.

Comments: 16 pages, 9 figures

MSC Class: 00A79 ACM Class: J.2

Sparsely Grouped Input Variables for Neural Networks

Authors: Beibin Li, Nicholas Nuechterlein, Erin Barney, Caitlin Hudac, Pamela Ventola, Linda Shapiro, Frederick Shic

Abstract: In genomic analysis, biomarker discovery, image recognition, and other systems involving machine learning, input variables can often be organized into different groups by their source or semantic category. Eliminating some groups of variables can expedite the process of data acquisition and avoid over-fitting. Researchers have used the group lasso to ensure group sparsity in linear models and have… ▽ More In genomic analysis, biomarker discovery, image recognition, and other systems involving machine learning, input variables can often be organized into different groups by their source or semantic category. Eliminating some groups of variables can expedite the process of data acquisition and avoid over-fitting. Researchers have used the group lasso to ensure group sparsity in linear models and have extended it to create compact neural networks in meta-learning. Different from previous studies, we use multi-layer non-linear neural networks to find sparse groups for input variables. We propose a new loss function to regularize parameters for grouped input variables, design a new optimization algorithm for this loss function, and test these methods in three real-world settings. We achieve group sparsity for three datasets, maintaining satisfying results while excluding one nucleotide position from an RNA splicing experiment, excluding 89.9% of stimuli from an eye-tracking experiment, and excluding 60% of image rows from an experiment on the MNIST dataset. △ Less

Submitted 29 November, 2019; originally announced November 2019.

Clogging by sieving in microchannels: Application to the detection of contaminants in colloidal suspensions

Authors: Alban Sauret, Erin C. Barney, Adeline Perro, Emmanuel Villermaux, Howard A. Stone, Emilie Dressaire

Abstract: We report on a microfluidic method that allows measurement of a small concentration of large contaminants in suspensions of solid micrometer-scale particles. To perform the measurement, we flow the colloidal suspension through a series of constrictions, i.e. a microchannel of varying cross-section. We show and quantify the role of large contaminants in the formation of clogs at a constriction and… ▽ More We report on a microfluidic method that allows measurement of a small concentration of large contaminants in suspensions of solid micrometer-scale particles. To perform the measurement, we flow the colloidal suspension through a series of constrictions, i.e. a microchannel of varying cross-section. We show and quantify the role of large contaminants in the formation of clogs at a constriction and the growth of the resulting filter cake. By measuring the time interval between two clogging events in an array of parallel microchannels, we are able to estimate the concentration of contaminants whose size is selected by the geometry of the microfluidic device. This technique for characterizing colloidal suspensions offers a versatile and rapid tool to explore the role of contaminants on the properties of the suspensions. △ Less

Submitted 22 August, 2014; originally announced August 2014.

Journal ref: Applied Physics Letters 105 (2014) 074101

Structure and properties of an amorphous metal--organic framework

Authors: Thomas D. Bennett, Andrew L. Goodwin, Martin T. Dove, David A. Keen, Matthew G. Tucker, Emma R. Barney, Alan K. Soper, Erica G. Bithell, **-Chong Tan, Anthony K. Cheetham

Abstract: We show that ZIF-4, a metal-organic framework (MOF) with a zeolitic structure, undergoes a crystal--amorphous transition on heating to 300 $^\circ$C. The amorphous form, which we term a-ZIF, is recoverable to ambient conditions or may be converted to a dense crystalline phase of the same composition by heating to 400 $^\circ$C. Neutron and X-ray total scattering data collected during the amorphi… ▽ More We show that ZIF-4, a metal-organic framework (MOF) with a zeolitic structure, undergoes a crystal--amorphous transition on heating to 300 $^\circ$C. The amorphous form, which we term a-ZIF, is recoverable to ambient conditions or may be converted to a dense crystalline phase of the same composition by heating to 400 $^\circ$C. Neutron and X-ray total scattering data collected during the amorphization process are used as a basis for reverse Monte Carlo refinement of an atomistic model of the structure of a-ZIF. We show that the structure is best understood in terms of a continuous random network analogous to that of a-SiO$_2$. Optical microscopy, electron diffraction and nanoindentation measurements reveal a-ZIF to be an isotropic glass-like phase capable of plastic flow on its formation. Our results suggest an avenue for designing broad new families of amorphous and glass-like materials that exploit the chemical and structural diversity of MOFs. △ Less

Submitted 8 January, 2010; originally announced January 2010.

Comments: 4 pages, formatted for PRL

Journal ref: Physical Review Letters 104, 115503 (2010)