-
Fairness and Bias in Multimodal AI: A Survey
Authors:
Tosin Adewumi,
Lama Alkhaled,
Namrata Gurung,
Goya van Boven,
Irene Pagliai
Abstract:
The importance of addressing fairness and bias in artificial intelligence (AI) systems cannot be over-emphasized. Mainstream media has been awashed with news of incidents around stereotypes and bias in many of these systems in recent years. In this survey, we fill a gap with regards to the minimal study of fairness and bias in Large Multimodal Models (LMMs) compared to Large Language Models (LLMs)…
▽ More
The importance of addressing fairness and bias in artificial intelligence (AI) systems cannot be over-emphasized. Mainstream media has been awashed with news of incidents around stereotypes and bias in many of these systems in recent years. In this survey, we fill a gap with regards to the minimal study of fairness and bias in Large Multimodal Models (LMMs) compared to Large Language Models (LLMs), providing 50 examples of datasets and models along with the challenges affecting them; we identify a new category of quantifying bias (preuse), in addition to the two well-known ones in the literature: intrinsic and extrinsic; we critically discuss the various ways researchers are addressing these challenges. Our method involved two slightly different search queries on Google Scholar, which revealed that 33,400 and 538,000 links are the results for the terms "Fairness and bias in Large Multimodal Models" and "Fairness and bias in Large Language Models", respectively. We believe this work contributes to filling this gap and providing insight to researchers and other stakeholders on ways to address the challenge of fairness and bias in multimodal A!.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Data Bias According to Bipol: Men are Naturally Right and It is the Role of Women to Follow Their Lead
Authors:
Irene Pagliai,
Goya van Boven,
Tosin Adewumi,
Lama Alkhaled,
Namrata Gurung,
Isabella Södergren,
Elisa Barney
Abstract:
We introduce new large labeled datasets on bias in 3 languages and show in experiments that bias exists in all 10 datasets of 5 languages evaluated, including benchmark datasets on the English GLUE/SuperGLUE leaderboards. The 3 new languages give a total of almost 6 million labeled samples and we benchmark on these datasets using SotA multilingual pretrained models: mT5 and mBERT. The challenge of…
▽ More
We introduce new large labeled datasets on bias in 3 languages and show in experiments that bias exists in all 10 datasets of 5 languages evaluated, including benchmark datasets on the English GLUE/SuperGLUE leaderboards. The 3 new languages give a total of almost 6 million labeled samples and we benchmark on these datasets using SotA multilingual pretrained models: mT5 and mBERT. The challenge of social bias, based on prejudice, is ubiquitous, as recent events with AI and large language models (LLMs) have shown. Motivated by this challenge, we set out to estimate bias in multiple datasets. We compare some recent bias metrics and use bipol, which has explainability in the metric. We also confirm the unverified assumption that bias exists in toxic comments by randomly sampling 200 samples from a toxic dataset population using the confidence level of 95% and error margin of 7%. Thirty gold samples were randomly distributed in the 200 samples to secure the quality of the annotation. Our findings confirm that many of the datasets have male bias (prejudice against women), besides other types of bias. We publicly release our new datasets, lexica, models, and codes.
△ Less
Submitted 7 April, 2024;
originally announced April 2024.
-
Context-based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting
Authors:
Edgar Medina,
Leyong Loh,
Namrata Gurung,
Kyung Hun Oh,
Niels Heller
Abstract:
Human motion prediction is still an open problem extremely important for autonomous driving and safety applications. Due to the complex spatiotemporal relation of motion sequences, this remains a challenging problem not only for movement prediction but also to perform a preliminary interpretation of the joint connections. In this work, we present a Context-based Interpretable Spatio-Temporal Graph…
▽ More
Human motion prediction is still an open problem extremely important for autonomous driving and safety applications. Due to the complex spatiotemporal relation of motion sequences, this remains a challenging problem not only for movement prediction but also to perform a preliminary interpretation of the joint connections. In this work, we present a Context-based Interpretable Spatio-Temporal Graph Convolutional Network (CIST-GCN), as an efficient 3D human pose forecasting model based on GCNs that encompasses specific layers, aiding model interpretability and providing information that might be useful when analyzing motion distribution and body behavior. Our architecture extracts meaningful information from pose sequences, aggregates displacements and accelerations into the input model, and finally predicts the output displacements. Extensive experiments on Human 3.6M, AMASS, 3DPW, and ExPI datasets demonstrate that CIST-GCN outperforms previous methods in human motion prediction and robustness. Since the idea of enhancing interpretability for motion prediction has its merits, we showcase experiments towards it and provide preliminary evaluations of such insights here. available code: https://github.com/QualityMinds/cistgcn
△ Less
Submitted 21 February, 2024;
originally announced February 2024.
-
Melting of magnetic order in ${\mathrm{NaOsO}}_{3}$ by fs laser pulses
Authors:
Flavio Giorgianni,
Max Burian,
Namrata Gurung,
Martin Kubli,
Vincent Esposito,
Urs Staub,
Paul Beaud,
Steven Lee Johnson,
Yoav William Windsor,
Laurenz Rettig,
Dmitry Ozerov,
Henrik Lemke,
Susmita Saha,
Federico Pressacco,
Stephen Patrick Collins,
Tadashi Togashi,
Tetsuo Katayama,
Shigeki Owada,
Makina Yabashi,
Kazunari Yamaura,
Yoshikazu Tanaka,
Valerio Scagnoli
Abstract:
NaOsO$_3$ has recently attracted significant attention for the strong coupling between its electronic band structure and magnetic ordering. Here, we used time-resolved magnetic X-ray diffraction to determine the timescale of the photoinduced \afm dynamics in NaOsO$_3$. Our measurements are consistent with a sub-100~fs melting of the \afm long-range order, that occurs significantly faster than the…
▽ More
NaOsO$_3$ has recently attracted significant attention for the strong coupling between its electronic band structure and magnetic ordering. Here, we used time-resolved magnetic X-ray diffraction to determine the timescale of the photoinduced \afm dynamics in NaOsO$_3$. Our measurements are consistent with a sub-100~fs melting of the \afm long-range order, that occurs significantly faster than the lattice dynamics as monitored by the transient change in intensity of selected Bragg structural reflections, which instead show a decrease of intensity on a timescale of several ps.
△ Less
Submitted 6 April, 2022;
originally announced April 2022.
-
Adapting Quantum Approximation Optimization Algorithm (QAOA) for Unit Commitment
Authors:
Samantha Koretsky,
Pranav Gokhale,
Jonathan M. Baker,
Joshua Viszlai,
Honghao Zheng,
Niroj Gurung,
Ryan Burg,
Esa Aleksi Paaso,
Amin Khodaei,
Rozhin Eskandarpour,
Frederic T. Chong
Abstract:
In the present Noisy Intermediate-Scale Quantum (NISQ), hybrid algorithms that leverage classical resources to reduce quantum costs are particularly appealing. We formulate and apply such a hybrid quantum-classical algorithm to a power system optimization problem called Unit Commitment, which aims to satisfy a target power load at minimal cost. Our algorithm extends the Quantum Approximation Optim…
▽ More
In the present Noisy Intermediate-Scale Quantum (NISQ), hybrid algorithms that leverage classical resources to reduce quantum costs are particularly appealing. We formulate and apply such a hybrid quantum-classical algorithm to a power system optimization problem called Unit Commitment, which aims to satisfy a target power load at minimal cost. Our algorithm extends the Quantum Approximation Optimization Algorithm (QAOA) with a classical minimizer in order to support mixed binary optimization. Using Qiskit, we simulate results for sample systems to validate the effectiveness of our approach. We also compare to purely classical methods. Our results indicate that classical solvers are effective for our simulated Unit Commitment instances with fewer than 400 power generation units. However, for larger problem instances, the classical solvers either scale exponentially in runtime or must resort to coarse approximations. Potential quantum advantage would require problem instances at this scale, with several hundred units.
△ Less
Submitted 24 October, 2021;
originally announced October 2021.
-
Structurally assisted melting of excitonic correlations in 1T-TiSe2
Authors:
Max Burian,
Michael Porer,
Jose R. L. Mardegan,
Vincent Esposito,
Sergii Parchenko,
Bulat Burganov,
Namrata Gurung,
Mahesh Ramakrishnan,
Valerio Scagnoli,
Hiroki Ueda,
Sonia Francoual,
Federica Fabrizi,
Yoshikazu Tanaka,
Tadashi Togashi,
Yuya Kubota,
Makina Yabashi,
Kai Rossnagel,
Steven L. Johnson,
Urs Staub
Abstract:
The simultaneous condensation of electronic and structural degrees of freedom gives rise to new states of matter, including superconductivity and charge-density-wave formation. When exciting such a condensed system, it is commonly assumed that the ultrafast laser pulse disturbs primarily the electronic order, which in turn destabilizes the atomic structure. Contrary to this conception, we show her…
▽ More
The simultaneous condensation of electronic and structural degrees of freedom gives rise to new states of matter, including superconductivity and charge-density-wave formation. When exciting such a condensed system, it is commonly assumed that the ultrafast laser pulse disturbs primarily the electronic order, which in turn destabilizes the atomic structure. Contrary to this conception, we show here that structural destabilization of few atoms causes melting of the macroscopic ordered charge-density wave in 1T-TiSe2. Using ultrafast pump-probe non-resonant and resonant X-ray diffraction, we observe full suppression of the Se 4p orbital order and the atomic structure at excitation energies more than one order of magnitude below the suggested excitonic binding energy. Complete melting of the charge-density wave occurs 4-5 times faster than expected from a purely electronic charge-screening process, strongly suggesting a structurally assisted breakup of excitonic correlations. Our experimental data clarifies several questions on the intricate coupling between structural and electronic order in stabilizing the charge-density-wave in 1T-TiSe2. The results further show that electron-phonon-coupling can lead to different, energy dependent phase-transition pathways in condensed matter systems, opening new possibilities in the conception of non-equilibrium phenomena at the ultrafast scale.
△ Less
Submitted 24 June, 2020;
originally announced June 2020.
-
Correlation between electronic and structural orders in 1T-TiSe2
Authors:
Hiroki Ueda,
Michael Porer,
José R. L. Mardegan,
Sergii Parchenko,
Namrata Gurung,
Federica Fabrizi,
Mahesh Ramakrishnan,
Larissa Boie,
Martin Josef Neugebauer,
Bulat Burganov,
Max Burian,
Steven Lee Johnson,
Kai Rossnagel,
Urs Staub
Abstract:
The correlation between electronic and crystal structures of 1T-TiSe2 in the charge density wave (CDW) state is studied by x-ray diffraction. Three families of reflections are used to probe atomic displacements and the orbital asymmetry in Se. Two distinct onset temperatures are found, TCDW and a lower T* indicative for an onset of Se out-of-plane atomic displacements. T* coincides with a DC resis…
▽ More
The correlation between electronic and crystal structures of 1T-TiSe2 in the charge density wave (CDW) state is studied by x-ray diffraction. Three families of reflections are used to probe atomic displacements and the orbital asymmetry in Se. Two distinct onset temperatures are found, TCDW and a lower T* indicative for an onset of Se out-of-plane atomic displacements. T* coincides with a DC resistivity maximum and the onset of the proposed gyrotropic electronic structure. However, no indication for chirality is found. The relation between the atomic displacements and the transport properties is discussed in terms of Ti 3d and Se 4p states that only weakly couple to the CDW order.
△ Less
Submitted 16 June, 2020;
originally announced June 2020.
-
Ultrafast electron localization in a correlated metal
Authors:
Jose R. L. Mardegan,
Serhane Zerdane,
Giulia Mancini,
Vincent Esposito,
Jeremy Rouxel,
Roman Mankowsky,
Cristian Svetina,
Namrata Gurung,
Sergii Parchenko,
Michael Porer,
Bulat Burganov,
Yunpei Deng,
Paul Beaud,
Gerhard Ingold,
Bill Pedrini,
Christopher Arrell,
Christian Erny,
Andreas Dax,
Henrik Lemke,
Martin Decker,
Nazaret Ortiz,
Chris Milne,
Grigory Smolentsev,
Laura Maurel,
Steven L. Johnson
, et al. (5 additional authors not shown)
Abstract:
Ultrafast electron delocalization induced by a fs laser pulse is a well-known process and is the initial step for important applications such as fragmentation of molecules or laser ablation in solids. It is well understood that an intense fs laser pulse can remove several electrons from an atom within its pulse duration. [1] However, the speed of electron localization out of an electron gas, the c…
▽ More
Ultrafast electron delocalization induced by a fs laser pulse is a well-known process and is the initial step for important applications such as fragmentation of molecules or laser ablation in solids. It is well understood that an intense fs laser pulse can remove several electrons from an atom within its pulse duration. [1] However, the speed of electron localization out of an electron gas, the capture of an electron by ion, is unknown. Here, we demonstrate that electronic localization out of the conduction band can occur within only a few hundred femtoseconds. This ultrafast electron localization into 4f states has been directly quantified by transient x-ray absorption spectroscopy following photo-excitation of a Eu based correlated metal with a fs laser pulse. Our x-ray experiments show that the driving force for this process is either an ultrafast reduction of the energy of the 4f states, a change of their bandwidth or an increase of the hybridization between the 4f and the 3d states. The observed ultrafast electron localization process raises further basic questions for our understanding of electron correlations and their coupling to the lattice.
△ Less
Submitted 27 February, 2020;
originally announced February 2020.
-
Field-induced double spin spiral in a frustrated chiral magnet
Authors:
M. Ramakrishnan,
E. Constable,
A. Cano,
M. Mostovoy,
J. S. White,
N. Gurung,
E. Schierle,
S. de Brion,
C. V. Colin,
F. Gay,
P. Lejay,
E. Ressouche,
E. Weschke,
V. Scagnoli,
R. Ballou,
V. Simonet,
U. Staub
Abstract:
We report the direct observation of a magnetic-feld induced long-wavelength spin spiral modulation in the chiral compound Ba3TaFe3Si2O14. This new spin texture emerges out of a chiral helical ground state, and is hallmarked by the onset of a unique contribution to the bulk electric polarization, the sign of which depends on the crystal chirality. The periodicity of the feld induced modulation, sev…
▽ More
We report the direct observation of a magnetic-feld induced long-wavelength spin spiral modulation in the chiral compound Ba3TaFe3Si2O14. This new spin texture emerges out of a chiral helical ground state, and is hallmarked by the onset of a unique contribution to the bulk electric polarization, the sign of which depends on the crystal chirality. The periodicity of the feld induced modulation, several hundreds of nm depending on the field value, is comparable to the length scales of mesoscopic topological defects such as skyrmions, merons and solitons. The phase transition and observed threshold behavior are consistent with a phenomenology based on the allowed Lifshitz invariants for the chiral symmetry of langasite, which intriguingly contain all the ingredients for the possible realization of topologically stable antiferromagnetic skyrmions.
△ Less
Submitted 5 October, 2018;
originally announced October 2018.
-
A Unique Crystal Structure of Ca$_2$RuO$_4$ in the Current Stabilized Semi-Metallic State
Authors:
J. Bertinshaw,
N. Gurung,
P. Jorba,
H. Liu,
M. Schmid,
D. T. Mantadakis,
M. Daghofer,
M. Krautloher,
A. Jain,
G. H. Ryu,
O. Fabelo,
P. Hansmann,
G. Khaliullin,
C. Pfleiderer,
B. Keimer,
B. J. Kim
Abstract:
The electric-current stabilized semi-metallic state in the quasi-two-dimensional Mott insulator Ca$_2$RuO$_4$ exhibits an exceptionally strong diamagnetism. Through a comprehensive study using neutron and X-ray diffraction, we show that this non-equilibrium phase assumes a crystal structure distinct from those of equilibrium metallic phases realized in the ruthenates by chemical do**, high press…
▽ More
The electric-current stabilized semi-metallic state in the quasi-two-dimensional Mott insulator Ca$_2$RuO$_4$ exhibits an exceptionally strong diamagnetism. Through a comprehensive study using neutron and X-ray diffraction, we show that this non-equilibrium phase assumes a crystal structure distinct from those of equilibrium metallic phases realized in the ruthenates by chemical do**, high pressure and epitaxial strain, which in turn leads to a distinct electronic band structure. Dynamical mean field theory calculations based on the crystallographically refined atomic coordinates and realistic Coulomb repulsion parameters indicate a semi-metallic state with partially gapped Fermi surface. Our neutron diffraction data show that the non-equilibrium behavior is homogeneous, with antiferromagnetic long-range order completely suppressed. These results provide a new basis for theoretical work on the origin of the unusual non-equilibrium diamagnetism in Ca$_2$RuO$_4$.
△ Less
Submitted 21 May, 2019; v1 submitted 17 June, 2018;
originally announced June 2018.
-
Direct observation of electron density reconstruction at the metal-insulator transition in NaOsO3
Authors:
N. Gurung,
N. Leo,
S. P. Collins,
G. Nisbet,
G. Smolentsev,
M. Garcia-Fernandez,
K. Yamaura,
L. J. Heyderman,
U. Staub,
Y. Joly,
D. D. Khalyavin,
S. W. Lovesey,
V. Scagnoli
Abstract:
5d transition metal oxides offer new opportunities to test our understanding of the interplay of correlation effects and spin-orbit interactions in materials in the absence of a single dominant interaction. The subtle balance between solid-state interactions can result in new mechanisms that minimize the interaction energy, and in material properties of potential use for applications. We focus her…
▽ More
5d transition metal oxides offer new opportunities to test our understanding of the interplay of correlation effects and spin-orbit interactions in materials in the absence of a single dominant interaction. The subtle balance between solid-state interactions can result in new mechanisms that minimize the interaction energy, and in material properties of potential use for applications. We focus here on the 5d transition metal oxide NaOsO3, a strong candidate for the realization of a magnetically driven transition from a metallic to an insulating state exploiting the so-called Slater mechanism. Experimental results are derived from non-resonant and resonant x-ray single crystal diffraction at the Os L-edges. A change in the crystallographic symmetry does not accompany the metal-insulator transition in the Slater mechanism and, indeed, we find no evidence of such a change in NaOsO3. An equally important experimental observation is the emergence of the (300) Bragg peak in the resonant condition with the onset of magnetic order. The intensity of this space-group forbidden Bragg peak continuously increases with decreasing temperature in line with the square of intensity observed for an allowed magnetic Bragg peak. Our main experimental results, the absence of crystal symmetry breaking and the emergence of a space-group forbidden Bragg peak with develo** magnetic order, support the use of the Slater mechanism to interpret the metal-insulator transition in NaOsO3. We successfully describe our experimental results with simulations of the electronic structure and, also, with an atomic model based on the established symmetry of the crystal and magnetic structure.
△ Less
Submitted 22 September, 2018; v1 submitted 22 March, 2018;
originally announced March 2018.
