-
IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce
Authors:
Wenxuan Ding,
Weiqi Wang,
Sze Heng Douglas Kwok,
Minghao Liu,
Tianqing Fang,
Jiaxin Bai,
Junxian He,
Yangqiu Song
Abstract:
Enhancing Language Models' (LMs) ability to understand purchase intentions in E-commerce scenarios is crucial for their effective assistance in various downstream tasks. However, previous approaches that distill intentions from LMs often fail to generate meaningful and human-centric intentions applicable in real-world E-commerce contexts. This raises concerns about the true comprehension and utili…
▽ More
Enhancing Language Models' (LMs) ability to understand purchase intentions in E-commerce scenarios is crucial for their effective assistance in various downstream tasks. However, previous approaches that distill intentions from LMs often fail to generate meaningful and human-centric intentions applicable in real-world E-commerce contexts. This raises concerns about the true comprehension and utilization of purchase intentions by LMs. In this paper, we present IntentionQA, a double-task multiple-choice question answering benchmark to evaluate LMs' comprehension of purchase intentions in E-commerce. Specifically, LMs are tasked to infer intentions based on purchased products and utilize them to predict additional purchases. IntentionQA consists of 4,360 carefully curated problems across three difficulty levels, constructed using an automated pipeline to ensure scalability on large E-commerce platforms. Human evaluations demonstrate the high quality and low false-negative rate of our benchmark. Extensive experiments across 19 language models show that they still struggle with certain scenarios, such as understanding products and intentions accurately, jointly reasoning with products and intentions, and more, in which they fall far behind human performances. Our code and data are publicly available at https://github.com/HKUST-KnowComp/IntentionQA.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
Simultaneous linear connectivity of neural networks modulo permutation
Authors:
Ekansh Sharma,
Devin Kwok,
Tom Denton,
Daniel M. Roy,
David Rolnick,
Gintare Karolina Dziugaite
Abstract:
Neural networks typically exhibit permutation symmetries which contribute to the non-convexity of the networks' loss landscapes, since linearly interpolating between two permuted versions of a trained network tends to encounter a high loss barrier. Recent work has argued that permutation symmetries are the only sources of non-convexity, meaning there are essentially no such barriers between traine…
▽ More
Neural networks typically exhibit permutation symmetries which contribute to the non-convexity of the networks' loss landscapes, since linearly interpolating between two permuted versions of a trained network tends to encounter a high loss barrier. Recent work has argued that permutation symmetries are the only sources of non-convexity, meaning there are essentially no such barriers between trained networks if they are permuted appropriately. In this work, we refine these arguments into three distinct claims of increasing strength. We show that existing evidence only supports "weak linear connectivity"-that for each pair of networks belonging to a set of SGD solutions, there exist (multiple) permutations that linearly connect it with the other networks. In contrast, the claim "strong linear connectivity"-that for each network, there exists one permutation that simultaneously connects it with the other networks-is both intuitively and practically more desirable. This stronger claim would imply that the loss landscape is convex after accounting for permutation, and enable linear interpolation between three or more independently trained models without increased loss. In this work, we introduce an intermediate claim-that for certain sequences of networks, there exists one permutation that simultaneously aligns matching pairs of networks from these sequences. Specifically, we discover that a single permutation aligns sequences of iteratively trained as well as iteratively pruned networks, meaning that two networks exhibit low loss barriers at each step of their optimization and sparsification trajectories respectively. Finally, we provide the first evidence that strong linear connectivity may be possible under certain conditions, by showing that barriers decrease with increasing network width when interpolating among three networks.
△ Less
Submitted 9 April, 2024;
originally announced April 2024.
-
Dataset Difficulty and the Role of Inductive Bias
Authors:
Devin Kwok,
Nikhil Anand,
Jonathan Frankle,
Gintare Karolina Dziugaite,
David Rolnick
Abstract:
Motivated by the goals of dataset pruning and defect identification, a growing body of methods have been developed to score individual examples within a dataset. These methods, which we call "example difficulty scores", are typically used to rank or categorize examples, but the consistency of rankings between different training runs, scoring methods, and model architectures is generally unknown. T…
▽ More
Motivated by the goals of dataset pruning and defect identification, a growing body of methods have been developed to score individual examples within a dataset. These methods, which we call "example difficulty scores", are typically used to rank or categorize examples, but the consistency of rankings between different training runs, scoring methods, and model architectures is generally unknown. To determine how example rankings vary due to these random and controlled effects, we systematically compare different formulations of scores over a range of runs and model architectures. We find that scores largely share the following traits: they are noisy over individual runs of a model, strongly correlated with a single notion of difficulty, and reveal examples that range from being highly sensitive to insensitive to the inductive biases of certain model architectures. Drawing from statistical genetics, we develop a simple method for fingerprinting model architectures using a few sensitive examples. These findings guide practitioners in maximizing the consistency of their scores (e.g. by choosing appropriate scoring methods, number of runs, and subsets of examples), and establishes comprehensive baselines for evaluating scores in the future.
△ Less
Submitted 3 January, 2024;
originally announced January 2024.
-
Neural Networks as Paths through the Space of Representations
Authors:
Richard D. Lange,
Devin Kwok,
Jordan Matelsky,
Xinyue Wang,
David S. Rolnick,
Konrad P. Kording
Abstract:
Deep neural networks implement a sequence of layer-by-layer operations that are each relatively easy to understand, but the resulting overall computation is generally difficult to understand. We consider a simple hypothesis for interpreting the layer-by-layer construction of useful representations: perhaps the role of each layer is to reformat information to reduce the "distance" to the desired ou…
▽ More
Deep neural networks implement a sequence of layer-by-layer operations that are each relatively easy to understand, but the resulting overall computation is generally difficult to understand. We consider a simple hypothesis for interpreting the layer-by-layer construction of useful representations: perhaps the role of each layer is to reformat information to reduce the "distance" to the desired outputs. With this framework, the layer-wise computation implemented by a deep neural network can be viewed as a path through a high-dimensional representation space. We formalize this intuitive idea of a "path" by leveraging recent advances in *metric* representational similarity. We extend existing representational distance methods by computing geodesics, angles, and projections of representations, going beyond mere layer distances. We then demonstrate these tools by visualizing and comparing the paths taken by ResNet and VGG architectures on CIFAR-10. We conclude by sketching additional ways that this kind of representational geometry can be used to understand and interpret network training, and to describe novel kinds of similarities between different models.
△ Less
Submitted 27 November, 2022; v1 submitted 22 June, 2022;
originally announced June 2022.
-
Can a Tesla Turbine be Utilised as a Non-Magnetic Actuator for MRI-Guided Robotic Interventions?
Authors:
David Navarro-Alarcon,
Luiza Labazanova,
Man Kiu Chow,
Kwun Wang Ng,
Derek Kwok
Abstract:
This paper introduces a new type of nonmagnetic actuator for MRI interventions. Ultrasonic and piezoelectric motors are one the most commonly used actuators in MRI applications. However, most of these actuators are only MRI-safe, which means they cannot be operated while imaging as they cause significant visual artifacts. To cope with this issue, we developed a new pneumatic rotary servo-motor (ba…
▽ More
This paper introduces a new type of nonmagnetic actuator for MRI interventions. Ultrasonic and piezoelectric motors are one the most commonly used actuators in MRI applications. However, most of these actuators are only MRI-safe, which means they cannot be operated while imaging as they cause significant visual artifacts. To cope with this issue, we developed a new pneumatic rotary servo-motor (based on the Tesla turbine) that can be effectively used during continuous MR imaging. We thoroughly tested the performance and magnetic properties of our MRI-compatible actuator with several experiments, both inside and outside an MRI scanner. The reported results confirm the feasibility to use this motor for MRI-guided robotic interventions.
△ Less
Submitted 19 August, 2021;
originally announced August 2021.
-
Quotients of complex algebraic supergroups
Authors:
R. Fioresi,
S. D. Kwok,
D. W. Taylor
Abstract:
In this paper we prove that the etale sheafification of the functor arising from the quotient of an algebraic supergroup by a closed subsupergroup is representable by a smooth superscheme.
In this paper we prove that the etale sheafification of the functor arising from the quotient of an algebraic supergroup by a closed subsupergroup is representable by a smooth superscheme.
△ Less
Submitted 6 April, 2020; v1 submitted 13 November, 2019;
originally announced November 2019.
-
SUSY $N$-supergroups and their real forms
Authors:
R. Fioresi,
S. D. Kwok
Abstract:
We study SUSY $N$-supergroups, $N=1,2$, their classification and explicit realization, together with their real forms. In the end, we give the supergroup of SUSY preserving automorphism of $\mathbf{C}^{1|1}$ and we identify it with a subsupergroup of the SUSY preserving automorphisms of $\mathbf{P}^{1|1}$.
We study SUSY $N$-supergroups, $N=1,2$, their classification and explicit realization, together with their real forms. In the end, we give the supergroup of SUSY preserving automorphism of $\mathbf{C}^{1|1}$ and we identify it with a subsupergroup of the SUSY preserving automorphisms of $\mathbf{P}^{1|1}$.
△ Less
Submitted 9 December, 2017; v1 submitted 10 December, 2016;
originally announced December 2016.
-
Toward the intrinsic limit of topological insulator Bi2Se3
Authors:
Jixia Dai,
Damien West,
Xueyun Wang,
Yazong Wang,
Daniel Kwok,
S-W. Cheong,
S. B. Zhang,
Weida Wu
Abstract:
Combining high resolution scanning tunneling microscopy and first principle calculations, we identified the major native defects, in particular the Se vacancies and Se interstitial defects that are responsible for the bulk conduction and nanoscale potential fluctuation in single crystals of archetypal topological insulator Bi2Se3. Here it is established that the defect concentrations in Bi2Se3 are…
▽ More
Combining high resolution scanning tunneling microscopy and first principle calculations, we identified the major native defects, in particular the Se vacancies and Se interstitial defects that are responsible for the bulk conduction and nanoscale potential fluctuation in single crystals of archetypal topological insulator Bi2Se3. Here it is established that the defect concentrations in Bi2Se3 are far above the thermodynamic limit, and that the growth kinetics dominate the observed defect concentrations. Furthermore, through careful control of the synthesis, our tunneling spectroscopy suggests that our best samples are approaching the intrinsic limit with the Fermi level inside the band gap without introducing extrinsic dopants.
△ Less
Submitted 16 June, 2016;
originally announced June 2016.
-
The Peter-Weyl Theorem for SU(1|1)
Authors:
C. Carmeli,
R. Fioresi,
S. D. Kwok
Abstract:
We study a generalization of the results \in \cite{cfk} to the case of $SU(1|1)$ interpreted as the supercircle $S^{1|2}$. We describe all of its finite dimensional complex irreducible representations, we give a reducibility result for representations not containing the trivial character, and we compute explicitly the corresponding matrix elements. In the end we give the Peter-Weyl theorem for…
▽ More
We study a generalization of the results \in \cite{cfk} to the case of $SU(1|1)$ interpreted as the supercircle $S^{1|2}$. We describe all of its finite dimensional complex irreducible representations, we give a reducibility result for representations not containing the trivial character, and we compute explicitly the corresponding matrix elements. In the end we give the Peter-Weyl theorem for $S^{1|2}$.
△ Less
Submitted 25 September, 2015;
originally announced September 2015.
-
The Projective Linear Supergroup and the SUSY-preserving automorphisms of ${\mathbf P}^{1|1}$
Authors:
R. Fioresi,
S. D. Kwok
Abstract:
The purpose of this paper is to describe the projective linear supergroup, its relation with the automorphisms of the projective superspace and to determine the supergroup of SUSY preserving automorphisms of ${\mathbf P}^{1|1}$
The purpose of this paper is to describe the projective linear supergroup, its relation with the automorphisms of the projective superspace and to determine the supergroup of SUSY preserving automorphisms of ${\mathbf P}^{1|1}$
△ Less
Submitted 9 December, 2017; v1 submitted 17 April, 2015;
originally announced April 2015.
-
SUSY structures, representations and Peter-Weyl theorem for $S^{1|1}$
Authors:
C. Carmeli,
R. Fioresi,
S. D. Kwok
Abstract:
The real compact supergroup $S^{1|1}$ is analized from different perspectives and its representation theory is studied. We prove it is the only (up to isomorphism) supergroup, which is a real form of $({\mathbf C}^{1|1})^\times$ with reduced Lie group $S^1$, and a link with SUSY structures on ${\mathbf C}^{1|1}$ is established. We describe a large family of complex semisimple representations of…
▽ More
The real compact supergroup $S^{1|1}$ is analized from different perspectives and its representation theory is studied. We prove it is the only (up to isomorphism) supergroup, which is a real form of $({\mathbf C}^{1|1})^\times$ with reduced Lie group $S^1$, and a link with SUSY structures on ${\mathbf C}^{1|1}$ is established. We describe a large family of complex semisimple representations of $S^{1|1}$ and we show that any $S^{1|1}$-representation whose weights are all nonzero is a direct sum of members of our family. We also compute the matrix elements of the members of this family and we give a proof of the Peter-Weyl theorem for $S^{1|1}$.
△ Less
Submitted 10 July, 2014;
originally announced July 2014.
-
On SUSY curves
Authors:
R. Fioresi,
S. D. Kwok
Abstract:
In this note we give a summary of some elementary results in the theory of super Riemann surfaces (SUSY curves).
In this note we give a summary of some elementary results in the theory of super Riemann surfaces (SUSY curves).
△ Less
Submitted 10 June, 2013;
originally announced June 2013.
-
Proximity-induced high-temperature superconductivity in topological insulators Bi2Se3 and Bi2Te3
Authors:
Parisa Zareapour,
Alex Hayat,
Shu Yang F. Zhao,
Michael Kreshchuk,
Achint Jain,
Daniel C. Kwok,
Nara Lee,
Sang-Wook Cheong,
Zhijun Xu,
Alina Yang,
G. D. Gu,
Shuang Jia,
Robert J. Cava,
Kenneth S. Burch
Abstract:
Interest in the superconducting proximity effect has been reinvigorated recently by novel optoelectronic applications as well as by the possible emergence of the elusive Majorana fermion at the interface between topological insulators and superconductors. Here we produce high-temperature superconductivity in Bi2Se3 and Bi2Te3 via proximity to Bi2Sr2CaCu2O8+δ, in order to access increasing temperat…
▽ More
Interest in the superconducting proximity effect has been reinvigorated recently by novel optoelectronic applications as well as by the possible emergence of the elusive Majorana fermion at the interface between topological insulators and superconductors. Here we produce high-temperature superconductivity in Bi2Se3 and Bi2Te3 via proximity to Bi2Sr2CaCu2O8+δ, in order to access increasing temperature and energy scales for this phenomenon. This was achieved by a new mechanical bonding technique we developed, enabling the fabrication of high-quality junctions between materials, unobtainable by conventional approaches. We observe proximity-induced superconductivity in Bi2Se3 and Bi2Te3 persisting up to at least 80K, a temperature an order of magnitude higher than any previous observations. Moreover, the induced superconducting gap in our devices reaches values of 10mV, significantly enhancing the relevant energy scales. Our results open new directions for fundamental studies in condensed matter physics and enable a wide range of applications in spintronics and quantum computing.
△ Less
Submitted 1 November, 2012;
originally announced November 2012.
-
Fabrication and Characterization of Topological Insulator Bi$_2$Se$_3$ Nanocrystals
Authors:
S. Y. F. Zhao,
C. Beekman,
L. J. Sandilands,
J. E. J. Bashucky,
D. Kwok,
N. Lee,
A. D. LaForge,
S. W. Cheong,
K. S. Burch
Abstract:
In the recently discovered class of materials known as topological insulators, the presence of strong spin-orbit coupling causes certain topological invariants in the bulk to differ from their values in vacuum. The sudden change of invariants at the interface results in metallic, time reversal invariant surface states whose properties are useful for applications in spintronics and quantum computat…
▽ More
In the recently discovered class of materials known as topological insulators, the presence of strong spin-orbit coupling causes certain topological invariants in the bulk to differ from their values in vacuum. The sudden change of invariants at the interface results in metallic, time reversal invariant surface states whose properties are useful for applications in spintronics and quantum computation. However, a key challenge is to fabricate these materials on the nanoscale appropriate for devices and probing the surface. To this end we have produced 2 nm thick nanocrystals of the topological insulator Bi$_2$Se$_3$ via mechanical exfoliation. For crystals thinner than 10 nm we observe the emergence of an additional mode in the Raman spectrum. The emergent mode intensity together with the other results presented here provide a recipe for production and thickness characterization of Bi$_2$Se$_3$ nanocrystals.
△ Less
Submitted 8 April, 2011; v1 submitted 2 August, 2010;
originally announced August 2010.
-
X-ray absorption spectroscopy measurement on the LaO1-xFxFeAs system
Authors:
A. Ignatov,
C. L. Zhang,
M. Vannucci,
M. Croft,
T. A. Tyson,
D. Kwok,
Z. Qin,
S. -W. Cheong
Abstract:
Results of Fe K-, As K-, and La L3-edge x-ray absorption near edge structure (XANES) measurements on LaO1-xFxFeAs compounds are presented. The Fe K- edge exhibits a chemical shift to lower energy, near edge feature modifications, and pre-edge feature suppression as a result of F substitution for O. The former two changes provide evidence of electron charge transfer to the Fe sites and the latter…
▽ More
Results of Fe K-, As K-, and La L3-edge x-ray absorption near edge structure (XANES) measurements on LaO1-xFxFeAs compounds are presented. The Fe K- edge exhibits a chemical shift to lower energy, near edge feature modifications, and pre-edge feature suppression as a result of F substitution for O. The former two changes provide evidence of electron charge transfer to the Fe sites and the latter directly supports the delivery of this charge into the Fe-3d orbitals. The As K- edge measurements show spectral structures typical of compounds with planes of transition-metal tetrahedrally coordinated to p-block elements as is illustrated by comparison to other such materials. The insensitivity of the As-K edge to do**, along with the strong Fe-K do** response, is consistent with band structure calculations showing essentially pure Fe-d character near the Fermi energy in these materials. The energy of the continuum resonance feature above the La-L3 edge is shown to be quantitatively consistent with the reported La-O inter-atomic separation and with other oxide compounds containing rare earth elements.
△ Less
Submitted 10 November, 2008; v1 submitted 15 August, 2008;
originally announced August 2008.
