Skip to main content

Showing 1–19 of 19 results for author: Webster, K

Searching in archive cs. Search in all archives.
.
  1. arXiv:2307.01583  [pdf, other

    cs.LG cs.CV

    Learning Lie Group Symmetry Transformations with Neural Networks

    Authors: Alex Gabel, Victoria Klein, Riccardo Valperga, Jeroen S. W. Lamb, Kevin Webster, Rick Quax, Efstratios Gavves

    Abstract: The problem of detecting and quantifying the presence of symmetries in datasets is useful for model selection, generative modeling, and data analysis, amongst others. While existing methods for hard-coding transformations in neural networks require prior knowledge of the symmetries of the task at hand, this work focuses on discovering and characterizing unknown symmetries present in the dataset, n… ▽ More

    Submitted 4 July, 2023; originally announced July 2023.

    Comments: 9 pages, 5 figures, Proceedings of the 2nd Annual Workshop on Topology, Algebra, and Geometry in Machine Learning (TAG-ML) at the 40th International Conference on Machine Learning, Honolulu, Hawaii, USA. 2023

  2. arXiv:2301.10201  [pdf

    cs.ET

    Safety of self-assembled neuromorphic hardware

    Authors: Can Rager, Kyle Webster

    Abstract: The scalability of modern computing hardware is limited by physical bottlenecks and high energy consumption. These limitations could be addressed by neuromorphic hardware (NMH) which is inspired by the human brain. NMH enables physically built-in capabilities of information processing at the hardware level. In other words, brain-like features bias hardware towards intelligence at scale. Neuropmorp… ▽ More

    Submitted 24 January, 2023; originally announced January 2023.

    Comments: 7 pages, 1 figure

  3. arXiv:2212.08037  [pdf, other

    cs.CL

    Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models

    Authors: Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie Webster

    Abstract: Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of… ▽ More

    Submitted 10 February, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

  4. arXiv:2210.17525  [pdf, ps, other

    cs.CL

    Query Refinement Prompts for Closed-Book Long-Form Question Answering

    Authors: Reinald Kim Amplayo, Kellie Webster, Michael Collins, Dipanjan Das, Shashi Narayan

    Abstract: Large language models (LLMs) have been shown to perform well in answering questions and in producing long-form texts, both in few-shot closed-book settings. While the former can be validated using well-known evaluation metrics, the latter is difficult to evaluate. We resolve the difficulties to evaluate long-form output by doing both tasks at once -- to do question answering that requires long-for… ▽ More

    Submitted 31 October, 2022; originally announced October 2022.

  5. arXiv:2207.08026  [pdf, other

    stat.ML cs.LG

    Rewiring Networks for Graph Neural Network Training Using Discrete Geometry

    Authors: Jakub Bober, Anthea Monod, Emil Saucan, Kevin N. Webster

    Abstract: Information over-squashing is a phenomenon of inefficient information propagation between distant nodes on networks. It is an important problem that is known to significantly impact the training of graph neural networks (GNNs), as the receptive field of a node grows exponentially. To mitigate this problem, a preprocessing procedure known as rewiring is often applied to the input network. In this p… ▽ More

    Submitted 16 July, 2022; originally announced July 2022.

    Comments: 21 pages, 8 figures, 7 tables

  6. arXiv:2206.13757  [pdf, other

    cs.CL cs.CY

    Flexible text generation for counterfactual fairness probing

    Authors: Zee Fryer, Vera Axelrod, Ben Packer, Alex Beutel, Jilin Chen, Kellie Webster

    Abstract: A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the input is changed? Existing counterfactual generation methods typically rely on wordlists or templates, producing simple counterfactuals that don't take into account grammar, context, or subtle sensitive attribute references… ▽ More

    Submitted 28 June, 2022; originally announced June 2022.

  7. arXiv:2112.06905  [pdf, other

    cs.CL

    GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

    Authors: Nan Du, Yan** Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V Le, Yonghui Wu , et al. (2 additional authors not shown)

    Abstract: Scaling language models with more data, compute and parameters has driven significant progress in natural language processing. For example, thanks to scaling, GPT-3 was able to achieve strong results on in-context learning tasks. However, training these large dense models requires significant amounts of computing resources. In this paper, we propose and develop a family of language models named GL… ▽ More

    Submitted 1 August, 2022; v1 submitted 13 December, 2021; originally announced December 2021.

    Comments: Accepted to ICML 2022

  8. arXiv:2104.07571  [pdf, other

    cs.CL

    Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracy

    Authors: Maharshi Gor, Kellie Webster, Jordan Boyd-Graber

    Abstract: The goal of question answering (QA) is to answer any question. However, major QA datasets have skewed distributions over gender, profession, and nationality. Despite that skew, model accuracy analysis reveals little evidence that accuracy is lower for people based on gender or nationality; instead, there is more variation on professions (question topic). But QA's lack of representation could itsel… ▽ More

    Submitted 10 September, 2021; v1 submitted 15 April, 2021; originally announced April 2021.

    Comments: Accepted at EMNLP 2021

  9. arXiv:2104.03026  [pdf, ps, other

    cs.CL

    How to Write a Bias Statement: Recommendations for Submissions to the Workshop on Gender Bias in NLP

    Authors: Christian Hardmeier, Marta R. Costa-jussà, Kellie Webster, Will Radford, Su Lin Blodgett

    Abstract: At the Workshop on Gender Bias in NLP (GeBNLP), we'd like to encourage authors to give explicit consideration to the wider aspects of bias and its social implications. For the 2020 edition of the workshop, we therefore requested that all authors include an explicit bias statement in their work to clarify how their work relates to the social context in which NLP systems are used. The programme co… ▽ More

    Submitted 7 April, 2021; originally announced April 2021.

    Comments: This document was originally published as a blog post on the web site of GeBNLP 2020

  10. arXiv:2102.06788  [pdf, other

    cs.CL

    They, Them, Theirs: Rewriting with Gender-Neutral English

    Authors: Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, Melvin Johnson

    Abstract: Responsible development of technology involves applications being inclusive of the diverse set of users they hope to support. An important part of this is understanding the many ways to refer to a person and being able to fluently change between the different forms as needed. We perform a case study on the singular they, a common way to promote gender inclusion in English. We define a re-writing t… ▽ More

    Submitted 12 February, 2021; originally announced February 2021.

  11. arXiv:2011.03395  [pdf, other

    cs.LG stat.ML

    Underspecification Presents Challenges for Credibility in Modern Machine Learning

    Authors: Alexander D'Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D. Hoffman, Farhad Hormozdiari, Neil Houlsby, Shaobo Hou, Ghassen Jerfel, Alan Karthikesalingam, Mario Lucic, Yian Ma, Cory McLean, Diana Mincu, Akinori Mitani, Andrea Montanari, Zachary Nado, Vivek Natarajan, Christopher Nielson, Thomas F. Osborne , et al. (15 additional authors not shown)

    Abstract: ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predict… ▽ More

    Submitted 24 November, 2020; v1 submitted 6 November, 2020; originally announced November 2020.

    Comments: Updates: Updated statistical analysis in Section 6; Additional citations

  12. arXiv:2010.06032  [pdf, other

    cs.CL

    Measuring and Reducing Gendered Correlations in Pre-trained Models

    Authors: Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, Slav Petrov

    Abstract: Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore such gendered correlations as a case study for how to address unintended correlations in pre-trained models. We define metrics and reveal that it is possible for… ▽ More

    Submitted 2 March, 2021; v1 submitted 12 October, 2020; originally announced October 2020.

  13. arXiv:2009.11982  [pdf, ps, other

    cs.CL cs.LG

    Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias

    Authors: Ana Valeria Gonzalez, Maria Barrett, Rasmus Hvingelby, Kellie Webster, Anders Søgaard

    Abstract: The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are "hallucinatory", e.g., disambiguating gender-ambiguous occurrences of 'doctor' as male doctors. We show that for languages with type B reflexivization, e.g., Swedish and Russian, we can co… ▽ More

    Submitted 28 September, 2020; v1 submitted 24 September, 2020; originally announced September 2020.

    Comments: To appear in EMNLP 2020

  14. arXiv:2006.08881  [pdf, ps, other

    cs.CL

    Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation

    Authors: Kellie Webster, Emily Pitler

    Abstract: Machine translation systems with inadequate document understanding can make errors when translating dropped or neutral pronouns into languages with gendered pronouns (e.g., English). Predicting the underlying gender of these pronouns is difficult since it is not marked textually and must instead be inferred from coreferent mentions in the context. We propose a novel cross-lingual pivoting techniqu… ▽ More

    Submitted 15 June, 2020; originally announced June 2020.

  15. arXiv:2005.00813  [pdf, other

    cs.CL cs.AI cs.LG

    Social Biases in NLP Models as Barriers for Persons with Disabilities

    Authors: Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, Stephen Denuyl

    Abstract: Building equitable and inclusive NLP technologies demands consideration of whether and how social attitudes are represented in ML models. In particular, representations encoded in models often inadvertently perpetuate undesirable social biases from the data on which they are trained. In this paper, we present evidence of such undesirable biases towards mentions of disability in two different Engli… ▽ More

    Submitted 2 May, 2020; originally announced May 2020.

    Comments: ACL 2020 short paper. 5 pages

    Journal ref: ACL 2020

  16. arXiv:2004.14065  [pdf, other

    cs.CL

    Automatically Identifying Gender Issues in Machine Translation using Perturbations

    Authors: Hila Gonen, Kellie Webster

    Abstract: The successful application of neural methods to machine translation has realized huge quality advances for the community. With these improvements, many have noted outstanding challenges, including the modeling and treatment of gendered language. While previous studies have identified issues using synthetic examples, we develop a novel technique to mine examples from real world data to explore chal… ▽ More

    Submitted 14 October, 2020; v1 submitted 29 April, 2020; originally announced April 2020.

    Comments: Findings of EMNLP 2020

  17. arXiv:1810.05201  [pdf, other

    cs.CL

    Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns

    Authors: Kellie Webster, Marta Recasens, Vera Axelrod, Jason Baldridge

    Abstract: Coreference resolution is an important task for natural language understanding, and the resolution of ambiguous pronouns a longstanding challenge. Nonetheless, existing corpora do not capture ambiguous pronouns in sufficient volume or diversity to accurately indicate the practical utility of models. Furthermore, we find gender bias in existing corpora and systems favoring masculine entities. To ad… ▽ More

    Submitted 11 October, 2018; originally announced October 2018.

  18. arXiv:1412.5557  [pdf

    cs.DC

    Standing Together for Reproducibility in Large-Scale Computing: Report on reproducibility@XSEDE

    Authors: Doug James, Nancy Wilkins-Diehr, Victoria Stodden, Dirk Colbry, Carlos Rosales, Mark Fahey, Justin Shi, Rafael F. Silva, Kyo Lee, Ralph Roskies, Laurence Loewe, Susan Lindsey, Rob Kooper, Lorena Barba, David Bailey, Jonathan Borwein, Oscar Corcho, Ewa Deelman, Michael Dietze, Benjamin Gilbert, Jan Harkes, Seth Keele, Praveen Kumar, Jong Lee, Erika Linke , et al. (30 additional authors not shown)

    Abstract: This is the final report on reproducibility@xsede, a one-day workshop held in conjunction with XSEDE14, the annual conference of the Extreme Science and Engineering Discovery Environment (XSEDE). The workshop's discussion-oriented agenda focused on reproducibility in large-scale computational research. Two important themes capture the spirit of the workshop submissions and discussions: (1) organiz… ▽ More

    Submitted 2 January, 2015; v1 submitted 17 December, 2014; originally announced December 2014.

    MSC Class: 68N01 ACM Class: D.2.9

  19. arXiv:1109.2785  [pdf, ps, other

    cs.SC nlin.SI

    Solving large linear algebraic systems in the context of integrable non-abelian Laurent ODEs

    Authors: Thomas Wolf, Eberhard Schruefer, Kenneth Webster

    Abstract: The paper reports on a computer algebra program LSSS (Linear Selective Systems Solver) for solving linear algebraic systems with rational coefficients. The program is especially efficient for very large sparse systems that have a solution in which many variables take the value zero. The program is applied to the symmetry investigation of a non-abelian Laurent ODE introduced recently by M. Kontsevi… ▽ More

    Submitted 13 September, 2011; originally announced September 2011.

    Comments: 15 pages, talk given at AMMCS 2011, submitted for publication in Programming and Computer Software

    MSC Class: 34M55; 37J35; 37K10 ACM Class: I.1.2