-
The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources
Authors:
Shayne Longpre,
Stella Biderman,
Alon Albalak,
Hailey Schoelkopf,
Daniel McDuff,
Sayash Kapoor,
Kevin Klyman,
Kyle Lo,
Gabriel Ilharco,
Nay San,
Maribeth Rauh,
Aviya Skowron,
Bertie Vidgen,
Laura Weidinger,
Arvind Narayanan,
Victor Sanh,
David Adelani,
Percy Liang,
Rishi Bommasani,
Peter Henderson,
Sasha Luccioni,
Yacine Jernite,
Luca Soldaini
Abstract:
Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications. To help shape responsible development practices, we introduce the Foundation Model Development Cheatsheet: a growing collection of 250+ tools and resources spanning text, vision, and speech modalities. We draw on a large body of prior work to survey resources (e.g. software, documentation,…
▽ More
Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications. To help shape responsible development practices, we introduce the Foundation Model Development Cheatsheet: a growing collection of 250+ tools and resources spanning text, vision, and speech modalities. We draw on a large body of prior work to survey resources (e.g. software, documentation, frameworks, guides, and practical tools) that support informed data selection, processing, and understanding, precise and limitation-aware artifact documentation, efficient model training, advance awareness of the environmental impact from training, careful model evaluation of capabilities, risks, and claims, as well as responsible model release, licensing and deployment practices. We hope this curated collection of resources helps guide more responsible development. The process of curating this list, enabled us to review the AI development ecosystem, revealing what tools are critically missing, misused, or over-used in existing practices. We find that (i) tools for data sourcing, model evaluation, and monitoring are critically under-serving ethical and real-world needs, (ii) evaluations for model safety, capabilities, and environmental impact all lack reproducibility and transparency, (iii) text and particularly English-centric analyses continue to dominate over multilingual and multi-modal analyses, and (iv) evaluation of systems, rather than just models, is needed so that capabilities and impact are assessed in context.
△ Less
Submitted 25 June, 2024; v1 submitted 24 June, 2024;
originally announced June 2024.
-
Lessons from the Trenches on Reproducible Evaluation of Language Models
Authors:
Stella Biderman,
Hailey Schoelkopf,
Lintang Sutawika,
Leo Gao,
Jonathan Tow,
Baber Abbasi,
Alham Fikri Aji,
Pawan Sasanka Ammanamanchi,
Sidney Black,
Jordan Clive,
Anthony DiPofi,
Julen Etxaniz,
Benjamin Fattori,
Jessica Zosa Forde,
Charles Foster,
Jeffrey Hsu,
Mimansa Jaiswal,
Wilson Y. Lee,
Haonan Li,
Charles Lovering,
Niklas Muennighoff,
Ellie Pavlick,
Jason Phang,
Aviya Skowron,
Samson Tan
, et al. (5 additional authors not shown)
Abstract:
Effective evaluation of language models remains an open challenge in NLP. Researchers and engineers face methodological issues such as the sensitivity of models to evaluation setup, difficulty of proper comparisons across methods, and the lack of reproducibility and transparency. In this paper we draw on three years of experience in evaluating large language models to provide guidance and lessons…
▽ More
Effective evaluation of language models remains an open challenge in NLP. Researchers and engineers face methodological issues such as the sensitivity of models to evaluation setup, difficulty of proper comparisons across methods, and the lack of reproducibility and transparency. In this paper we draw on three years of experience in evaluating large language models to provide guidance and lessons for researchers. First, we provide an overview of common challenges faced in language model evaluation. Second, we delineate best practices for addressing or lessening the impact of these challenges on research. Third, we present the Language Model Evaluation Harness (lm-eval): an open source library for independent, reproducible, and extensible evaluation of language models that seeks to address these issues. We describe the features of the library as well as case studies in which the library has been used to alleviate these methodological concerns.
△ Less
Submitted 29 May, 2024; v1 submitted 23 May, 2024;
originally announced May 2024.
-
On the Societal Impact of Open Foundation Models
Authors:
Sayash Kapoor,
Rishi Bommasani,
Kevin Klyman,
Shayne Longpre,
Ashwin Ramaswami,
Peter Cihon,
Aspen Hopkins,
Kevin Bankston,
Stella Biderman,
Miranda Bogen,
Rumman Chowdhury,
Alex Engler,
Peter Henderson,
Yacine Jernite,
Seth Lazar,
Stefano Maffulli,
Alondra Nelson,
Joelle Pineau,
Aviya Skowron,
Dawn Song,
Victor Storchan,
Daniel Zhang,
Daniel E. Ho,
Percy Liang,
Arvind Narayanan
Abstract:
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to bo…
▽ More
Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.
△ Less
Submitted 27 February, 2024;
originally announced March 2024.
-
A Safe Harbor for AI Evaluation and Red Teaming
Authors:
Shayne Longpre,
Sayash Kapoor,
Kevin Klyman,
Ashwin Ramaswami,
Rishi Bommasani,
Borhane Blili-Hamelin,
Yangsibo Huang,
Aviya Skowron,
Zheng-Xin Yong,
Suhas Kotha,
Yi Zeng,
Weiyan Shi,
Xianjun Yang,
Reid Southen,
Alexander Robey,
Patrick Chao,
Diyi Yang,
Ruoxi Jia,
Daniel Kang,
Sandy Pentland,
Arvind Narayanan,
Percy Liang,
Peter Henderson
Abstract:
Independent evaluation and red teaming are critical for identifying the risks posed by generative AI systems. However, the terms of service and enforcement strategies used by prominent AI companies to deter model misuse have disincentives on good faith safety evaluations. This causes some researchers to fear that conducting such research or releasing their findings will result in account suspensio…
▽ More
Independent evaluation and red teaming are critical for identifying the risks posed by generative AI systems. However, the terms of service and enforcement strategies used by prominent AI companies to deter model misuse have disincentives on good faith safety evaluations. This causes some researchers to fear that conducting such research or releasing their findings will result in account suspensions or legal reprisal. Although some companies offer researcher access programs, they are an inadequate substitute for independent research access, as they have limited community representation, receive inadequate funding, and lack independence from corporate incentives. We propose that major AI developers commit to providing a legal and technical safe harbor, indemnifying public interest safety research and protecting it from the threat of account suspensions or legal reprisal. These proposals emerged from our collective experience conducting safety, privacy, and trustworthiness research on generative AI systems, where norms and incentives could be better aligned with public interests, without exacerbating model misuse. We believe these commitments are a necessary step towards more inclusive and unimpeded community efforts to tackle the risks of generative AI.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Authors:
Stella Biderman,
Hailey Schoelkopf,
Quentin Anthony,
Herbie Bradley,
Kyle O'Brien,
Eric Hallahan,
Mohammad Aflah Khan,
Shivanshu Purohit,
USVSN Sai Prashanth,
Edward Raff,
Aviya Skowron,
Lintang Sutawika,
Oskar van der Wal
Abstract:
How do large language models (LLMs) develop and evolve over the course of training? How do these patterns change as models scale? To answer these questions, we introduce \textit{Pythia}, a suite of 16 LLMs all trained on public data seen in the exact same order and ranging in size from 70M to 12B parameters. We provide public access to 154 checkpoints for each one of the 16 models, alongside tools…
▽ More
How do large language models (LLMs) develop and evolve over the course of training? How do these patterns change as models scale? To answer these questions, we introduce \textit{Pythia}, a suite of 16 LLMs all trained on public data seen in the exact same order and ranging in size from 70M to 12B parameters. We provide public access to 154 checkpoints for each one of the 16 models, alongside tools to download and reconstruct their exact training dataloaders for further study. We intend \textit{Pythia} to facilitate research in many areas, and we present several case studies including novel results in memorization, term frequency effects on few-shot performance, and reducing gender bias. We demonstrate that this highly controlled setup can be used to yield novel insights toward LLMs and their training dynamics. Trained models, analysis code, training code, and training data can be found at \url{https://github.com/EleutherAI/pythia}.
△ Less
Submitted 31 May, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
