Computer Science > Computation and Language
[Submitted on 12 Jul 2023 (this version), latest version 9 Apr 2024 (v9)]
Title:A Comprehensive Overview of Large Language Models
View PDFAbstract:Large Language Models (LLMs) have shown excellent generalization capabilities that have led to the development of numerous models. These models propose various new architectures, tweaking existing architectures with refined training strategies, increasing context length, using high-quality training data, and increasing training time to outperform baselines. Analyzing new developments is crucial for identifying changes that enhance training stability and improve generalization in LLMs. This survey paper comprehensively analyses the LLMs architectures and their categorization, training strategies, training datasets, and performance evaluations and discusses future research directions. Moreover, the paper also discusses the basic building blocks and concepts behind LLMs, followed by a complete overview of LLMs, including their important features and functions. Finally, the paper summarizes significant findings from LLM research and consolidates essential architectural and training strategies for develo** advanced LLMs. Given the continuous advancements in LLMs, we intend to regularly update this paper by incorporating new sections and featuring the latest LLM models.
Submission history
From: Humza Naveed [view email][v1] Wed, 12 Jul 2023 20:01:52 UTC (542 KB)
[v2] Fri, 18 Aug 2023 13:53:06 UTC (734 KB)
[v3] Wed, 13 Sep 2023 12:13:45 UTC (1,354 KB)
[v4] Thu, 5 Oct 2023 10:29:02 UTC (1,369 KB)
[v5] Thu, 2 Nov 2023 07:59:50 UTC (1,438 KB)
[v6] Thu, 23 Nov 2023 19:23:19 UTC (1,440 KB)
[v7] Wed, 27 Dec 2023 10:15:51 UTC (1,481 KB)
[v8] Tue, 20 Feb 2024 07:19:41 UTC (1,533 KB)
[v9] Tue, 9 Apr 2024 21:38:33 UTC (1,691 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.