Skip to content
Review Open access

Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models

Sep 2026 · ACM Computing Surveys · 0 citations · 49 references

TL;DR

This survey aims to systematically address resource efficiency challenges in Large Language Models by reviewing a broad spectrum of techniques designed to enhance the resource efficiency of LLMs, and introduces a nuanced categorization of resource efficiency techniques by their specific resource types.

Abstract

The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated models like OpenAI’s ChatGPT, represents a significant advancement in artificial intelligence. These models, however, bring forth substantial challenges in high consumption of computational, memory, energy, and financial resources, especially in environments with limited resource capabilities. This survey aims to systematically address these challenges by reviewing a broad spectrum of techniques designed to enhance the resource efficiency of LLMs. We categorize methods based on their optimization focus—covering computational, memory, energy, financial, and network resources—and their applicability across various stages of an LLM’s lifecycle, including architecture design, pre-training, fine-tuning, and system design. Additionally, the survey introduces a nuanced categorization of resource efficiency techniques by their specific resource types, which uncovers the intricate relationships and mappings between various resources and corresponding optimization techniques. A standardized set of evaluation metrics and datasets is also presented to facilitate consistent and fair comparisons across different models and techniques. By offering a comprehensive overview of the current state-of-the-art and identifying open research avenues, this survey serves as a foundational reference for researchers and practitioners, aiding them in developing more sustainable and efficient LLMs in a rapidly evolving landscape. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured taxonomy of resource-efficient LLMs but also release a website including a constantly-updated paper list https://github.com/tiingweii-shii/Awesome-Resource-Efficient-LLM-Papers.

Read PDF

Similar papers

#small language model Review Sep 2026

Decoding the Giants: a systematic taxonomy and empirical analysis of large language model architectures, training, and applications

This comprehensive survey provides an in-depth analysis of modern LLMs, with a systematic comparison of cutting-edge proprietary and open-source architectures including DeepSeek, GPT-4, Gemini, LLaMA, and Claude.

Mohamed Nazih Omri, B. Louhichi · 0 citations
Conference Open access Sep 2026

Mapping the Efficiency Landscape of Small Language Models

This work evaluates 70+ SLMs from 2023–2025 on five task-specific benchmarks and compares them with two popular LLMs, revealing key trade-offs between energy, performance, and model selection and highlighting the need for informed, task-aware model selection rather than size-driven choices.

Fabian Reichwald, Lukas Schiesser, Christiane Plociennik et al. · 0 citations
Conference Open access Sep 2026

Graph4LLM: A Systematic Survey of Graph-Enhanced Large Language Models

This survey examines how these methods integrate graphs into various stages of the LLM pipeline, including the input, model, and output phases, and outlines the challenges and future research directions for developing more efficient and interpretable solutions.

Xin-Yan Zhu, Cheng Yang, Qiu-Yue Wang et al. · 0 citations
Book Open access Aug 2026

Discovery, Validation and Editing of Large Language Models Mechanisms: Recent Advances and Future Perspectives

This tutorial provides a comprehensive and up-to-date overview of LLM mechanism discovery, validation, and editing, and surveys mechanistic editing techniques that leverage MI insights to modify behavior at varying granularity.

Yin-Han He, Wendy Zheng, Tianyi Zhao et al. · 0 citations
Review Open access Aug 2026

The Versatility of Large Language Models: A Comprehensive Review and Structured Survey of Architectures, Applications, Challenges, and Future Trajectories

This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods.

P. Peykani, V. Charles, A. Emrouznejad et al. · 0 citations

What Do We Expect from LLMs? Mapping the Design of LLM Benchmarks

This work systematically map 14,767 papers introducing or updating evaluation resources from arXiv submissions between January 2022 and August 2026, using staged screening and automated full-text coding to examine changes in target systems and domains, evaluation materials and conditions, and scoring mechanisms.

Chao Wang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.