Jun 2026· NTU Journal of Engineering and Technology· Vol 5· 0 citations
TL;DR
A hardware-aware, system-level analysis of key optimization techniques, including pruning, quantization, knowledge distillation, Low-Rank Adaptation (LoRA), and Neural Architecture Search (NAS), shows that quantization consistently achieves the highest inference speedups and memory efficiency.
Abstract
Deep neural networks (DNNs), particularly large language models (LLMs), have achieved remarkable performance; however, they introduce significant computational and memory challenges that hinder practical deployment. This paper presents a hardware-aware, system-level analysis of key optimization techniques, including pruning, quantization, knowledge distillation, Low-Rank Adaptation (LoRA), and Neural Architecture Search (NAS). The study evaluates these methods in terms of scalability, hardware compatibility, and deployment efficiency. The results show that quantization consistently achieves the highest inference speedups and memory efficiency, while LoRA enables scalable fine-tuning with minimal resource overhead. In contrast, pruning provides limited real-world benefits unless supported by structured sparsity and specialized hardware. These findings highlight the importance of integrated optimization and hardware–algorithm co-design for efficient large-scale AI systems.
Experimental results demonstrate that effective compression significantly reduces model size and computational cost with minimal performance loss, highlighting the importance of compression-aware design and concluding as a valuable reference for building efficient and scalable AI systems.
Daniel Rodríguez· International Journal of App...· 0 citations
The proposed hybrid pipeline includes structured pruning, INT8 quantization and task-specific knowledge distillation, which is benchmarked against standalone methods and reinforces the idea of upper bound projection based approach for accuracy-oriented, multi-level compression.
Krishna Kumar Tiwari, Komal Tahiliani, Uma Shankar Birthare et al.· International journal of com...· 0 citations
Findings confirm that combining complementary compression strategies yields substantially better performance-efficiency trade-offs than any single technique applied in isolation.
Upma Sharma Archana· International Journal of Res...· 0 citations
This paper thoroughly analyzes the evolution of parameter optimization strategies, ranging from initial methodologies to modern advancements, elucidating their principles and applications in natural language processing (NLP) and machine learning (ML), paying special emphasis to parameter-efficient fine-tuning approaches.
M. Najafi, Ehsan Tavan, Stefan W. Kuhn et al.· Artificial Intelligence Revi...· 0 citations
The design of transformer-based Large Language Models (LLMs) is being radically changed through new architectures that are able to overcome scalability limitations of previous designs, including Mixture-of-Experts (MoE), Multi-Head Latent Attention (MLA), and Multi-Token Prediction (MTP). As an open-weighted model released at the end of 2024, which has both state of the art architectural transparency and production scale efficiency, DeepSeeek-V3 represents the ultimate testing ground for investigating these modern technologies. This paper provides a comprehensive analysis of the architectural structure of DeepSeek-V3 based upon information from the DeepSeek-V3 Technical Report, industry benchmarking data and independent latency testing, to demonstrate how various techniques can be used to optimize training while still providing competitive performance in code generation and mathematical reasoning. In addition, latency testing conducted on a Distilled version of DeepSeek-V3, with approximately 14 billion parameters, running on a T4 GPU, reveals that although significant improvements have been made in optimizing latency there remains substantial barriers to deploying these models. Through this context, this research will serve as a reference document for practitioners and researchers who wish to understand current trends and challenges in increasing accessibility to high performance AI models.
Yassine Zouhdi, B. Hdioud· EPJ Web of Conferences· 0 citations
Large Language Models (LLMs) have brought transformative advancements to natural language processing. However, their massive computational and memory requirements—stemming from tens of billions of parameters—make deployment on resource-constrained edge devices such as mobile phones and embedded systems highly challenging. Consequently, extensive research efforts have focused on enabling efficient on-device LLM inference. This paper provides an in-depth survey of two major inference acceleration techniques for on-device LLMs: KV cache optimization and speculative decoding. We present the core principles, representative studies, and performance characteristics of each approach. Furthermore, we discuss open challenges and future research directions, aiming to contribute to the advancement of efficient on-device LLM systems.
Mingyu Jo, Sangoh Park· International Conference on...· 0 citations