2023· International Journal of Applied Data Science & Modern Computing· 0 citations
TL;DR
Experimental results demonstrate that effective compression significantly reduces model size and computational cost with minimal performance loss, highlighting the importance of compression-aware design and concluding as a valuable reference for building efficient and scalable AI systems.
Abstract
Deep neural networks (DNNs) have achieved outstanding performance in areas such as computer vision, speech recognition, natural language processing, and autonomous systems. However, their high computational cost, memory usage, and energy consumption limit deployment in resource-constrained environments like mobile and edge devices. Model compression has emerged as a crucial solution to improve efficiency while maintaining accuracy. This paper provides a comprehensive study of neural network compression techniques, including pruning, quantization, low-rank factorization, knowledge distillation, and neural architecture optimization. These methods are analyzed based on compression ratio, latency, memory efficiency, and accuracy trade-offs. The study also explores hybrid compression approaches and proposes a systematic workflow from model training to deployment on constrained hardware. Experimental results demonstrate that effective compression significantly reduces model size and computational cost with minimal performance loss. The paper highlights the importance of compression-aware design and concludes as a valuable reference for building efficient and scalable AI systems.
Findings confirm that combining complementary compression strategies yields substantially better performance-efficiency trade-offs than any single technique applied in isolation.
Upma Sharma Archana· International Journal of Res...· 0 citations
This paper proposes a novel compression method that uses Singular Value Decomposition (SVD) informed by data examples that achieves substantial reductions in model size and computation with minimal impact on accuracy and demonstrates its applicability in compressing deep neural networks to a fraction of their original size.
Abdelfattah Toulaoui, H. Khalfi, I. Hafidi· Journal of Artificial Intell...· 0 citations
The proposed hybrid pipeline includes structured pruning, INT8 quantization and task-specific knowledge distillation, which is benchmarked against standalone methods and reinforces the idea of upper bound projection based approach for accuracy-oriented, multi-level compression.
Krishna Kumar Tiwari, Komal Tahiliani, Uma Shankar Birthare et al.· International journal of com...· 0 citations
The problem of deep neural network compression is discussed using Convolutional Neural Networks (CNNs) as an example. The size of deep neural networks is an obstacle to their practical application under conditions of limited computing resources, energy, and inference latency requirements. One of the developing approaches to compressing deep neural network models is thinning-removing some of the parameters or structural elements of the neural network model. It is shown that thinning is a tradeoff between classification accuracy and computational efficiency. The problem of compressing CNN models was formulated by thinning parameters while maintaining classification quality at the level of the original model, reducing the number of parameters, computational complexity, and inference latency. Standard Top-1 and Top-5 classification accuracy metrics were used to evaluate classification quality. The degree of compression and inference latency were assessed using metrics, such as the number of model parameters, model size, computational complexity, and inference latency. To ensure a fair comparison of the proposed method with others, inference latency measurements were conducted under identical conditions with a fixed input data size. The concept of representation geometry is introduced to control the stability of the internal feature space. The change in the interclass similarity matrix, calculated from the class centroids in the feature space, is used as a metric for preserving the structure of the feature space. The main result of this work is the development of a new compression method for CNNs based on geometrically controlled thinning of the CNN model. In this method, thinning is performed greedily across CNN blocks, and the admissibility of each step is determined not only by the local importance of channels but also by a global constraint on changing the geometry of the representations. The results of the experiment demonstrate that the proposed method consistently maintains classification quality while reducing computational complexity and the number of parameters compared to the baseline method without thinning and the depth-based thinning method of the CNN model. The proposed method for geometrically controlled thinning of a CNN model can be applied to problems requiring real-time decision making as well as on mobile and embedded devices.
Татьяна Михайловна, Татарникова1, Анастасия Сергеевна Раскопина et al.· Scientific and Technical Jou...· 0 citations
Extreme compression of deep neural networks, up to full binarization, dramatically reduces memory footprint and arithmetic complexity, facilitating deployment on constrained edge hardware with field-programmable gate arrays (FPGAs) and microcontrollers. Although combining binarization with pruning promises additional efficiency gains, existing pruning strategies are ill-suited to binarized representations and rarely translate into meaningful hardware savings. We introduce a PyTorch-based, research-oriented framework that incorporates freezing and pruning mechanisms for designing and optimizing binarized neural networks. The framework enables rapid and reproducible evaluation of state-of-the-art approaches and the fast prototyping of new ones. Leveraging this framework, we propose a novel pruning method that accounts for the relative importance of learned parameters across abstraction levels. Such a global weighting mechanism consistently achieves a superior trade-off between model accuracy and pruning rate, achieving a 70% pruning rate on VGG11 with constant accuracy, while state-of-the-art results reach only 41% in the binarized setting.
The rapid development of programmable network devices and the widespread adoption of machine learning (ML) in networking have facilitated efficient research into intelligent data planes (IDPs). Offloading ML to programmable data planes (PDPs) enables quick analysis and responses to network traffic dynamics, and efficient management of network links. Compared to using an external low-cost board with sufficient memory and a general-purpose CPU, IDP deployment keeps inference inside the switch forwarding pipeline, avoiding inter-device transfer and coordination overhead. This enables line-rate processing and faster response for real-time network control. However, the hardware pipeline presents significant resource limitations. For instance, Intel Tofino ASIC has only 10Mb SRAM in each stage, and lacks support for multiplication, division, and floating-point operations. These constraints significantly hinder the development of IDP. This paper presents Quark, a framework that automatically compresses the convolutional neural network (CNN) and fully offloads quantized inference onto PDP. Quark employs model pruning to simplify the CNN model, uses quantization to support floating-point operations, and utilizes neural architecture search to balance accuracy and PDP resource constraints. Additionally, Quark divides the CNN into smaller units to improve resource utilization on the PDP. We have implemented a testbed prototype of Quark on both P4 hardware switch (Intel Tofino ASIC) and software switch (i.e., BMv2). Extensive evaluation results on the ISCX Botnet dataset demonstrate that Quark achieves 97.3% accuracy while using only 24.27% of the SRAM resources on the Intel Tofino ASIC switch, completing inference tasks at line rate with an average latency of $42.66\mu s$ .
Xiaoquan Zhang, Mai Zhang, Lin Cui et al.· IEEE Transactions on Network...· 0 citations