Feng Wang

From Cloud to Crowd: Democratizing LLM Service With Decentralized Edge Collaboration for RAG

Results show that DEFRAG narrows the SLM-LLM accuracy gap, while reducing cost by up to 98.4% and increasing peak throughput by up to 97.8% over centralized services, demonstrating the potential of DEFRAG for democratized LLM services at the edge.