Avinash Krishna

When Tuning Isn't Enough: Limits of Inference-Time Controls and Collaboration in Small Local LLMs

We study the limits of inference-time parameter tuning and multi-model collaboration for locally-deployed small LLMs (3B-14B) in a small on-premises deployment using quantized models on a 16 GB GPU. Although these techniques are widely used in local, compliance-constrained deployments to improve model quality without r...