Author

Dev Bali

1 paper indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#machine learning Preprint Sep 2026

Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving

FairInference provides the novelelta-token fairness guarantee: for a well-behaved client, if a token is generated in d time units in isolation, it will be generated within d + {\delta} time units in multi-tenant execution, providing strong latency isolation guarantees for LLM serving.

Dev Bali, Soujanya Ponnapalli, Yi-Chu Wang et al. · 0 citations