Whose turn is it? Context-guided model routing for stateful agents
Model routing can reduce the cost of LLM applications by assigning simpler requests to cheaper models without sacrificing the quality of the response. However, most routers treat requests as independent. Stateful agents do not operate under this assumption: a user turn may require multiple model and tool calls, its dif...
Anikesh Parashar, Avik Dutta, Vashu Chauhan et al.
· 0 citations