Prediction-Assisted Pricing and Admission for LLM APIs with Stochastic Token Consumption
An LLM application often sells or internally allocates several service products: a small or premium model, a short or long token cap, and possibly multiple posted prices. The operational decision is not merely which model answers a prompt. A price changes purchase probability, a token cap changes both user value and th...
Peter Wong
· 0 citations