Peter Wong

Prediction-Assisted Pricing and Admission for LLM APIs with Stochastic Token Consumption

An LLM application often sells or internally allocates several service products: a small or premium model, a short or long token cap, and possibly multiple posted prices. The operational decision is not merely which model answers a prompt. A price changes purchase probability, a token cap changes both user value and th...