Inference-Aware Load Balancing for LLMs (part 5): prefix-cache-aware routing, measured
Sending each LLM request to the replica that already holds its prompt: how the llm-d router's approximate prefix index works, what the picker scores, router against Service in two passes, four at a time, and where it stops working.