Inference-Aware Load Balancing for LLMs (part 6): many models behind one gateway, model routing, LoRA and a rollout split
Two models and a LoRA adapter behind one Gateway: a pool and picker per model, a header that picks the pool and the 404 when it disagrees with the body, a real LoRA adapter and its measured cost, and an 80/20 rollout split.