Inference-Aware Load Balancing for LLMs (part 4): the endpoint picker, how the llm-d router scores replicas
The llm-d endpoint picker opened up: its scorers and weights, every decision logged, a herd under concurrency traced to a stale metrics snapshot, and two other profiles applied with helm and measured run by run.