Network Engineering

Part 7 of the Agentic NOC series. How you earn the right to trust a system that watches and sometimes changes your network: 131 offline tests against real captured output, mutated-data fault detection, a live 30-fault campaign, and the worst bug of all, a monitor that mistakes silence for success.

Part 5 of the Agentic NOC series. Three read-only MCP servers over a live fabric, firewall pair, and cluster; three A2A specialists and a supervisor; a local-first planner; one guarded write path; an autonomous monitor. The one design law, and how to rebuild it.

One HTTPS request followed from an external client to a pod, with the show commands and packet captures from every device it touches: BGP unnumbered underlay, type-2, type-3 and type-5 EVPN routes, symmetric routing proven on the wire, a Palo Alto A/A pair and Cilium.

At a few thousand GPUs, the backend network becomes a probability problem: something is always broken and the hash is always unlucky. The fix is two identical fabrics with zero cross-links. A deep dive into dual-plane design, sized for B300 nodes with 800G per GPU.

The finale: one request traced live through all eight hops the series built, the five traps ranked by hours lost, and what I would do differently next time. Plus the honest single point of failure the design still carries.

Every chip in the modern data center exists because one bottleneck got expensive enough to deserve its own silicon. The whole zoo in one sitting: real photos, one table, and a memory trick that makes the differences stick.

One helm install, three jobs: the pod network, kube-proxy's whole duty in eBPF, and BGP from every node to its leaf pair's anycast gateway. Plus the consent chain that keeps VIP announcements deliberate, and the anycast peering quirk worth knowing.