On this page
A server sits in a rack with two cables into two different Nexus 9000 leaves, bonded with LACP. The bond only comes up if both switches present the same LACP system ID, so the two leaves have to agree to be one partner. NX-OS gives you three ways to make that agreement. Virtual port channel, vPC, uses a dedicated peer-link and a keepalive. vPC Fabric Peering keeps the vPC model but replaces the physical peer-link with a VXLAN tunnel through the spines. EVPN ESI multihoming drops vPC altogether and lets BGP route types do the coordination. All three forward on both cables. This post builds all three in one VXLAN EVPN fabric, puts a host behind each, and measures what each one does when a link, a leaf's uplinks, the host leg, and the peer-link go away.
Where the designs differ
The lab
Two spines and eight leaves, all Nexus 9300v under containerlab, in four pairs. Pair A is traditional vPC. Pair B is vPC Fabric Peering. Pairs C and D are EVPN ESI multihoming, two segments so the vPC pairs have more than one ESI to resolve and so ESI to ESI traffic is in the mix. One host behind each pair, all four in VLAN 10 mapped to VNI 10010, with the anycast gateway at 192.168.10.254. Every host is an 802.3ad bond running fast LACP with the layer3+4 transmit hash policy. The hosts behind the ESI pairs send VLAN 10 tagged, because the first-generation ESI implementation does not support a native VLAN on the segment trunk.
The underlay is OSPF on point to point /31 links, the overlay is iBGP in AS 65001 with the spines as route reflectors, BUM replication is ingress replication. Pairs A and B and the spines run NX-OS 10.6(1)F, pairs C and D run 9.3(16): on the 9300v, virtual peer-link exists in 10.6(1)F and not in 9.3(16), and the evpn esi multihoming command set exists in 9.3(16) and in no 10.x 9300v release. On hardware the release is set by the platform: fabric peering arrived in the 9.2(x) train for the 9332C, 9364C and 9300-FX, FXP and FX2 switches, the first-generation ESI implementation ran on first-generation 9300s through 9.3(x), and the current one arrived in 10.6(1)F for the 9300-FX2, FX3, GX and later families.
Four failure scenarios, each applied on the vPC primary or the first ESI leaf of the pair and removed 30 seconds later. One uplink shut. Both uplinks shut, so the leaf keeps its host bond and loses the fabric. The host leg, the leaf side of the bond, shut. And the peer-link, which only pair A has. Each drill runs one ping probe of 350 packets at 0.2 second intervals from a host on a different pair to the host behind the pair under test, starting 10 seconds before the failure. Before every drill the script waits until both leaves show the host port-channel bundled and the tunnel source or segment up, and warms the probe's ARP. The whole matrix ran three times on the same lab, and every number below is the range across those passes. I report probes never answered and the longest stretch with nothing arriving; both include the probe's own scheduling and round trip, so they bound the disruption rather than measure convergence exactly.
