On this page
There is a network inside a GPU server that moves more data than your entire fabric, and you will never configure a single port of it. This post is about that network, NVLink and NVSwitch, written for the person who owns the other network, the one outside the box. The goal is not to teach you to operate NVLink, nobody operates NVLink, it just works. The goal is to draw the boundary precisely, because knowing exactly where the GPU's private network ends is how you know where your job begins, and how you avoid spending a day debugging a fabric problem that was never on your fabric.
A necessary confession up front, because this series has a rule: I do not own a GPU rack. Every performance number below is NVIDIA's published specification, linked so you can check it against their own datasheets, not something I measured. What I can show you live is the other half, the scale-out fabric, because that half is my leaf-spine, and it is what posts one and two measured.
NVLink is a switched network, not a bus
The mental trap for a network engineer is to file NVLink under "internal bus, like PCIe, not my department." It is more interesting than that. PCIe is a tree with a root complex, and GPU-to-GPU traffic across it has to climb up to a shared parent and back down, contending the whole way, at PCIe 5.0 x16 speeds of about 126 GB/s bidirectional (the PCI-SIG spec figure: sixteen lanes at 32 GT/s). NVLink is a dedicated point-to-point interconnect between GPUs, and on the H100 its fourth generation delivers 900 GB/s per GPU across 18 links, which is about seven times that PCIe figure when you compare bidirectional to bidirectional, on links whose only job is moving tensors between GPUs. It is, in every sense a network person would recognize, a purpose-built transport, just one that lives on a circuit board.
NVSwitch is the crossbar that makes it all-to-all
Point-to-point links alone do not scale: eight GPUs fully meshed would need twenty-eight direct connections, which is a wiring problem you have met before under the name "why we invented switches." NVIDIA's answer is the same as yours. NVSwitch is a crossbar switch for NVLink, and in an eight-GPU server the GPUs connect to a set of NVSwitches rather than to each other, so any GPU reaches any other GPU at full NVLink bandwidth with no shared bottleneck. It is a non-blocking fabric in miniature, the exact concept as your leaf-spine, implemented in silicon inside one chassis.
This is the single most useful thing for a network engineer to internalize: an all-reduce across the eight GPUs in one server completes entirely inside NVSwitch and never touches a NIC, a cable, or a leaf. To your fabric it is invisible. The traffic that reaches you is what happens when a job is too big for one server and has to synchronize across servers, and that crossing is the boundary between two words worth defining precisely.
Scale-up versus scale-out, and why the difference is your job description
Scale-up means making one node bigger: more GPUs, all joined by NVLink into a single high-bandwidth domain where each GPU has that full 900 GB/s NVLink budget to reach the others through NVSwitch. Scale-out means joining many nodes with a network: your network, InfiniBand or RoCE, at 400 to 800 gigabits per NIC on current adapters, which is a real firehose and still several times below what NVLink does inside the box, roughly nine times on a 400G NIC and four to five times on an 800G one, comparing bidirectional bandwidth. The whole art of distributed training is arranging the computation so the heavy, chatty communication stays inside the scale-up domain where bandwidth is nearly free, and only the unavoidable minimum crosses the scale-out fabric where bandwidth is precious and, as post one showed, synchronized.
For you this draws a clean line of responsibility. Inside the NVLink domain, performance is NVIDIA's problem and it is genuinely excellent. The moment a flow crosses a NIC, it is your problem: your hashing, your buffers, your congestion, your barrier. When a training run is slower than it should be, the first diagnostic question is which side of the NIC the bottleneck is on, and knowing the boundary exists is what keeps you from instrumenting the wrong network for an afternoon.
GB200 NVL72 moved the boundary
The reason this matters more every year is that NVIDIA keeps pushing the boundary outward. With GB200 NVL72, the NVLink domain is no longer one server, it is an entire rack of 72 Blackwell GPUs connected by an NVLink switch fabric at 1.8 TB/s per GPU and 130 TB/s across the whole domain. A full rack now behaves, to the software, like one enormous GPU.
Read the consequence for your fabric carefully, because it cuts two ways. More of the intense collective traffic now stays inside the NVLink domain, off your wires entirely, which sounds like your job got easier. But the flows that still must cross between domains are fewer and dramatically larger, because each endpoint is now a 72-GPU rack rather than an 8-GPU box. Your scale-out fabric shrinks in port count and grows in per-flow intensity. That is elephants getting bigger, the exact animal from post two, and every problem that post raised about a hash function meeting an elephant gets sharper as the elephants do. The trend does not retire the scale-out fabric; it concentrates it.
And GB300 keeps the boundary exactly where it is
The follow-on system is the sharpest illustration of this post's whole point. GB300 NVL72, built on Blackwell Ultra, is a large upgrade over GB200 in almost every dimension that is not the network. Each GPU jumps to 288 GB of HBM3e, 1.5 times the memory, the tensor cores add 1.5 times the FP4 compute and twice the attention acceleration, and the rack reaches 1.1 exaFLOPS of dense FP4. And the network boundary? Unmoved. GB300 is still a 72-GPU NVLink domain, still fifth-generation NVLink at 1800 GB/s per GPU, still 130 TB/s across the rack, the same figures as GB200. NVIDIA poured the generational gain into memory and math, not into the interconnect that reaches your fabric.
Sit with what that means for the person who owns the scale-out network. Your fabric's job description did not change from GB200 to GB300, the domain size and the per-GPU bandwidth are identical, but the thing your fabric can stall got considerably more valuable: half again the compute and half again the memory behind every GPU-second that spends idle waiting on the barrier from post one. The fabric spec held still while the cost of getting the fabric wrong went up. That is the quiet pattern in every GPU generation lately, the scale-up domain absorbs the headline improvements, and the scale-out fabric inherits a higher price for every stall it fails to prevent.
What to actually take from this
Three things worth keeping. First, the NIC is the boundary: everything GPU-side of it is a network you benefit from and never touch, everything fabric-side of it is yours, and most GPU-cluster debugging begins with correctly placing a symptom on one side or the other. Second, the design intent you are serving is to keep traffic in the scale-up domain and minimize what crosses to scale-out, which tells you what your fabric is actually for: not the bulk of the communication, but the expensive, synchronized remainder that decides whether the whole job waits. Third, the industry trend of ever-larger NVLink domains is quietly reshaping your fabric into fewer, fatter, more punishing flows, which means the fabric-side techniques the rest of this series covers, adaptive routing, rail optimization, careful congestion control, matter more over time, not less.
That is where this short series lands, because it is where the boundary is clearest. Three posts in, the throughline is one idea seen from three angles: AI traffic is a handful of giant, synchronized flows, so the whole job runs at the speed of its worst path (post one), a static hash was never built to place flows that large and collides them by the birthday problem (post two), and the fastest network of all, NVLink, is the one you never touch, which is precisely why knowing where it ends matters (this post). The techniques that answer the fabric side, rail-optimized topologies, adaptive routing, lossless congestion control, are real and worth their own treatment, but I will write those when I can measure them honestly rather than merely describe them, because a lab that cannot flip its own link to Ethernet without bouncing a production bond has no business pretending to benchmark a rail design. The measured half of this series is the half I could stand behind, and I would rather stop there than pad it.
Sources: NVIDIA NVLink and NVLink Switch, NVIDIA GB200 NVL72, NVIDIA GB300 NVL72, NVIDIA H100.
