On this page
The setting almost nobody checks
Every ConnectX VPI card, the Virtual Protocol Interconnect models rather than the Ethernet-only EN and Lx SKUs, has carried a firmware setting called LINK_TYPE, one per port, with two values: ETH(2) and IB(1). It decides, before any driver loads, whether that port wakes up speaking Ethernet or InfiniBand. Not a driver option, not a kernel parameter: a personality choice burned into firmware config, switchable per port whenever you want.
My lab's two ConnectX-6 cards are running split-brained right now, and I mean that as a compliment:
What actually changes
In ETH mode the port is a normal (very fast) Ethernet NIC: MACs, VLANs, LLDP, bonding, everything. RDMA still works, but as RoCEv2, with the InfiniBand transport headers riding inside UDP port 4791 across your existing switches. In IB mode the port speaks native InfiniBand: LIDs instead of MACs, a subnet manager instead of MAC learning, credit-based flow control instead of PFC tuning.
The part that matters: your software cannot tell the difference. The verbs API, the perftest tools, NCCL, NVMe-oF, all of it runs unmodified on either personality. Here is the same tooling on both, from this lab:
Full honesty about those numbers: the Ethernet run used proper 100G optics, the IB run used a DAC whose EEPROM only admits to SDR, so the comparison says nothing about which protocol is faster. What it proves is better: the application code and the tools didn't change at all. (The cable saga has its own post: The Cable Decides.)
Choosing a personality
ETH plus RoCEv2 when the port plugs into Ethernet switching you already own, which is most homelabs and most enterprises. Native IB when the fabric is dedicated and you want its determinism: managed addressing, credit-based lossless behavior by construction rather than by careful PFC configuration, and partitions as access control. The wrong answer is assuming you're locked into whatever the card shipped with.
Practical notes from flipping these ports more times than I'd like: the change needs mstfwreset or a reboot to take effect, the reset bounces both ports of that card for a few seconds (plan around anything bonded on the sibling port), and your interface names will change since Ethernet ports show up as ethX or enpX while IB ports appear as ibpX with a different driver personality on top of the same mlx5 core.
The nugget: RDMA-capable cards don't make you choose a religion at purchase time. LINK_TYPE is per port, reversible, and takes about a minute; the verbs code you wrote runs happily on either side of it.
