Kubernetes BGP Networking: Cilium VXLAN, FRRouting, and Full-Stack Observability

Cilium VXLAN, FRRouting eBGP, Hubble, and Kubeshark on a 4-node ARM64 cluster. Covers routing limitation, the FRR self-peering problem, and a 52-check validation suite from scratch.
Kubernetes BGP Networking: Cilium VXLAN, FRRouting, and Full-Stack Observability

On this page

This is the third post in a hands-on Kubernetes series. If you are starting from scratch, read Building a Production-Ready Kubernetes Cluster on Ubuntu 24.04 LTS with Cilium CNI first, then Kubernetes CNI Options: A Comprehensive Comparison for context on why Cilium was chosen over the alternatives.

In the previous post we stood up a 4-node cluster and installed Cilium. This time we go deeper: BGP route advertisement using FRRouting, full observability with Hubble, and packet-level traffic inspection with Kubeshark — all running on ARM64 VMs inside Parallels on Apple Silicon.

The lab is fully scripted. Everything in this post corresponds to scripts in the GitHub repository. The goal is not just to get things working, but to understand why each decision was made and what breaks when you get it wrong.

The Lab Environment

Node IP Role Pod CIDR
k8s-control-1 10.211.55.18 Control plane 10.244.0.0/24
k8s-worker-1 10.211.55.19 Worker 10.244.2.0/24
k8s-worker-2 10.211.55.20 Worker 10.244.3.0/24
k8s-worker-3 10.211.55.21 Worker 10.244.1.0/24

All nodes: Ubuntu 24.04.4 LTS, ARM64, kernel 6.8.0-101, containerd 2.2.1, Kubernetes 1.32.13.

This post is for subscribers only

Subscribe to LevelUp I.T. newsletter and stay updated.

Don't miss anything. Get all the latest posts delivered straight to your inbox. It's free!
Great! Check your inbox and click the link to confirm your subscription.
Error! Please enter a valid email address!