Explain the Kubernetes networking model. How do pods on different nodes talk to each other?
Show the answer
Say this first
Every pod gets its own IP, and every pod can reach every other pod on any node without NAT. Each node gets a slice of the pod CIDR, often a /24. The CNI plugin creates the pod's interface, assigns the IP and makes the cross-node path work, either with an overlay (VXLAN or IP-in-IP), with plain routing (BGP or cloud route tables), or by giving pods real VPC addresses like the AWS VPC CNI and Azure CNI do.
Go deeper
Inside a node
Each pod has its own network namespace. The CNI plugin creates a veth pair: one end is eth0 in the
pod, the other sits on the host, attached to a bridge or routed by the host. Containers in one pod share
the namespace and talk over localhost.
Between nodes
| Approach | How | Examples |
|---|---|---|
| Overlay | Encapsulate pod packets between node IPs | Flannel VXLAN, Calico VXLAN or IP-in-IP, Cilium VXLAN or Geneve |
| Routed | Advertise each node's pod CIDR | Calico with BGP, cloud route tables |
| VPC-native | Pods get VPC addresses, the cloud fabric routes them | AWS VPC CNI, Azure CNI, GKE VPC-native (alias IPs) |
Overlay works anywhere but adds encapsulation overhead and lowers the pod MTU. VPC-native avoids the overlay and makes pods first-class in security groups and flow logs, but uses VPC addresses fast.
What CNI is
The Container Network Interface is a spec and a set of plugins. The container runtime calls the plugin with ADD and DEL when a pod sandbox is created or removed. The plugin handles interface setup and IPAM. Network policy enforcement is a separate capability that some CNIs provide (Calico, Cilium) and others do not (Flannel on its own).
Three ranges to plan
Node addresses, the pod CIDR and the service CIDR. None of them may overlap with each other or with networks pods must reach.
They will ask next
A pod gets stuck in ContainerCreating with a CNI error. What do you check?
kubectl describe podevents for the CNI message, the CNI daemonset pods on that node, and IP exhaustion: on the AWS VPC CNI the subnet or the node's ENI limit may have no addresses left.Why might large packets fail between pods on an overlay?
VXLAN adds 50 bytes. If the pod MTU is not reduced to fit the node MTU, large packets are dropped or fragmented. Most CNIs set the pod MTU automatically; custom setups often get it wrong.
Traps
- Saying pods talk through NAT by default. The model requires no NAT between pods.
- Forgetting that the pod CIDR must not overlap with on-prem networks the pods call.