How does a backend learn the real client IP behind cloud load balancers?
Show the answer
Say this first
It depends on whether the load balancer proxies or passes packets through. An ALB, Application Gateway or GCP proxy load balancer adds the client IP to X-Forwarded-For. An NLB preserves the source IP for instance targets by default, but not for IP targets with TCP or TLS, where you enable client IP preservation or proxy protocol v2. Azure Load Balancer and GCP passthrough load balancers do not rewrite the source at all.
Go deeper
By load balancer
| Load balancer | How the client IP arrives |
|---|---|
| AWS ALB | X-Forwarded-For, plus X-Forwarded-Proto and X-Forwarded-Port |
| AWS NLB, instance targets | Source IP preserved by default |
| AWS NLB, IP targets, TCP or TLS | Not preserved by default; turn on preservation or proxy protocol v2 |
| AWS NLB, UDP | Preserved |
| Azure Load Balancer | Passthrough, source preserved |
| Azure Application Gateway | X-Forwarded-For |
| GCP passthrough Network LB | Source preserved |
| GCP Application LB and proxy Network LB | X-Forwarded-For, or PROXY protocol for TCP proxy |
Side effects of preserving the client IP
- The target's security group must allow the real client addresses, not just the load balancer.
- Replies must go back through the load balancer path. With a passthrough design, a server whose default route points elsewhere breaks the flow.
- Hairpin: with NLB client IP preservation, a target that connects to its own NLB and is balanced to itself sees a packet from its own IP to its own IP, and the connection fails.
Kubernetes
A LoadBalancer Service with externalTrafficPolicy: Cluster lets kube-proxy send traffic to a pod on
another node, and it SNATs the packet, so the pod sees a node IP. externalTrafficPolicy: Local keeps
traffic on nodes that run a pod and preserves the client IP, at the cost of possibly uneven spread.
They will ask next
Can you trust X-Forwarded-For?
Only the entries added by your own proxies. A client can send its own header, so the application should take the address added by the last trusted hop, not the first value in the list.
What does proxy protocol v2 add on an NLB?
A binary header at the start of the TCP stream with the original source and destination. Through PrivateLink it also carries the VPC endpoint ID. The backend must be configured to expect it, or it will treat the header as garbage.
Traps
- Turning on proxy protocol on the NLB without enabling it on the backend, which then fails every request.
- Assuming an NLB always preserves the client IP whatever the target type.