Build a two-host network inside one Linux box with namespaces, veth and a bridge

LinuxWorking

Create two isolated hosts, red and blue, as network namespaces on HOST1. Connect them to a bridge that acts as their switch and default gateway, let them talk to each other, and give them internet access through HOST1's uplink with NAT. This is the same plumbing Docker builds for docker0.

Item	Value
HOST1 uplink	`ens192`, 10.0.10.21/24, default via 10.0.10.1
Bridge	`br0`, 10.50.0.1/24 (gateway for the namespaces)
Namespace red	`veth-red`, 10.50.0.10/24
Namespace blue	`veth-blue`, 10.50.0.20/24
Test address outside	198.51.100.1

Tasks

Create namespaces red and blue and a bridge br0 with the gateway address.
Connect each namespace to br0 with a veth pair.
Address both namespaces, bring up their interfaces including loopback, and add default routes.
Enable forwarding and MASQUERADE the 10.50.0.0/24 subnet out of ens192.
Make sure the host firewall forwards the traffic if its FORWARD policy is DROP.

Solution

1Namespaces and the bridgeHOST1

HOST1 on Linux

ip netns add red
ip netns add blue
ip link add br0 type bridge
ip addr add 10.50.0.1/24 dev br0
ip link set br0 up

2One veth pair per namespaceHOST1

HOST1 on Linux

ip link add veth-red type veth peer name veth-red-br
ip link set veth-red netns red
ip link set veth-red-br master br0
ip link set veth-red-br up

ip link add veth-blue type veth peer name veth-blue-br
ip link set veth-blue netns blue
ip link set veth-blue-br master br0
ip link set veth-blue-br up

Interface names are limited to 15 characters. The -br end stays in the root namespace as a bridge port.

3Addresses, loopback and default routes inside the namespacesHOST1

HOST1 on Linux

ip -n red link set lo up
ip -n red addr add 10.50.0.10/24 dev veth-red
ip -n red link set veth-red up
ip -n red route add default via 10.50.0.1

ip -n blue link set lo up
ip -n blue addr add 10.50.0.20/24 dev veth-blue
ip -n blue link set veth-blue up
ip -n blue route add default via 10.50.0.1

ip -n red ... is short for ip netns exec red ip ....

4Forwarding and NAT to the outsideHOST1
HOST1 on Linux
```
sysctl -w net.ipv4.ip_forward=1
iptables -t nat -A POSTROUTING -s 10.50.0.0/24 ! -o br0 -j MASQUERADE
```
! -o br0 keeps traffic between red and blue untranslated. Only traffic leaving through another interface is NATed.
5Allow forwarding if the FORWARD policy is DROPHOST1
HOST1 on Linux
```
iptables -I FORWARD 1 -i br0 ! -o br0 -j ACCEPT
iptables -I FORWARD 2 -o br0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -I FORWARD 3 -i br0 -o br0 -j ACCEPT
```
Needed on hosts with Docker, which sets the FORWARD policy to DROP. The third rule matters only when br_netfilter is loaded, because then even bridged red-to-blue frames pass through FORWARD.

Verify

HOST1 output

HOST1# ip -br link show master br0
veth-red-br@if5  UP             d2:5f:0a:91:3c:11 <BROADCAST,MULTICAST,UP,LOWER_UP>
veth-blue-br@if7 UP             6e:21:b4:07:9a:5d <BROADCAST,MULTICAST,UP,LOWER_UP>

Both bridge ports are UP with LOWER_UP, which for a veth means the namespace end is up too. @if5 is the peer's interface index inside its namespace; your numbers will differ.

HOST1 output

HOST1# ip netns exec red ip -br -4 addr; ip netns exec red ip route
lo               UNKNOWN        127.0.0.1/8
veth-red@if4     UP             10.50.0.10/24
default via 10.50.0.1 dev veth-red
10.50.0.0/24 dev veth-red proto kernel scope link src 10.50.0.10

Loopback is up, the address is on veth-red, and the default route points at br0's address.

HOST1 output

HOST1# ip netns exec red ping -c 2 10.50.0.20
PING 10.50.0.20 (10.50.0.20) 56(84) bytes of data.
64 bytes from 10.50.0.20: icmp_seq=1 ttl=64 time=0.071 ms
64 bytes from 10.50.0.20: icmp_seq=2 ttl=64 time=0.049 ms

--- 10.50.0.20 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1027ms
rtt min/avg/max/mdev = 0.049/0.060/0.071/0.011 ms

TTL 64 means the packet was bridged, not routed. red and blue share a layer 2 segment.

HOST1 output

HOST1# ip netns exec red ping -c 1 198.51.100.1 >/dev/null; conntrack -L -p icmp
icmp     1 29 src=10.50.0.10 dst=198.51.100.1 type=8 code=0 id=4121 src=198.51.100.1 dst=10.0.10.21 type=0 code=0 id=4121 mark=0 use=1
conntrack v1.4.7 (conntrack-tools): 1 flow entries have been shown.

The reply tuple is addressed to 10.0.10.21, HOST1's uplink address. MASQUERADE translated red's traffic.

Mistakes people make

Forgetting ip link set lo up inside the namespace, which breaks anything that talks to 127.0.0.1 there.
Bringing up only one end of a veth pair. The other end shows NO-CARRIER and nothing passes.
Writing MASQUERADE without ! -o br0 or -o ens192, so traffic between the namespaces is translated too.
Expecting DNS to work inside the namespace when the host's resolv.conf points at 127.0.0.53. Create /etc/netns/red/resolv.conf with a reachable server.
Forgetting that all of this is runtime only. A reboot removes the namespaces, the bridge and the rules.

On other platforms

The NAT step with nftables instead of iptables.

Linux

nft add table ip nat
nft add chain ip nat postrouting '{ type nat hook postrouting priority srcnat; policy accept; }'
nft add rule ip nat postrouting ip saddr 10.50.0.0/24 oifname != "br0" masquerade

Say it in the interview

"Each namespace is a host with its own stack, a veth pair is the cable, and the bridge is the switch with the gateway address on it. Then forwarding plus MASQUERADE out the uplink. Docker's default network is exactly this: docker0, a veth per container and one MASQUERADE rule."

Book 3.5 Implementation