Using one Linux host, build two isolated hosts that can talk to each other and reach the internet. Walk me through it.
Show the answer
Say this first
Two namespaces as the hosts, a bridge in the root namespace as their switch and gateway, one veth pair per
namespace with one end on the bridge. Address the namespaces in one subnet, put the gateway address on the
bridge, add a default route in each namespace. For the internet: turn on ip_forward and MASQUERADE the
subnet out the uplink. That is exactly what Docker does with docker0, and building it by hand shows you
understand every piece.
Go deeper
The build
ip netns add red
ip netns add blue
ip link add br0 type bridge
ip addr add 10.50.0.1/24 dev br0
ip link set br0 up
for ns in red blue; do
ip link add veth-$ns type veth peer name veth-$ns-br
ip link set veth-$ns netns $ns
ip link set veth-$ns-br master br0
ip link set veth-$ns-br up
ip -n $ns link set lo up
ip -n $ns link set veth-$ns up
done
ip -n red addr add 10.50.0.10/24 dev veth-red
ip -n blue addr add 10.50.0.20/24 dev veth-blue
ip -n red route add default via 10.50.0.1
ip -n blue route add default via 10.50.0.1
sysctl -w net.ipv4.ip_forward=1
iptables -t nat -A POSTROUTING -s 10.50.0.0/24 ! -o br0 -j MASQUERADETest in order
ip netns exec red ping -c 2 10.50.0.20: bridging works (TTL 64, nothing routed).ip netns exec red ping -c 2 10.50.0.1: the gateway answers.ip netns exec red ping -c 2 198.51.100.1: forwarding and NAT work.conntrack -L -p icmpshows the translation to the host's uplink address.
What usually breaks
- A FORWARD policy of DROP, typical on a host with Docker. Allow
-i br0out and established traffic back. br_netfilterloaded withbridge-nf-call-iptables=1: then even red-to-blue bridged traffic must pass FORWARD.- DNS inside the namespace:
ip netns execuses/etc/netns/red/resolv.confif it exists, else the host's, which may point at 127.0.0.53, a stub that does not exist inside the namespace.
They will ask next
How would you add a second subnet so red and blue are routed instead of bridged?
Put them on separate bridges (or straight veth to the root namespace) with different subnets, give the host an address in each, and point each namespace's default route at its own gateway. With
ip_forwardon, the host routes between them, and the TTL drops by one.Why can blue not resolve names when red can ping by IP?
Most likely the host's resolv.conf points at the systemd-resolved stub 127.0.0.53, which only listens in the root namespace. Create
/etc/netns/blue/resolv.confwith a reachable server.
Traps
- Forgetting
loand one veth end, then debugging routing that is fine. - Writing MASQUERADE without
! -o br0and translating the traffic between the namespaces too.