Bond two NICs with LACP and run VLAN interfaces on top with netplan

LinuxWorking

Connect server SRV1 to switch SW1 with two 1G links in an LACP bond, carry VLANs 10 and 20 tagged over the bond, and give SRV1 an address in each VLAN. Configure both ends, apply it safely, and prove LACP and the VLANs are working.

VLAN	Subnet	SRV1 address	Gateway
10	10.10.0.0/24	10.10.0.21	10.10.0.1 (default route)
20	10.20.0.0/24	10.20.0.21	none, local subnet only

SW1 ports Gi1/0/1 and Gi1/0/2 connect to SRV1's ens1f0 and ens1f1.

Tasks

Configure Po10 on SW1 as an LACP trunk carrying VLANs 10 and 20.
Bond ens1f0 and ens1f1 into bond0 in 802.3ad mode with link monitoring and layer3+4 hashing.
Create VLAN interfaces bond0.10 and bond0.20 with their addresses and the default route.
Check the netplan file, then apply it from the server console.
Verify the LACP aggregator, the addresses and the switch side.

Solution

1Switch sideSW1

SW1 on Cisco IOS

interface Port-channel10
 description SRV1 bond0
 switchport mode trunk
 switchport trunk allowed vlan 10,20
!
interface range GigabitEthernet1/0/1 - 2
 description SRV1 ens1f0/ens1f1
 switchport mode trunk
 switchport trunk allowed vlan 10,20
 channel-group 10 mode active

mode active sends LACPDUs. Linux bonds in 802.3ad mode are active too, so either end may start the negotiation.

2Netplan on SRV1SRV1

SRV1 on YAML

# /etc/netplan/01-bond.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    ens1f0: {}
    ens1f1: {}
  bonds:
    bond0:
      interfaces: [ens1f0, ens1f1]
      parameters:
        mode: 802.3ad
        lacp-rate: fast
        mii-monitor-interval: 100
        transmit-hash-policy: layer3+4
  vlans:
    bond0.10:
      id: 10
      link: bond0
      addresses: [10.10.0.21/24]
      routes:
        - to: default
          via: 10.10.0.1
    bond0.20:
      id: 20
      link: bond0
      addresses: [10.20.0.21/24]

The member NICs must be declared under ethernets, even with no settings. Netplan files should be readable by root only (chmod 600), or newer netplan versions warn.

3Check, then apply from the consoleSRV1
SRV1 on Linux
```
netplan generate
netplan apply
```
netplan generate catches YAML and schema mistakes without touching the interfaces. netplan try would roll back after 120 seconds, but it refuses configs that set custom bond or bridge parameters, so apply this one from the out-of-band console (iDRAC, iLO, IPMI), not over SSH through the links you are changing.

Verify

SRV1 output

SRV1# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v6.8.0-48-generic

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100

802.3ad info
LACP active: on
LACP rate: fast
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 2
        Partner Key: 10
        Partner Mac Address: 00:1c:7f:3a:11:80

Slave Interface: ens1f0
MII Status: up
Speed: 1000 Mbps
Aggregator ID: 1

Slave Interface: ens1f1
MII Status: up
Speed: 1000 Mbps
Aggregator ID: 1

Trimmed. Both slaves are up and in Aggregator ID 1, the active aggregator has 2 ports, and the partner MAC is SW1's. A partner MAC of 00:00:00:00:00:00 means the switch is not speaking LACP. Different aggregator IDs mean the two links did not bundle. Cisco uses the channel-group number as its LACP key, hence Partner Key 10.

SRV1 output

SRV1# ip -br -4 addr show
lo               UNKNOWN        127.0.0.1/8
bond0.10@bond0   UP             10.10.0.21/24
bond0.20@bond0   UP             10.20.0.21/24

The addresses sit on the VLAN interfaces, not on bond0 or the physical NICs.

SW1 output

SW1# show etherchannel 10 summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        S - Layer2      U - in use
Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
10     Po10(SU)        LACP        Gi1/0/1(P)  Gi1/0/2(P)

Trimmed legend. Po10 is layer 2 and in use (SU), both members bundled (P).

SRV1 output

SRV1# ping -c 2 -I bond0.10 10.10.0.1
PING 10.10.0.1 (10.10.0.1) from 10.10.0.21 bond0.10: 56(84) bytes of data.
64 bytes from 10.10.0.1: icmp_seq=1 ttl=255 time=0.842 ms
64 bytes from 10.10.0.1: icmp_seq=2 ttl=255 time=0.611 ms

--- 10.10.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 0.611/0.726/0.842/0.115 ms

The gateway in VLAN 10 answers. TTL 255 is the IOS-XE default, so the reply came straight from the Cisco gateway.

Mistakes people make

Configuring 802.3ad on the server against plain trunk ports with no channel-group. No LACP partner answers, so the links do not bundle and only one carries traffic.
Leaving the default layer2 hash, so all traffic to the gateway's MAC uses one link.
Putting an IP on bond0 as well as on the VLAN interfaces, which puts the host in the native VLAN by accident.
Pulling one cable as the only test. Also check Link Failure Count and that traffic moves to the remaining link without the session dropping.
Running netplan apply over SSH through the links you are changing. netplan try cannot roll back bond parameters, so use the console.

On other platforms

The same bond and VLANs with iproute2, runtime only. Members must be down before they are enslaved.

Linux

ip link add bond0 type bond mode 802.3ad miimon 100 lacp_rate fast xmit_hash_policy layer3+4
ip link set ens1f0 down
ip link set ens1f0 master bond0
ip link set ens1f1 down
ip link set ens1f1 master bond0
ip link set bond0 up
ip link add link bond0 name bond0.10 type vlan id 10
ip link add link bond0 name bond0.20 type vlan id 20
ip addr add 10.10.0.21/24 dev bond0.10
ip addr add 10.20.0.21/24 dev bond0.20
ip link set bond0.10 up
ip link set bond0.20 up
ip route add default via 10.10.0.1

NetworkManager equivalent on RHEL family hosts.

Linux

nmcli con add type bond con-name bond0 ifname bond0 bond.options "mode=802.3ad,miimon=100,lacp_rate=fast,xmit_hash_policy=layer3+4" ipv4.method disabled ipv6.method ignore
nmcli con add type ethernet con-name bond0-p1 ifname ens1f0 slave-type bond master bond0
nmcli con add type ethernet con-name bond0-p2 ifname ens1f1 slave-type bond master bond0
nmcli con add type vlan con-name bond0.10 ifname bond0.10 vlan.parent bond0 vlan.id 10 ipv4.method manual ipv4.addresses 10.10.0.21/24 ipv4.gateway 10.10.0.1
nmcli con add type vlan con-name bond0.20 ifname bond0.20 vlan.parent bond0 vlan.id 20 ipv4.method manual ipv4.addresses 10.20.0.21/24

Say it in the interview

"LACP channel-group in active mode on the switch, 802.3ad bond with miimon and layer3+4 hashing on the server, and VLAN interfaces on the bond rather than the NICs. I check the file with netplan generate and apply it from the console, because netplan try will not roll back bond parameters. Then /proc/net/bonding: both slaves in the same aggregator and a real partner MAC means LACP is truly up."

Book 3.2 Switched Ethernet