Octet

Mirror a port locally, across switches with RSPAN, and over IP with ERSPAN

Cisco IOSWorking

Tasks

  1. Mirror both directions of Gi1/0/5 to a laptop on ASW1 Gi1/0/48 with local SPAN.
  2. Remove the local session, then carry the same traffic to SNIF1 on ASW2 Gi1/0/48 with RSPAN VLAN 900.
  3. Make sure VLAN 900 exists as a remote-span VLAN and is allowed on every trunk in the path, including DSW1.
  4. Replace RSPAN with an ERSPAN session that sends received traffic on Gi1/0/5 to 10.9.9.50, ERSPAN ID 100, sourced from 10.0.0.11.
  5. Verify each session and explain the bandwidth and MTU limits to the person reading the capture.

Solution

  1. 1Local SPAN on ASW1ASW1
    ASW1 on Cisco IOS
    monitor session 1 source interface GigabitEthernet1/0/5 both
    monitor session 1 destination interface GigabitEthernet1/0/48

    Gi1/0/48 stops working as a normal port as soon as it becomes a destination: it forwards only mirrored traffic and learns no MAC addresses. Add encapsulation replicate to the destination line if you need to see tags and layer 2 protocols such as CDP and STP as they were.

  2. 2Remove the local sessionASW1
    ASW1 on Cisco IOS
    no monitor session 1

    Free the source port before building the RSPAN session for it, so one port is not a source in two sessions.

  3. 3RSPAN VLAN on every switch in the pathASW1
    ASW1 on Cisco IOS
    vlan 900
     name RSPAN
     remote-span
    !
    interface GigabitEthernet1/1/1
     switchport trunk allowed vlan add 900

    Repeat on ASW2 (and its trunk Gi1/1/1) and on DSW1 (and both trunks Gi1/0/1 and Gi1/0/2). remote-span turns off MAC learning in the VLAN. Never put access ports in it. Use add, or you replace the allowed list and cut VLANs 10, 20 and 99.

  4. 4RSPAN source session on ASW1ASW1
    ASW1 on Cisco IOS
    monitor session 1 source interface GigabitEthernet1/0/5 both
    monitor session 1 destination remote vlan 900
  5. 5RSPAN destination session on ASW2ASW2
    ASW2 on Cisco IOS
    monitor session 1 source remote vlan 900
    monitor session 1 destination interface GigabitEthernet1/0/48

    Session numbers are local to each switch. They do not have to match, but matching them helps the next engineer.

  6. 6Replace RSPAN with ERSPAN on ASW1ASW1
    ASW1 on Cisco IOS
    no monitor session 1
    !
    interface Loopback0
     ip address 10.0.0.11 255.255.255.255
    !
    monitor session 2 type erspan-source
     source interface GigabitEthernet1/0/5 rx
     no shutdown
     destination
      erspan-id 100
      ip address 10.9.9.50
      origin ip address 10.0.0.11

    An ERSPAN session is created shut down, so no shutdown is required. The mirrored frames travel inside GRE to 10.9.9.50, so they can cross any routed path, including a WAN. Wireshark decodes ERSPAN on the analyzer without extra setup. Support for ERSPAN depends on the platform and software release.

Verify

ASW1 output
ASW1# show monitor session 1
Session 1
---------
Type                     : Local Session
Source Ports             :
    Both                 : Gi1/0/5
Destination Ports        : Gi1/0/48
    Encapsulation        : Native
          Ingress        : Disabled

The local session mirrors both directions of Gi1/0/5 to Gi1/0/48, and the laptop sees PC1's traffic in Wireshark.

ASW1 output
ASW1# show vlan remote-span
Remote SPAN VLANs
------------------------------------------------------------------------------
900

VLAN 900 is a remote-span VLAN on this switch. Run the same command on DSW1 and ASW2.

DSW1 output
DSW1# show interfaces trunk
Port        Mode             Encapsulation  Status        Native vlan
Gi1/0/1     on               802.1q         trunking      999
Gi1/0/2     on               802.1q         trunking      999

Port        Vlans allowed on trunk
Gi1/0/1     10,20,99,900
Gi1/0/2     10,20,99,900

Port        Vlans allowed and active in management domain
Gi1/0/1     10,20,99,900
Gi1/0/2     10,20,99,900

Port        Vlans in spanning tree forwarding state and not pruned
Gi1/0/1     10,20,99,900
Gi1/0/2     10,20,99,900

VLAN 900 is allowed, active and forwarding on both trunks of the transit switch. If it is missing here, RSPAN traffic dies at DSW1.

ASW1 output
ASW1# show monitor session 1
Session 1
---------
Type                     : Remote Source Session
Source Ports             :
    Both                 : Gi1/0/5
Dest RSPAN VLAN          : 900

The RSPAN source session sends both directions of Gi1/0/5 into VLAN 900.

ASW2 output
ASW2# show monitor session 1
Session 1
---------
Type                     : Remote Destination Session
Source RSPAN VLAN        : 900
Destination Ports        : Gi1/0/48
    Encapsulation        : Native
          Ingress        : Disabled

ASW2 takes VLAN 900 off the trunk and delivers it to SNIF1 on Gi1/0/48.

ASW1 output
ASW1# show monitor session 2
Session 2
---------
Type                     : ERSPAN Source Session
Status                   : Admin Enabled
Source Ports             :
    RX Only              : Gi1/0/5
Destination IP Address   : 10.9.9.50
Destination ERSPAN ID    : 100
Origin IP Address        : 10.0.0.11

The session is admin enabled. On ANA1, the Wireshark display filter erspan shows GRE packets from 10.0.0.11, each carrying one of PC1's frames.

Mistakes people make

  • Forgetting that the destination port is no longer a normal port. A laptop on it loses its own network access, and a server port chosen by accident goes down.
  • Oversubscribing the destination: both directions of a busy 1 Gbps port can add up to 2 Gbps, and a 1 Gbps destination drops the excess without telling you. Mirror one direction or use a faster destination.
  • Missing the RSPAN VLAN on one transit switch or trunk. The source session looks fine and the analyzer sees nothing.
  • Leaving the ERSPAN session shut down, or using an origin address that is not on the switch or not routable to the analyzer.
  • Ignoring ERSPAN overhead: about 50 bytes per frame (outer IP 20, GRE 8, ERSPAN type II header 8, original Ethernet header 14). Full-size frames need a larger MTU on the path or they are fragmented or dropped.
  • Assuming unlimited sessions. Each platform limits how many source sessions it supports and what can be a source, so check before you rely on several at once.

On other platforms

Local port mirroring on an EX switch with the ELS configuration style.

Junos
set forwarding-options analyzer SPAN1 input ingress interface ge-0/0/5.0
set forwarding-options analyzer SPAN1 input egress interface ge-0/0/5.0
set forwarding-options analyzer SPAN1 output interface ge-0/0/47.0

Local mirroring on Arista EOS. Sessions are named rather than numbered.

Arista EOS
monitor session SPAN1 source Ethernet5 both
monitor session SPAN1 destination Ethernet48

On the analyzer, write everything the mirror port receives to a file. For ERSPAN, filter on GRE.

Linux
sudo tcpdump -ni ens192 -w /var/tmp/span.pcap
sudo tcpdump -ni ens192 -w /var/tmp/erspan.pcap 'ip proto 47 and host 10.0.0.11'

Say it in the interview

"Local SPAN copies a port to another port on the same switch. RSPAN carries the copy across trunks in a dedicated remote-span VLAN, which has to exist and be allowed on every switch in the path. ERSPAN wraps the copy in GRE so it can reach an analyzer anywhere over IP, at the cost of about 50 bytes of overhead. In all three, I watch the destination's bandwidth: mirroring both directions of a busy port can exceed it, and a mirror that drops frames makes a capture lie."