Octet

Prove what a router receives and sends with Embedded Packet Capture

Cisco IOSWorking

Tasks

  1. Write an ACL that matches traffic between PC1 and SRV1 in both directions.
  2. Create a capture named CAP on both Gi0/0/1 and Gi0/0/0, filtered by that ACL, with a 10 MB buffer and a 120 second limit.
  3. Start the capture, have PC1 ping SRV1, and stop the capture.
  4. Read the buffer on the router and decide whether the problem is before, at or after R1.
  5. Export the capture to a pcap file, copy it off the router, and remove the capture.

Solution

  1. 1Filter both directions of the conversationR1
    R1 on Cisco IOS
    ip access-list extended CAP-FILTER
     permit ip host 10.10.10.25 host 10.60.0.15
     permit ip host 10.60.0.15 host 10.10.10.25

    A capture filter matches source and destination as written. If you only list PC1 as the source, you never see the replies and may wrongly conclude there are none.

  2. 2Define the capture point (privileged EXEC)R1
    R1 on Cisco IOS
    monitor capture CAP interface GigabitEthernet0/0/1 both
    monitor capture CAP interface GigabitEthernet0/0/0 both
    monitor capture CAP access-list CAP-FILTER
    monitor capture CAP buffer size 10
    monitor capture CAP limit duration 120

    Two interfaces in one capture show the same packet twice: once arriving on the LAN, once leaving on the WAN. That pair is the proof that R1 forwarded it. The buffer size is in megabytes and the duration in seconds.

  3. 3Start, generate traffic, stopR1
    R1 on Cisco IOS
    monitor capture CAP start
    monitor capture CAP stop

    Between the two commands, run ping 10.60.0.15 on PC1. The capture also stops by itself after 120 seconds.

  4. 4Export, copy off and clean upR1
    R1 on Cisco IOS
    monitor capture CAP export bootflash:cap.pcap
    copy bootflash:cap.pcap scp://netops@10.1.20.15/cap.pcap
    no monitor capture CAP
    delete bootflash:cap.pcap

    Open cap.pcap in Wireshark on your PC. Remove the capture and the file when you are done so nobody finds a forgotten capture running months later.

Verify

R1 output
R1# show monitor capture CAP buffer brief
Starting the packet display ........ Press Ctrl + Shift + 6 to exit

 1   0.000000  10.10.10.25 -> 10.60.0.15   ICMP 74 Echo (ping) request  id=0x0001, seq=41/10496, ttl=128 (no response found!)
 2   0.000022  10.10.10.25 -> 10.60.0.15   ICMP 74 Echo (ping) request  id=0x0001, seq=41/10496, ttl=127 (no response found!)
 3   1.004113  10.10.10.25 -> 10.60.0.15   ICMP 74 Echo (ping) request  id=0x0001, seq=42/10752, ttl=128 (no response found!)
 4   1.004135  10.10.10.25 -> 10.60.0.15   ICMP 74 Echo (ping) request  id=0x0001, seq=42/10752, ttl=127 (no response found!)
 5   2.008270  10.10.10.25 -> 10.60.0.15   ICMP 74 Echo (ping) request  id=0x0001, seq=43/11008, ttl=128 (no response found!)
 6   2.008291  10.10.10.25 -> 10.60.0.15   ICMP 74 Echo (ping) request  id=0x0001, seq=43/11008, ttl=127 (no response found!)

Each request appears twice, 21 to 22 microseconds apart. The first copy has TTL 128 (as sent by Windows, arriving on Gi0/0/1). The second has TTL 127 (decremented by R1, leaving on Gi0/0/0). R1 receives and forwards every request. No reply arrives on the WAN side, so the problem is beyond R1: the WAN, R2 or SRV1. In a healthy capture, each reply also appears twice, arriving on Gi0/0/0 and leaving on Gi0/0/1 with a TTL one lower.

R1 output
R1# monitor capture CAP export bootflash:cap.pcap
Exported Successfully

The file is written to bootflash and can be copied off with SCP or TFTP.

Mistakes people make

  • Typing the monitor capture commands in configuration mode on IOS-XE 16 or later. They are privileged EXEC commands there.
  • Filtering only one direction, then reporting that the server never replies.
  • Capturing a busy interface with no filter. The buffer fills in seconds, and a linear buffer then stops recording.
  • Forgetting that a packet captured at two points appears twice. Wireshark's TCP analysis can flag the second copy as a retransmission. Check the TTL or the IP ID before calling it loss.
  • Leaving captures and pcap files on the router after the incident.

On other platforms

Classic IOS 15 (for example ISR G2) uses a buffer and a capture point that you associate.

Cisco IOS
monitor capture buffer BUF size 2048 max-size 1518 linear
monitor capture buffer BUF filter access-list CAP-FILTER
monitor capture point ip cef POINT GigabitEthernet0/1 both
monitor capture point associate POINT BUF
monitor capture point start POINT
monitor capture point stop POINT
show monitor capture buffer BUF dump
monitor capture buffer BUF export tftp://10.1.20.15/cap.pcap

monitor traffic shows only traffic to and from the Routing Engine, not transit traffic. For transit packets use port mirroring or a firewall filter with a counter or log action.

Junos
monitor traffic interface ge-0/0/1 no-resolve

On a Linux router, one capture per interface proves the same in and out story.

Linux
sudo tcpdump -ni eth1 'host 10.10.10.25 and host 10.60.0.15'
sudo tcpdump -ni eth0 'host 10.10.10.25 and host 10.60.0.15'

Say it in the interview

"When someone says the router drops their traffic, I capture on the router itself with EPC: one capture on the inbound and outbound interfaces, filtered by an ACL that matches both directions. If each packet appears twice with the TTL one lower the second time, the router forwarded it. If no replies arrive on the WAN side, the problem is past my router, and I have a pcap to prove it."

Book 3.3 Internet (IP)