Datacenter: Troubleshooting steps for EVPN VXLAN DC environment

Today I am going to talk about EVPN VXLAN Troubleshooting steps. we are taking through one by one to see the issues and verify the issues in each step. We are going to troubleshoot EVPN VXLAN in 16 steps. Before we start with the troubleshooting steps, let's talk about EVPN VXLAN.

An EVPN-VXLAN architecture supports efficient Layer 2/Layer 3 network connectivity with scale, simplicity, agility and with low cost. EVPN-VXLAN decouples the underlay network (physical topology) from the overlay network (virtual topology). EVPN resolves challenges to built data centers to offer cloud and virtualization services. The main application of EVPN is Data Center Interconnect (DCI)

By using overlays, you gain the flexibility of providing Layer 2/Layer 3 connectivity between endpoints across campus and data centers, while maintaining a consistent underlay architecture.

EVPN with VXLAN encapsulation handles Layer 2 connectivity at the scale required by cloud server providers and replaces limiting protocols like Spanning Tree Protocol (STP), freeing up your Layer 3 network to use more robust routing protocols.

Fig 1.1- VXLAN EVPN Multi-Site

Features of EVPN VXLAN

  • Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
  • Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
  • Forwarding decision based on Control-Plane (minimizes flooding)
  • Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay
  • Multi-Tenancy At Scale

Following are the steps to troubleshoot the basic scenario of EVPN VXLAN

Step 1: Verify Underlay IGP/BGP or eBGP is configured properly. For that you need to check IGP and BGP show commands like show BGP peers, IGP adjacencies established and all. 

Step 2: Verify Underlay Multicast is configured properly and below are the commands used for it
sh ip mroute
sh ip mfib
sh ip pim rp

Step 3:Verify L2 VNI is provisioned properly in NVE
sh nve vni

Step 4: Verify EVPN Instance is provisioned properly in EVPN Manager
sh l2vpn evpn evi xx detail

Step 5: Verify L2 Topology for the Access VLAN is properly provisioned in L2RIB
show l2rib topologies detail

Step 6: Verify EVI context is properly added to BGP
show bgp l2vpn evpn evi context

Step 7a- Verify MAC Table in IOS-MATM (local MACs only)
show mac address-table vlan xx

Step 7b- Verify MAC Table in FED-MATM
show platform software fed switch active matm macTable vlan xx

Step 7c- Verify MAC entries in SISF
show device-tracking database mac 

Step 8 : Verify MAC entries in EVPN Mgr
show l2vpn evpn mac/mac ip  (MAC only or MAC/IP route)

Step9: Verify MAC routes  in L2RIB
show l2route evpn mac/mac ip (MAC only or MAC/IP route)

Step 10: Verify MAC/IP, Prefix routes in BGP
show bgp l2vpn evpn evi xx     (xx is evi #)
show bgp l2vpn evpn evi 1 route-type 2
show bgp l2vpn evpn route-type 5

Step 11: Verify MAC routes check in L2FIB
show l2fib bridge-domain xx detail   (xx is bridge domain #)

Step 12:Verify Access SVIs, Core SVIs and NVE Interface are all UP
show ip interface brief

Step 13:Verify EVPN Mgr got all L2 and IRB attributes from NVE
show l2vpn evpn evi detail 

Step 14: Verify Remote L3 VNIs are received by NVE from BGP
show nve peers

Step 15: Verify Remote MAC/IP route in IP VRF xx
show bgp vpnv4 unicast vrf xx    (xx is IP-VRF name)

Step 16a: Verify RNH in BGP
show bgp l2vpn evpn rnh vrf xx   (xx is IP-VRF name)

Step 16b: Verify Remote IP route in RIB
show ip route vrf xx  (xx is IP-VRF name)

Step 16c: Execute show tech command
show tech-support evpn
show tech-support evpn | redirect  xx   (xx is location to collect info)