Anycast Gateways in EVPN Networks
Keywords: Anycast Gateway EVPN • Distributed Anycast Gateway • VXLAN Anycast Gateway • BGP EVPN Anycast • Anycast Gateway vs HSRP VRRP • EVPN Type 2 Route Anycast • IRB Anycast Gateway • ARP Suppression EVPN • Cisco NX-OS Anycast Gateway • Arista EOS Anycast Gateway • SONiC Anycast Gateway • VXLAN L3 Gateway • Leaf Switch Gateway • Fabric Forwarding Anycast • EVPN Symmetric IRB • Data Center Gateway Redundancy
For twenty years, enterprise networks used HSRP or VRRP to provide gateway redundancy: one active router handling all traffic while a standby router watched and waited. In a spine-leaf data center with VXLAN and BGP EVPN, that model fails because it creates a bottleneck, wastes half your uplink bandwidth, and adds latency through suboptimal forwarding paths. Anycast gateway is the replacement — every leaf switch simultaneously answers ARP for the same gateway IP address. There is no active/standby. Every leaf is the gateway for every host it connects to. This guide explains how that works technically, why it’s correct behavior and not a duplicate address conflict, and how to configure it on Cisco NX-OS, Arista EOS, and SONiC.
May 2026 | ⏱ 35 min read | VXLAN • BGP EVPN • IRB • ARP Suppression • VTEP • VNI • NX-OS / EOS / SONiC | ⚙ Network Engineers • DC Architects • CCIE/CCNP Candidates
Sections in This Guide
|
1. The Problem Anycast Gateway Solves 2. How Anycast Gateway Works in EVPN 3. The Virtual Router MAC (vMAC) 4. BGP EVPN Route Types and Anycast 5. ARP Suppression with Anycast Gateway 6. Symmetric vs Asymmetric IRB |
7. Cisco NX-OS Configuration 8. Arista EOS Configuration 9. SONiC / FRRouting Configuration 10. Anycast Gateway vs HSRP/VRRP 11. Multi-Site Anycast Gateway 12. Troubleshooting Commands & FAQ |
1. The Problem Anycast Gateway Solves
Consider a standard enterprise network with VLANs 10 and 20 sharing a distribution-layer switch as their default gateway. HSRP makes one switch the active gateway and the other standby. When a server in VLAN 10 on Access Switch A sends traffic to a server in VLAN 20 on Access Switch Z, the traffic path is: Server A → Access A → Distribution (active HSRP) → Access Z → Server Z. If the active distribution switch is on the far side of the campus, traffic traverses the entire network to hit the gateway, then traverses back. This is the “traffic trombone” problem.
In a modern spine-leaf VXLAN fabric, the equivalent problem is worse. If VLAN 10’s gateway is a specific SVI on a specific leaf switch, then a server in VLAN 10 on Leaf-5 sends all inter-subnet traffic up to a spine, across to the gateway leaf, gets routed there, then routes back across to the destination leaf. Two fabric crossings instead of one, for every inter-VLAN packet. With thousands of VMs generating millions of flows, the cumulative inefficiency is significant — and HSRP/VRRP active/standby means half your gateway bandwidth is always idle.
What anycast gateway provides instead: Every leaf switch that serves VLAN 10 hosts is also the L3 gateway for those hosts. The server sends an ARP request for 192.168.10.1 (the gateway). Its directly connected leaf switch answers. Routing happens on that leaf, not somewhere else in the fabric. One fabric crossing for inter-subnet traffic instead of three.
| Aspect | HSRP/VRRP (Traditional) | Anycast Gateway (EVPN) |
| Gateway count | 1 active, 1+ standby | All leaf switches active simultaneously |
| Gateway IP | Virtual IP shared between active/standby | Same IP and MAC on every leaf |
| L3 routing location | Centralized at active gateway only | Distributed at every leaf (local routing) |
| Traffic path (inter-VLAN) | 3 hops (to active GW + back) | 1 hop (route locally on ingress leaf) |
| Failover time | HSRP: 3–10 seconds default; tuned: ~1s | No failover needed; leaf failure = connect to other leaf |
| STP dependency | Required for loop prevention | Eliminated; VXLAN routed fabric |
Why HSRP and VRRP are wrong for VXLAN fabrics: HSRP and VRRP were designed for a world where routers were expensive and you needed to make one router look like two. In a VXLAN fabric, every leaf already is a router. Running HSRP on top of a fabric where every device could be the gateway is like hiring a committee to decide who gets to make decisions, when everyone in the room is equally qualified. Anycast gateway lets every leaf be the gateway simultaneously, which is the architecturally correct answer for a distributed switching fabric.
2. How Anycast Gateway Works in EVPN
In a VXLAN/BGP EVPN fabric, each leaf switch functions as a VTEP (VXLAN Tunnel Endpoint). When anycast gateway is configured, every leaf that serves VLAN 10 is configured with:
|
Same on every leaf (shared):
● IP address: 192.168.10.1/24 |
Unique per leaf (individual):
● VTEP IP (loopback): 10.0.1.1, 10.0.1.2, etc. |
The ARP sequence, step by step:
1. VM on Leaf-3 (192.168.10.50) sends an ARP request for the default gateway (192.168.10.1).
2. Leaf-3 is configured as anycast gateway for 192.168.10.1. It answers the ARP directly, returning the virtual MAC 00:00:00:00:00:01.
3. The VM caches 192.168.10.1 → 00:00:00:00:00:01 and sends all inter-subnet traffic to that MAC.
4. Leaf-3 receives the frame, sees it destined for 00:00:00:00:00:01 (its own MAC), recognizes it as a routed packet, performs L3 lookup for the destination IP.
5. Leaf-3 routes the packet into the VXLAN fabric toward the destination leaf — one fabric crossing, done.
Now consider what happens if the VM moves from Leaf-3 to Leaf-7. It sends ARP again for 192.168.10.1. Leaf-7 answers with the same virtual MAC (00:00:00:00:00:01). The VM’s ARP cache entry doesn’t even change — the MAC address it was using is still valid because it’s configured identically on every leaf in the fabric. This seamless VM mobility without gateway reconfiguration is one of the most operationally significant benefits of anycast gateway.
| VM-A 192.168.10.50 on Leaf-3 |
→ | Leaf-3 (VTEP) Anycast GW: 192.168.10.1 MAC: 00:00:00:00:00:01 |
→ | Spine L3 VXLAN transport |
→ | Leaf-9 (VTEP) VXLAN decap deliver to VM-B |
→ | VM-B 192.168.20.30 on Leaf-9 |
| Routed at ingress leaf (Leaf-3). One fabric crossing. Destination leaf decapsulates and delivers. | ||||||||
3. The Virtual Router MAC (vMAC) — Why Duplicate MACs Don’t Cause Loops
The most common question when engineers first encounter anycast gateway is: “If every leaf has the same MAC address, won’t the MAC address table have conflicts and loops?” The answer is no, for a specific reason: the virtual MAC is a locally-significant address used only on the access side (between end hosts and their leaf). It is never advertised into the VXLAN overlay or learned by other VTEPs.
Each VTEP has two MAC addresses for the gateway function:
| MAC Type | Value & Scope | Purpose |
| Virtual Router MAC (vMAC) | Same on every leaf e.g., 0200.0000.00aa Scope: access-facing only |
Respond to host ARP. Hosts use this as destination MAC for all routed traffic. |
| VTEP Physical MAC | Unique per leaf switch e.g., 5254.001a.bc3f Scope: fabric-wide (in VXLAN headers) |
Used in VXLAN outer Ethernet headers for encapsulation. Each VTEP is uniquely identified in the fabric. |
When BGP EVPN advertises MAC/IP bindings (Type 2 routes), it advertises the endpoint MAC and IP addresses of hosts attached to each VTEP. The virtual router MAC is not advertised in EVPN Type 2 routes — it stays local to the access domain. Remote VTEPs never see or learn the vMAC. This is why no MAC address conflict exists in the overlay: every VTEP knows the vMAC is its own gateway MAC locally, and remote VTEPs reach hosts via VTEP IP (outer header) not via gateway MAC.
Cisco’s implementation detail: On Cisco NX-OS, the virtual MAC is automatically derived from the system-mac configured in the fabric forwarding mode anycast-gateway-mac command. Alternatively, you configure a specific MAC using fabric forwarding anycast-gateway-mac. The same MAC must be configured identically on every VTEP in the fabric — a MAC mismatch between leaves causes hosts that move between leaves to see ARP responses with different MACs for the same gateway IP, breaking the session.
4. BGP EVPN Route Types and Anycast Gateway Interaction
Anycast gateway works in combination with BGP EVPN’s MAC/IP advertisement to build the distributed forwarding database. Understanding which route types interact with anycast gateway clarifies the full data plane behavior.
| EVPN Route Type | What It Carries | Anycast Gateway Interaction |
| Type 2 (MAC/IP Advertisement) |
Host MAC address + optionally host IP address + originating VTEP IP | Primary interaction. Each leaf advertises its locally attached endpoint MAC+IP pairs. Remote leaves use this to build their ARP/MAC tables for ARP suppression. When a host sends a packet to the anycast gateway, the leaf routes it using the Type 2 database to find the destination VTEP. |
| Type 3 (Inclusive Multicast) |
VTEP membership in a VNI for BUM traffic | Enables ingress replication list (who receives broadcast/unknown unicast traffic in this VNI). Anycast gateway reduces BUM traffic volume because ARP suppression eliminates most ARP broadcasts. |
| Type 5 (IP Prefix Route) |
Routed IP prefixes between VRFs and external networks | Used in symmetric IRB for inter-subnet routing. The anycast gateway on each leaf routes intra-fabric traffic locally; external prefixes reach the fabric via border leaves through Type 5 routes. |
| Type 1/4 (ESI/AD Routes) |
Ethernet Segment Identifier for multihoming | Used when servers are dual-homed via EVPN ESI multihoming (or MLAG). The anycast gateway concept extends naturally to multihomed scenarios because both MLAG peers share the same gateway IP and vMAC. |
# Verify EVPN Type 2 route advertisement for anycast gateway (Cisco NX-OS)
show bgp l2vpn evpn route-type 2 0 5254.001a.bc3f 192.168.10.50 BGP routing table entry for [2]:[0]:[0]:[48]:[5254.001a.bc3f]:[32]:[192.168.10.50]/272 Paths: (1 available, best #1) Advertised to update-groups: 1 Path type: local, path is valid, is best path EVPN ESI: 00000000000000000000 Gateway IP: 0.0.0.0 Local VNI: 10010 ← L2 VNI for VLAN 10 Local Router MAC: 5254.001a.bc3f ← Physical VTEP MAC (NOT the vMAC) Extended Community: RT:65001:10010 ENCAP:8 # Notice: the virtual router MAC (vMAC) is NOT in the Type 2 route # Only the endpoint host MAC and the VTEP's physical router-mac are present
5. ARP Suppression with Anycast Gateway
ARP suppression and anycast gateway are complementary features that together eliminate nearly all broadcast traffic from the VXLAN fabric. ARP suppression works by building a local ARP database at each VTEP from the EVPN Type 2 MAC/IP routes. When a host sends an ARP request for another host, the local VTEP checks its suppression database. If it knows the answer (because it received a Type 2 route advertising that IP with its associated MAC), it responds directly without flooding the ARP into the fabric.
The relationship between anycast gateway and ARP suppression: the anycast gateway answers ARP requests for the gateway IP locally without any suppression database lookup (it’s always the local leaf). ARP suppression handles ARP requests for other hosts in the same subnet — when VM-A ARPs for VM-B, the leaf checks its Type 2 route database, finds VM-B’s MAC, and answers without flooding.
ARP Resolution Paths with Anycast Gateway + ARP Suppression
| ARP Request Type | Who Answers? | Fabric Flooded? |
| ARP for gateway IP (192.168.10.1) | Local leaf (anycast GW) | NO — answered locally |
| ARP for known host (in Type 2 DB) | Local leaf (suppression) | NO — proxy answer from Type 2 |
| ARP for unknown host (not in Type 2) | All VTEPs in VNI (flooded via BUM) | YES — BUM flood (rare after EVPN converges) |
# Verify ARP suppression database on Cisco NX-OS leaf
show ip arp suppression-cache detail vlan 10 Flags: + - Proxy ARP, L - Local, R - Remote, M - sMAC IP Address Age MAC Address Vlan Interface VTEP 192.168.10.50 00:03 5254.001a.bc3f 10 nve1 ← L = local host on this leaf 192.168.10.51 00:01 5254.002b.cd4e 10 nve1 10.0.1.4 ← R = remote on VTEP 10.0.1.4 192.168.10.1 00:00 0200.0000.00aa 10 Vlan10 ← Local gateway (vMAC) # Verify ARP suppression is enabled on the NVE interface show nve interface nve1 Interface: nve1, Admin State: Up, State: Up Host Reachability Mode: Control-Plane (BGP EVPN) Source-Interface: loopback0 (primary: 10.0.1.1) VNI BD Mode ARP-Suppression 10010 10 L2 Enabled ← ARP suppression active
6. Symmetric vs Asymmetric IRB with Anycast Gateway
Integrated Routing and Bridging (IRB) is the mechanism that allows a VTEP to both bridge (L2) and route (L3) traffic. There are two IRB modes, and the choice affects which VNIs each leaf must be configured with, how the L3 VNI is used, and the overall scalability of the fabric. Anycast gateway works with both modes, but symmetric IRB is the production standard.
| Attribute | Asymmetric IRB | Symmetric IRB |
| How it works | Ingress leaf routes into destination VNI and bridges on egress leaf. Both VNIs must exist on both leaves. | Both ingress and egress leaf route using a dedicated L3 VNI (per-VRF). Only local L2 VNI needed on each leaf. |
| VNI requirement | All L2 VNIs for all subnets must be present on every leaf — poor scalability | Only local L2 VNIs + one L3 VNI per VRF on every leaf. Highly scalable. |
| L3 VNI usage | No dedicated L3 VNI; routes within L2 VNI | Dedicated L3 VNI per VRF carries inter-subnet routed traffic |
| EVPN route used | Type 2 with MAC+IP for host tracking | Type 2 for host + Type 5 for inter-subnet/external prefixes |
| Production recommendation | Avoid for large fabrics; use only in simple/small topologies | Use in all production EVPN fabrics |
Symmetric IRB with anycast gateway data path: VM-A (192.168.10.50, VLAN 10) on Leaf-3 sends traffic to VM-B (192.168.20.30, VLAN 20) on Leaf-9.
Symmetric IRB Forwarding Steps
|
Ingress (Leaf-3): |
Egress (Leaf-9): |
7. Cisco NX-OS Anycast Gateway Configuration
Cisco implements anycast gateway through the fabric forwarding mode anycast-gateway command on VLAN SVIs. The global virtual MAC is configured once and applies to all SVIs that use anycast mode. This is the configuration used on Cisco Nexus 9000 series switches running NX-OS in VXLAN mode.
# Cisco NX-OS Anycast Gateway Configuration — Leaf Switch
# Step 1: Enable required features feature vn-segment-vlan-based feature nv overlay nv overlay evpn # Step 2: Configure global anycast gateway MAC (IDENTICAL on every leaf) fabric forwarding anycast-gateway-mac 0200.0000.00aa # Step 3: Define VLANs and map to VXLAN VNIs vlan 10 name Corporate-Users vn-segment 10010 ← L2 VNI for this VLAN vlan 20 name Voice-VLAN vn-segment 10020 vlan 3967 name VRF-PROD-L3VNI vn-segment 50001 ← L3 VNI for VRF PROD (symmetric IRB) # Step 4: Configure VRF with L3 VNI vrf context PROD vni 50001 rd auto address-family ipv4 unicast route-target both auto evpn # Step 5: Configure VLAN SVIs with anycast gateway interface Vlan10 no shutdown vrf member PROD ip address 192.168.10.1/24 fabric forwarding mode anycast-gateway ← Enable anycast gateway on this SVI no ip redirects no ipv6 redirects interface Vlan20 no shutdown vrf member PROD ip address 192.168.20.1/24 fabric forwarding mode anycast-gateway # Step 6: L3 VNI SVI (no IP, just VRF binding) interface Vlan3967 no shutdown vrf member PROD ip forward ← Forward but no address (L3 VNI) # Step 7: Configure NVE interface interface nve1 no shutdown source-interface loopback0 host-reachability protocol bgp member vni 10010 mcast-group 239.1.1.10 # or: ingress-replication protocol bgp suppress-arp ← Enable ARP suppression member vni 10020 mcast-group 239.1.1.20 suppress-arp member vni 50001 associate-vrf ← Associate L3 VNI with VRF PROD # Step 8: BGP EVPN configuration router bgp 65001 router-id 10.0.1.1 address-family l2vpn evpn advertise-all-vni ← Advertise all local VNIs into EVPN neighbor 10.0.0.1 remote-as 65100 # Spine-1 address-family l2vpn evpn send-community extended neighbor 10.0.0.2 remote-as 65100 # Spine-2 address-family l2vpn evpn send-community extended
Spine BGP EVPN Configuration (Route Reflector)
# Cisco NX-OS Spine — BGP EVPN Route Reflector
router bgp 65100 router-id 10.0.0.1 address-family l2vpn evpn # Spines don't need advertise-all-vni (they don't have VNIs) neighbor 10.0.1.0/24 remote-as 65001 # All leaf loopbacks address-family l2vpn evpn send-community extended route-reflector-client ← Spine reflects EVPN routes between leaves
8. Arista EOS Anycast Gateway Configuration
Arista EOS implements anycast gateway using the ip virtual-router construct. Arista’s implementation allows both a virtual IP and a virtual MAC to be shared across all VTEP leaves that participate in the same VRF. Arista’s VXLAN/EVPN implementation on the 7050, 7060, 7160, 7280, and 7500 series supports distributed anycast gateway with symmetric IRB using the same principles as NX-OS but with different configuration syntax.
# Arista EOS Anycast Gateway Configuration — Leaf Switch
# Step 1: Enable EVPN and VXLAN service routing protocols model multi-agent # (Required for EVPN on EOS 4.20+) # Step 2: Define global virtual router MAC ip virtual-router mac-address 00:1c:73:00:00:01 ← Same on every leaf # Step 3: Configure VRF vrf instance PROD rd 65001:1 route-target import evpn 65001:50001 route-target export evpn 65001:50001 # Step 4: Configure VLANs vlan 10 name Corporate-Users vlan 20 name Voice-VLAN # Step 5: Configure SVIs with virtual-router (anycast gateway) interface Vlan10 vrf PROD ip address 192.168.10.1/24 ip virtual-router address 192.168.10.1 ← Anycast GW IP arp aging timeout 3600 interface Vlan20 vrf PROD ip address 192.168.20.1/24 ip virtual-router address 192.168.20.1 # Step 6: Configure VXLAN interface interface Vxlan1 vxlan source-interface Loopback0 vxlan udp-port 4789 vxlan vlan 10 vni 10010 vxlan vlan 20 vni 10020 vxlan vrf PROD vni 50001 ← L3 VNI for VRF PROD vxlan arp proxy enable ← ARP suppression # Step 7: BGP EVPN configuration router bgp 65001 router-id 10.0.1.2 neighbor SPINES peer group neighbor SPINES remote-as 65100 neighbor SPINES send-community extended neighbor 10.0.0.1 peer group SPINES neighbor 10.0.0.2 peer group SPINES ! vlan 10 rd 65001:10010 route-target both 65001:10010 redistribute learned ← Advertise local MACs/IPs vlan 20 rd 65001:10020 route-target both 65001:10020 redistribute learned ! address-family evpn neighbor SPINES activate ! vrf PROD rd 65001:50001 route-target import evpn 65001:50001 route-target export evpn 65001:50001 redistribute connected ← Advertise connected subnets via EVPN Type 5
Arista’s ip virtual-router vs Cisco’s fabric forwarding: In Arista EOS, ip virtual-router address defines the anycast gateway IP (hosts ARP for this address and get the virtual MAC back). The SVI also has a regular IP address in addition to the virtual-router address — the regular IP is used for routing protocol adjacencies and control plane communication, while the virtual-router IP is the host-facing gateway. On Cisco NX-OS, the same IP serves as both (defined by the fabric forwarding mode anycast-gateway flag on the SVI).
9. SONiC & FRRouting Anycast Gateway Configuration
SONiC implements anycast gateway through its Redis-based configuration model. The SVI configuration sets the anycast gateway MAC via the INTERFACE table in CONFIG_DB. FRRouting (FRR), which runs as a container in SONiC, handles the BGP EVPN control plane. The key configuration piece is advertise-default-gw in FRR’s L2VPN EVPN address family, which tells FRR to advertise the gateway IP/MAC as a Type 2 route with the “default gateway” flag set. Remote VTEPs use this flag to install the anycast gateway into their ARP suppression databases.
# SONiC Anycast Gateway — config_db.json snippet
{ "VLAN": { "Vlan10": { "vlanid": "10" } }, "VLAN_INTERFACE": { "Vlan10": { "vrf_name": "PROD" }, "Vlan10|192.168.10.1/24": { "gw": "true" # Mark this as anycast gateway address } }, "VRF": { "PROD": { "vni": "50001" # L3 VNI for VRF PROD } }, "VXLAN_TUNNEL_MAP": { "vtep|map_10010_Vlan10": { "vlan": "Vlan10", "vni": "10010" } } }
# FRRouting (FRR) vtysh — BGP EVPN with anycast gateway advertisement
router bgp 65001 bgp router-id 10.0.1.1 no bgp ebgp-requires-policy neighbor SPINES peer-group neighbor SPINES remote-as 65100 neighbor 10.0.0.1 peer-group SPINES neighbor 10.0.0.2 peer-group SPINES ! address-family l2vpn evpn neighbor SPINES activate advertise-all-vni ← Advertise all local VNIs advertise-default-gw ← Flag gateway Type 2 routes as default GW autort rfc8365-compatible ← Auto route-targets RFC8365 format exit-address-family # Verify anycast gateway advertisement in FRR show bgp l2vpn evpn vni 10010 VNI: 10010 (known to the kernel) Type: L2 Tenant VRF: PROD RD: 10.0.1.1:10010 Originator IP: 10.0.1.1 Mcast group: (null) Advertise-gw-macip : Yes ← Gateway MAC/IP advertised Advertise-svi-macip : No Number of MACs (local and remote) known for this VNI: 4 Flags: * - Kernel MAC INTF VTEP #Paths * 5254.001a.bc3f Vlan10 Local - * 0200.0000.00aa Vlan10 Local (GW) - ← Anycast gateway MAC 5254.002b.cd4e - 10.0.1.4 1 5254.003c.de5f - 10.0.1.7 1
SONiC anycast gateway MAC configuration: The virtual router MAC in SONiC is configured in CONFIG_DB under the DEVICE_METADATA table key mac or by setting a specific anycast gateway MAC in the interface. Alternatively, many SONiC deployments use the same MAC address configured identically across all leaf switches. Dell Enterprise SONiC and Broadcom SONiC implementations expose this through their management interfaces with explicit anycast gateway CLI commands similar to NX-OS.
10. Anycast Gateway vs HSRP vs VRRP — Detailed Comparison
Understanding what specifically HSRP and VRRP get right and wrong helps justify why anycast gateway is the right architecture for VXLAN fabrics — and also explains why HSRP/VRRP still makes sense in non-VXLAN environments.
| Feature | HSRP (Cisco) | VRRP (RFC 5798) | Anycast Gateway (EVPN) |
| Standard | Cisco proprietary | IETF RFC 5798 (v2, v3) | Vendor-specific but conceptually universal |
| Active routers | 1 active per group | 1 master per group | All leaves simultaneously |
| Failover time | 3–10 seconds (tunable to ~1s) | ~3 seconds (tunable) | N/A (no failover; all active) |
| VM mobility | Gateway MAC changes if host moves across HSRP domain | Same issue as HSRP | Gateway MAC identical everywhere; seamless mobility |
| Traffic path | Must flow to active gateway | Must flow to master | Routed locally at ingress leaf |
| ARP suppression | No native ARP suppression | No native ARP suppression | Native ARP suppression via EVPN Type 2 |
| Appropriate when | Traditional campus/access layer; non-VXLAN network | Multi-vendor traditional network; non-VXLAN | Any VXLAN/EVPN spine-leaf fabric |
| Inappropriate when | VXLAN fabric; high VM mobility; large-scale DC | Same as HSRP | Traditional non-VXLAN routing environments |
HSRP in VXLAN: why it specifically doesn’t work: If you configure HSRP on the SVIs of leaf switches in a VXLAN fabric, hosts connected to the standby leaf will send all inter-VLAN traffic to the active leaf (because HSRP makes only one leaf respond to gateway ARP). That traffic crosses the spine to reach the active HSRP leaf, gets routed, then crosses the spine again to reach the destination leaf. You’re doing the opposite of what the fabric is designed for. Every packet crosses the fabric twice, and the active HSRP leaf becomes a chokepoint. The architecture fundamentally conflicts with distributed forwarding.
11. Multi-Site Anycast Gateway
Anycast gateway scales beyond a single data center. In Multi-Site EVPN deployments (connecting two or more DC fabrics), the anycast gateway concept extends across sites through border gateways. The same gateway IP and vMAC configured at Site-A leaves can also be configured at Site-B leaves, providing gateway continuity when workloads migrate between sites.
BGP EVPN Multi-Site (RFC 8365 / Cisco’s VXLAN EVPN Multi-Site feature) uses Border Gateways (BGWs) at each site to exchange EVPN routes between fabrics over an inter-site network (ISN). When Site-A advertises a Type 2 route for a VM, Site-B’s leaf switches receive it via their BGW and install the MAC/IP into their EVPN database. If that VM migrates to Site-B, the anycast gateway at Site-B answers its ARP immediately — no reconfiguration required.
| Site-A Leaves GW: 192.168.10.1 vMAC: 0200.0000.00aa VNI: 10010 |
↔ | Site-A BGW Border Gateway EVPN route exchange |
↔ | ISN Inter-Site Network |
↔ | Site-B BGW Border Gateway EVPN route exchange |
↔ | Site-B Leaves GW: 192.168.10.1 vMAC: 0200.0000.00aa VNI: 10010 |
| Same gateway IP and vMAC at both sites enables seamless VM mobility across data centers | ||||||||
Stretched L2 vs L3-only Multi-Site: Anycast gateway in Multi-Site can operate in two modes. In “stretched L2” mode, the same L2 VNI spans both sites and the anycast gateway at each site handles local hosts. In “L3-only” mode, each site has its own L2 domain and hosts communicate via L3 (routed through BGW using Type 5 prefix routes). The L3-only model is architecturally cleaner — it prevents broadcast domain issues from spanning sites — but requires applications to tolerate L3 routing between sites. L2 stretching via anycast gateway is necessary for workloads that require the same IP subnet to span both sites (VMware vMotion, stateful live migration).
12. Troubleshooting Anycast Gateway — Commands & Common Issues
Cisco NX-OS Troubleshooting Commands
# Verify anycast gateway is configured on the SVI show interface Vlan10 Vlan10 is up, line protocol is up, autostate disabled Anycast-gateway: Enable ← Anycast gateway active Anycast-gateway MAC: 0200.0000.00aa ← Virtual MAC in use Internet Address is 192.168.10.1/24 Traffic statistics: Unicast packets in/out: 45821/52341 # Verify the fabric forwarding mode setting show running-config interface Vlan10 | include fabric fabric forwarding mode anycast-gateway ← Present = correct # Verify ARP suppression cache (hosts known to this leaf) show ip arp suppression-cache detail vlan 10 # Check EVPN Type 2 route for a specific host show bgp l2vpn evpn route-type 2 0 5254.001a.bc3f 192.168.10.50 # Verify L3 VNI association show nve vni Interface VNI Multicast-group State Mode Type BD Cfg VRF nve1 10010 239.1.1.10 Up CP L2 10 CLI - nve1 10020 239.1.1.20 Up CP L2 20 CLI - nve1 50001 n/a Up CP L3 - CLI PROD ← L3 VNI # Verify traffic flow (debug - use carefully) show system internal evpn event-history anycast
Common Anycast Gateway Issues and Fixes
| Symptom | Likely Cause | Fix |
| Host can ping gateway but not remote hosts | L3 VNI not configured or not associated with VRF. Symmetric IRB not working. | Verify member vni <L3VNI> associate-vrf on NVE. Check show nve vni shows L3 VNI in VRF. |
| VM gets wrong gateway MAC after vMotion | Different anycast gateway MACs on different leaves. MAC mismatch. | Verify fabric forwarding anycast-gateway-mac is IDENTICAL on all leaves. This is the #1 deployment error. |
| ARP floods filling the fabric (BUM traffic high) | ARP suppression not enabled or BGP EVPN Type 2 routes not carrying IP (MAC-only advertisement). | Enable suppress-arp on NVE VNI. Verify hosts are sending ARP Replies (so leaf learns IP) not just Requests. |
| Inter-VLAN routing fails in one direction | Route-target mismatch between VLANs or asymmetric IRB where destination VNI not present on source leaf. | Verify route-target import/export configured correctly in VRF. Use symmetric IRB to avoid VNI presence requirement. |
| Gateway answers but routing is slow/lossy | MTU mismatch. VXLAN adds 50 bytes overhead; fabric MTU not set to 9216. | Set all fabric interfaces (spine-leaf uplinks) to MTU 9216. Verify with ping size 8972 df-bit (1500-byte inner, 8972 outer with headers). |
Anycast Gateway Verification Checklist
1. show interface Vlan<N> — Confirms Anycast-gateway: Enable and vMAC
2. show nve vni — All VNIs up; L3 VNI shows in correct VRF
3. show ip arp suppression-cache detail vlan <N> — Local and remote hosts visible
4. show bgp l2vpn evpn summary — All spines showing prefixes received
5. Ping from host to gateway IP — Should succeed (local leaf answers)
6. Ping from host in VLAN 10 to host in VLAN 20 on different leaf — Should succeed
7. Traceroute from VLAN 10 host to VLAN 20 host — Should show ONE hop (the local gateway IP, not a remote IP)
Frequently Asked Questions
Can anycast gateway and HSRP coexist in the same fabric?
They can coexist in different parts of the network — for example, HSRP in a traditional campus layer feeding into a VXLAN fabric at the distribution layer. But they should never be configured on the same SVI simultaneously, as this creates conflicting gateway behaviors and unpredictable ARP responses. If migrating from HSRP to anycast gateway on an existing network, the transition typically involves disabling HSRP on the SVIs and enabling fabric forwarding mode, which requires a brief traffic disruption per VLAN as the configuration changes. Plan this per-VLAN maintenance window carefully in production.
What happens to traffic if a leaf with an anycast gateway fails?
If a leaf switch fails, hosts on that leaf lose connectivity. This is handled by dual-homing servers: connect each server to two leaf switches (via MLAG or EVPN ESI multihoming). If the server is dual-homed, it fails over to the surviving leaf within sub-second (link detection). The surviving leaf also has the anycast gateway configured — same IP, same vMAC — so from the server’s perspective, the gateway IP is still 192.168.10.1, still at MAC 00:00:00:00:00:01. No ARP refresh is needed. The session resumes through the surviving leaf without the server being aware of the hardware failure.
Why does anycast gateway require the same vMAC on every leaf? Can leaves have different MACs for the same gateway IP?
Different vMACs would break VM mobility. Here’s why: a VM on Leaf-3 ARPs for 192.168.10.1 and caches MAC-A. The VM migrates to Leaf-7. It doesn’t immediately re-ARP (ARP caches last 60–1200 seconds by default on most OS). It sends traffic destined for MAC-A. But Leaf-7’s anycast gateway uses MAC-B. Leaf-7 doesn’t recognize MAC-A as its own gateway MAC — it treats the frame as destined for an unknown MAC and may drop it or flood it. The VM loses connectivity until its ARP cache expires and it re-ARPs. With the same vMAC on every leaf, the VM’s cached entry is always valid regardless of which leaf it migrates to.
Does anycast gateway work with IPv6?
Yes. IPv6 anycast gateway works the same way as IPv4, but using NDP (Neighbor Discovery Protocol) instead of ARP. The leaf switch answers Neighbor Solicitation messages for the gateway’s link-local and global IPv6 addresses with the same virtual MAC. BGP EVPN carries IPv6 MAC/IP bindings in Type 2 routes. NDP suppression on the VTEP prevents Neighbor Solicitation flooding — equivalent to ARP suppression for IPv4. Configuration on Cisco NX-OS uses ipv6 address on the SVI alongside fabric forwarding mode anycast-gateway, and the same anycast-gateway-mac applies to both IPv4 and IPv6 on the same SVI.
How does anycast gateway interact with firewall service chaining in an EVPN fabric?
Anycast gateway routes traffic locally at the ingress leaf by default. If you need all inter-VLAN traffic to pass through a central firewall (for East-West inspection), you need to override the anycast gateway’s local routing decision and redirect traffic to the firewall’s VTEP before final delivery. In Cisco ACI, this is done via Policy-Based Redirect (PBR) using service graphs. In a standard NX-OS/EOS EVPN fabric, you configure the VRF routing table to have a default route or specific routes pointing through the firewall (making the firewall the next-hop for specific prefixes). Alternatively, use a dedicated service VRF: hosts exist in a tenant VRF, traffic exits to a services VRF where the firewall routes it, and then enters the destination VRF. This “VRF leaking via firewall” model is the most common production pattern for E-W firewall inspection in EVPN fabrics.
Anycast Gateway — Key Concepts Summary
| Distributed routing | Every leaf routes locally for its attached hosts. No centralized gateway bottleneck. One fabric crossing for inter-VLAN traffic. |
| Same IP + Same MAC | 192.168.10.1 and 0200.0000.00aa configured identically on every leaf. Hosts ARP once; their cache stays valid across any VM migration. |
| vMAC not in overlay | The virtual MAC is access-side only. EVPN Type 2 routes carry endpoint MACs and physical VTEP router-MACs, not the vMAC. No duplicate MAC conflict in the overlay. |
| ARP suppression | EVPN Type 2 routes build a MAC/IP suppression database at every VTEP. Gateway ARPs and known-host ARPs are answered locally. BUM traffic is nearly eliminated. |
| Symmetric IRB required | Use symmetric IRB with a dedicated L3 VNI per VRF. Asymmetric IRB requires all L2 VNIs on all leaves and doesn’t scale. Symmetric IRB is the production standard. |
