Cisco ACI 3.0: Multisite fabric

Today I am going to talk about Cisco ACI 3.0 Multisite Fabric infrastructure and the components involved. Cisco ACI is one of the demanding technology in the market for data center policy based environment infrastructure.

Cisco ACI Multi-Site is an architectural approach for interconnecting and managing multiple sites, each serving as a single fabric and availability zone.

Fig 1.1- Cisco ACI Multisite Fabric

Multi-Site architecture has three main functional components:

• Two or more ACI fabrics built with Nexus 9000 switches deployed as leaf and spine nodes.
• One APIC cluster domain in each fabric.
• An inter-site policy manager, named Cisco ACI Multi-Site, which is used to manage the different fabrics and to define inter-site policies.

Multi-Site has the following benefits:

• Complementary with Cisco APIC, in Multi-Site each site is an availability zone (APIC cluster domain), which can be configured to be a shared or isolated change-control zone.
• MP-BGP EVPN is used as the control plane between sites, with data-plane VXLAN encapsulation across sites.
• The Multi-Site solution enables extending the policy domain end-to-end across fabrics. You can create policies in the Multi-Site GUI and push them to all sites or selected sites. Alternatively, you can import tenants and their policies from a single site and deploy them on other sites.
• Multi-Site enables a global view of site health.
• From the GUI of the Multi-Site Policy Manager, you can launch site APICs.
• Cross-site namespace normalization is performed by the connecting spine switches. This function requires Cisco Nexus 9000 Series switches with "EX" on the end of the name, or newer.
• ACI Multi-Zone pushes cross-fabric configuration to multiple APIC clusters providing scoping of all configuration changes
• Disaster recovery scenarios offering IP mobility across sites is one of the typical Multi-Site use cases.

Fig 1.2- Cisco ACI Multisite Multi-Zone

Hardware Requirement

• Support all ACI leaf switches (1st Generation, -EX and -FX)
• Only -EX spine nodes (or newer) to connect to the inter-site network
• New FX non modular spine (9364C, 64x40G/100G ports) will be supported in Q1CY18 timeframe
• 1st generation spines (including 9336PQ) not supported. Leverage those for intra-site leaf to leaf communication

Multi-Site considerations and restrictions:

• Multisite policy manager is a set of VM running concurrently on a hypervisor. At the time of this release it supports only vSphere
• The multisite policy manager just needs OOB connectivity to APIC clusters. Supports up to 500 msec to 1 sec RTT
• Recommended to deploy ACI Multi-Zone for a single ACI site to plan for a future Multi-Site deployment
• Hypervisors can be connected directly to the DC OOB network with each ACI Multi-zone VM having a unique routable IP