Cisco ASR 1000: CEF load Balancing Issue and Possible Solution
Today I am going to talk about one of the issue faced on the Cisco ASR 1000 where we are trying to load balance the traffic. I will talk about the requirement and what is the problem we are facing and then we will discuss on the discussion point and the possible solutions for the same.
Cisco Express Forwarding load balancing is based on a combination of source and destination packet information; it allows you to optimize resources by distributing traffic over multiple paths. We can configure load balancing on a per-destination or per-packet basis. Because load-balancing decisions are made on the outbound interface, load balancing must be configured on the outbound interface.
 |
| Fig 1.1- CEF Load Balancing issue on ASR 1000 Series router |
Per-Destination Load balancing
Per-destination load balancing is enabled by default when you enable Cisco Express Forwarding. To use per-destination load balancing, you do not perform any additional tasks once Cisco Express Forwarding is enabled. Per-destination is the load-balancing method of choice for most situations.
Because per-destination load balancing depends on the statistical distribution of traffic, load sharing becomes more effective as the number of source-destination host pairs increases.
Per Packet Load Balancing
Per-packet load balancing is good for single-path destinations, but packets for a given source-destination host pair might take different paths. Per-packet load balancing can therefore introduce reordering of packets. This type of load balancing is inappropriate for certain types of data traffic (such as voice traffic over IP) that depend on packets arriving at the destination in sequence.
Use per-packet load balancing to help ensure that a path for a single source-destination host pair does not get overloaded. If the bulk of the data passing through parallel links is for a single pair, per-destination load balancing overloads a single link while other links have very little traffic. Enabling per-packet load balancing allows you to use alternate paths to the same busy destination.
Let's start with the requirement first and then we can talk about the problem faced.
Requirement
ASR 1000 has two physical connections that can transport subscriber sessions. Both links are intended to be active. Each subscriber session may have input and output QoS service-policies. The policy-maps are pre-defined on the router and the activation is controlled by radius. The policy-maps only have police actions.
Problem
We intend to use ip load-sharing to split each subscriber's traffic between the two links. Specifically, each subscriber's traffic will be split based on destination address using 'ip load-sharing per-destination'. This configuration is conflicting with the QoS service-policies on the subscribers. The router throws an error message and the load-balancing is reverted to a single path for the sessions.
Explanation
QoS service-policies require all that all traffic transits a single forwarding interface. The conflict is occurring because per-packet load-sharing associates multiple forwarding interfaces with the subscriber sessions.
The QoS service-policy restriction exists because QoS queuing features (bandwidth, shape, fair-queue, queue-limit, priority and random-detect) need all the traffic to go through the same scheduler.
Schedulers and queues are almost always associated with ports, as is the case with ASR 1000, so the requirement for a common scheduler translates to a requirement for a common forwarding interface.
For example, to shape a session to 10 Mbps, all of the session traffic must go through a single queue with a scheduler set to drain it at the shape rate. If it ends up in multiple queues then it could exceed the shape rate (e.g, 10 Mbps rate applied to two forwarding interfaces resulting in 20 Mbps being sent) or fail to meet the shape rate (e.g., 10 Mbps divided into 5 Mbps per forwarding interface, but only one forwarding interface is active for the traffic due to, say, sharing the same destination address).
Some QoS features, such as police and marking actions, do not use schedulers and, thus, are not restricted to a single forwarding interface. The restriction is applied to the entire policy type rather than to individual actions to make the checking simpler and more reliable. Also, the use of queuing features in such configurations is very common.
Possible Solutions
- Load-balance at the subscriber level such that each subscriber session is assigned a single forwarding interface. For example, load-balancing could be set up per VLAN. This would entail a change to the network architecture.
- Only police in the input direction. Queuing features may only be applied to output service-policies, so the restriction is not present for input policies.
- Relax the restriction to allow service-policies without queuing features to be applied. This work will need to be scoped, and will be moderately complex since we need to carefully monitor changes to forwarding interfaces, service-policies and policy-maps, as well as having to inspect the policy-maps more closely. From an operational perspective, it will be important with this solution to control the definitions of the policy-maps on the router.
