Cisco ACE 101: Tony’s 5 Steps to a Happy VIP
August 14, 2012 15 Comments
I’ve been teaching Cisco ACE for over four years now, and I developed a quick trick/check list to teach students the minimum configuration to get a virtual service (VIP) up and running. And since the CCIE Data Center lab will soon be upon us, I’m sharing this little trick with you. I call it “Tony’s 5 Steps to a Happy VIP”. And here it is:
Step #1: ACL
Step #2: class-map: Defines the VIP address and port
Step #3: policy-map: Which server farm(s) do we send traffic to
Step #4: policy-map: Multi-match, will pair every class-map to its policy-map
Step #5: service-policy: Apply step #4 to the VLAN interface
Using that checklist, you can quickly troubleshoot/understand most ACE configurations. So what does that list mean?
First off, let’s define what a VIP even is: In load balancing terms, it refers to an IP and TCP or UDP port combination. In that regard, it’s a bit of a misnomer, since VIP is an acronym for “Virtual IP”, and only implies an IP address. Depending on the vendor, a VIP can be called a “Virtual Server”, “Virtual Service”, although it’s commonly referred to simply as “VIP”. It’s whatever you point the firehouse of network traffic to.
I’m not anti-GUI (in fact, I think the GUI is increasingly necessary in the network world), but in the case of the ACE (and CCIE DC) you’re going to want to use the CLI. It’s just faster, and you’re going to feel the need for speed in that 8 hour window. Also, when things go wrong, the CLI (and config file) is going to allow you to troubleshoot much more quickly than the GUI in the case of the ACE.
The CLI for Cisco ACE can be a little overwhelming. For some reason, Cisco decided to use the Modular QoS CLI (MQC) configuration framework. To me, it seems overly complicated. Other vendors have CLIs that tend to make a lot more sense, or at least is a lot easier to parse with your eyes. If you’re familiar with class-maps, policy-maps, and service-policies, the transition to the ACE CLI won’t be all that difficult. It works very similar to setting up QoS. However, if you’re new to MQC, it’s going to be a bit of a bumpy ride.
How I felt learning MQC for the first time
The Configuration
Here is a very basic configuration for an ACE:
access-list ANYANY line 10 extended permit ip any any rserver host SERVER1 ip address 192.168.10.100 inservice rserver host SERVER2 ip address 192.168.10.101 inservice rserver host SERVER3 ip address 192.168.10.101 inservice serverfarm host SERVERFARM1 rserver SERVER1 inservice rserver SERVER2 inservice rserver SERVER3 inservice class-map match-all VIP1-80 2 match virtual-address 192.168.1.200 tcp eq http class-map match-all VIP1-443 2 match virtual-address 192.168.1.200 tcp eq https policy-map type loadbalance first-match VIP1-POLICY class class-default serverfarm SERVERFARM1 policy-map multi-match CLIENT-VIPS class VIP1-80 loadbalance vip inservice loadbalance policy VIP1-POLICY class VIP1-443 loadbalance vip inservice loadbalance policy VIP1-POLICY interface vlan 200 description Client-facing interface ip address 192.168.1.10 255.255.255.0 access-group input ANYANY service-policy input CLIENT-VIPS no shutdown interface vlan 100 description Server VLAN ip address 192.168.10.1 255.255.255.0 no shutdown
Step #1: ACL
It’s not necessarily part of the VIP setup, but you do need to have an ACL rule in before a VIP will work. The reason is that the ACE, unlike most load balancers, is deny all by default. Without an ACL you can’t pass any traffic through the ACE. (However, ACLs have no effect on traffic to the ACE for management.)
Many an ACE configuration problem has been caused by forgetting to put an ACL rule in. My recommendation? Even if you plan on using specific ACLs, start out with an “any/any” rule.
access-list ANYANY line 10 extended permit ip any any
And don’t forget to put them on the interface facing the client (outside VLAN).
interface vlan 200
description Client-facing interface
ip address 192.168.1.10 255.255.255.0
access-group ANYANY input
service-policy input CLIENT-VIPS
no shutdown
Once you get everything working, then you can make a more nailed-down ACL if required, although most don’t since there is likely a firewall in place anyway (even the Cisco example configurations typically only have an any-any rule in place).
If you do use a more specific ACL, it’s often a good idea to switch back to any-any for troubleshooting. Put the more specific rule in place only when you’re sure your config works.
Step #2: class-map (VIP declaration)
The next step is to create a class-map that will catch traffic destined for the VIP. You should always include an IP address as well as a single TCP or UDP port. I’ve seen configurations that match any TCP/UDP port on a specific IP address, and this is usually a really, really bad idea.
class-map match-all VIP1-80 2 match virtual-address 192.168.1.200 tcp eq http
This defines a VIP with an address of 192.168.1.200 on port http (port 80). Even if you set up multiple ports on the same IP address, such as port 80 and 443, use different class-maps and configure them separately.
Step #3: policy-map (what do we do with traffic hitting the VIP)
Here is where the VIP is defined as either a Layer 4 VIP or a Layer 7 VIP. The example below is a simple Layer 4 VIP (the ACE is not aware of anything that happens above Layer 4). You can get a lot fancier in this section, such as sending certain matched traffic to one server farm, and other traffic to others, and/or setting up persistence. Again, this is the most basic configuration.
policy-map type loadbalance first-match VIP1-POLICY class class-default <-- This matches everything serverfarm SERVERFARM1 <-- And sends it all right here
Step #4: policy-map (round-up policy-map, pairs a VIP with a decision process, and all the pairs are joined into a single statement)
You will typically have multiple Step 2’s and Step 3’s, but they exist as independent declarations so you’ll need something to round them all up into a single place and join them. In most configurations, you will typically only have one multi-match policy-map. This multi-match is where you marry a Step 2 class-map to a Step 3 policy-map. In this example, two separate class-maps use the same policy-map (which is fine).
policy-map multi-match CLIENT-VIPS class VIP1-80 <-- This VIP... loadbalance vip inservice loadbalance policy VIP1-POLICY <-- ...sends traffic to this policy class VIP1-443 <-- This VIP... loadbalance vip inservice loadbalance policy VIP1-POLICY <-- ...sends traffic to this policy
Step #5: service-policy (apply the round-up to the client-facing interface)
interface vlan 200 description Client-facing interface ip address 192.168.1.10 255.255.255.0 access-group input ANYANY <-- Step 1's ACL is applied service-policy input CLIENT-VIPS <-- Step 5's multi-match policy map is applied no shutdown <-- Don't forget the no shut!
Hope this helps with demystifying the ACE configuration. A short little check list can really help save time, especially in a time-constrained environment like a CCIE lab.
While I agree with you and always specify exact TCP/UDP ports in my class maps, can you explain why this it’s a very bad idea to use any port?
Sure thing. Basically, if you let *any* port get load balanced, any port *does*. Like NetBIOS, IMAP, SMTP, probably ports you don’t want exposed to the public network. Hopefully a firewall would protect all that, but by specifying the port you’re assuring that it won’t happen.
Funny, I was reading the intro and thought to myself, “Why is Tony talking about QoS?” Except for a repeated policy-map, your 5 steps read an awful lot like the Modular QoS CLI (MQC) on IOS. For those of us R&S nerds, I think that will be a good thing. Provided we can remember the extra policy-map steps.
Yeah, the MQC was done on purpose, but I have no idea why. For people coming from the F5 world (which has more than double ACE’s market share) MQC is an alien language, and the barrier to entry is much higher than F5’s config.
If you’re coming from the R&S world, the learning curve is a lot lower of course (I did not come from the R&S world, so I’m still a bit bitter :))
That said, ACE’s GUI has gotten better (though still not as good as F5’s), but there are some things you’ll still need the CLI for.
Nice post Tony – but I, being a pedantic bastard, am going to suggest you add a few more steps:
1. configure the management policy and apply it to the inbound interface – you may not be able to assume that the default policy has been applied in the CCIE exam.
2. configure the server vlan ip address – inteface vlan 400 in your example
3. configure some rservers
4. put the rservers in a serverfarm. Your config seems to have magically found a serverfarm called SERVERFARM1 with some rservers in it. Wouldn’t count on that in the exam.
5. I think you should also create a VIP for the https before you use it in the round-up policy map (class VIP1-443)
6. And just to show how REALLY pedantic I am, I’d suggest you apply the access group with the command:
access-group input ANYANY
rather than
access-group ANYANY input
CW
Thanks Chris. Yeah, this is just a quick checklist for VIP creation/troubleshooting. It includes only the policy-maps and other MQC bits because that’s the part that’s unlike any other load balancer. Serverfarms and real servers etc. have analogs in all of the other LB products out there. And i needed to keep the list to around 5 🙂
You did catch my wrong order of my access-group declarations, so I added it.
I also did create a VIP for the HTTPS, just didn’t mention it in step 2.
I would also add “loadbalance vip icmp-reply” for troubleshooting to make sure VIP is actually operational from outside world.
You know, I never use ICMP for troubleshooting VIPs. I always use Telnet by telneting to the IP:port. Doing that, Telnet becomes a generic TCP client. I can also run things like “GET /” to test (in a rudimentary way) all the way through Layer 7.
Krunal, I actually use the following:
loadbalance vip icmp-reply active
Without “active” the VIP will reply to ping regardless of the state of the serverfarm. With “active” it will only reply if one or more of the rservers are passing their health check. It depends what you’re trying to achieve.
Tony, it’s a fairly common go-to test for other people outside of the networks area, especially if you’re load balancing for a website and while it’s certainly not needed I’ve found it can be useful to have it enabled. Probably more of an ACE 102 extra rather than an ACE 101 essential, I guess?
Hi Tony,
Great post. One question: how can I get rid of “Error: Entered VIP address is not the first address in the VIP range” when creating the VIP?
I’ve checked is a valid IP for the VLAN, in the correct range and is not used by any interface in the ACE (which is the reason stated in the manual for this error)
Thanks!!
hi, right now i’m working with an ace module using contexts and i try to create a VIP without using NAT within the class under de multi-match but it does not work for me, can you tell me if i’m missing something? please.
Hey Tony,
Thanks very much this helps a lot, being use to the Arrowpoint 11800, and CSS 11506’s.
Hi Tony
Does this mean I don’t have to declare nat-pool and how will my midtier servers (remedy web frontend) now how to route back to clients?
You do need a nat-pool if you’re doing SNAT, which is typically the case if the ACE isn’t the default gateway.
…sorry i mean, route back to clients, through the ACE interface 🙂