Saturday, July 15, 2017

A Story About Knowledge

I worked as a lab assistant while an undergrad Physics major at a small college in Pennsylvania. My duties involved setting up experiments in lab rooms. Some of the experiments required electricity, which I was not that comfortable with at the time (read: terrified). A professor told me that if I learned the proper respect for electricity, all would be well. I managed to not electrocute myself over the next few years.

In the early 1990s I was accepted into the PhD program in the department of Physics at Rensselaer Polytechnic Institute. I attended on an assistantship, so I was again required to help setup labs. One class I worked in was taught by Wayne Roberge, a PhD in astrophysics from Harvard.

For one lecture session I had to setup a demonstration on electricity and magnetism that required a car battery for a power source. As I was setting up the equipment Dr. Roberge watched with concern. He looked at me and said, "You're being pretty cavalier with that battery." I asked what he meant by that. He said that the battery was dangerous, and I should be more careful with it.

"Why do you think it is dangerous?," I asked. "Well, it can produce 500 amps!," he replied. He may have been referring to the old saying "volts jolts, amps kill," an oversimplification of the fact that only small amounts of electrical current are required to stop a human heart.

At this point, I had the car battery on a table, with the terminals exposed. I grabbed the terminals, one in each hand. Dr. Roberge's eyes looked on in horror, as he thought I was about to die. Surprisingly to him, but not to me, nothing happened. I then explained to Dr. Roberge: the 12 volts of "push" supplied by a car battery is not enough to push current through dry human skin. Even if it could, the human body acts a a capacitor, which will not conduct direct current well.

Here's a guy with a PhD in Astrophysics from Harvard who didn't have a practical working knowledge of everyday electricity. Does that make him dumb, or does it make me smarter than him? No way.

The moral of the story: there are differing kinds of knowledge out there. Theoretical and practical are two kinds, but there are more. All have their place, and all have benefit. Because you think that, in theory, something is bad, doesn't necessarily mean that the reality is bad. When you look at what other people do and think, "that's dumb, I would never do that," consider that you may not have all the information that person had when that decision was made.

PLUR,

Mark

Thursday, July 6, 2017

Cisco WLC Fastlane for iOS - What it Does

If  you run a Cisco wireless network, chances are you have heard about Fastlane for iOS. Introduced with 8.3 firmware, Fastlane is a set of configuration changes that tune the wireless network for iOS 10 devices. It is part of a suite of features that resulted from the Cisco and Apple partnership that includes Adaptive 802.11r, and robust 11k/v support on iOS devices. This partnership came about because of the increase in the use of mission-critical applications like Jabber and Citrix on iOS 10 devices.

The main idea of Fastlane is to allow certain apps on an iOS device to send traffic with Voice Access Category (AC) on a network with Call Admission Control without having to use TSPEC. If you have worked on Cisco wireless networks that support phones like the 7925 or 8821, you may be familiar with TSPEC, which is a method that wireless devices use to reserve bandwidth when accessing the Voice AC on a network where Admission Control Mandatory (a.k.a CAC) is enabled. Fastlane is basically a hack that allows iOS devices to access the Voice AC in a non-standard way. In addition, the APs and upstream WLC will preserve any inner DSCP markings on IP packets from iOS devices. This ensures that CAPWAP tunnels packets between the AP and controller are marked with the appropriate DSCP values across the wired portion of the network.

Fastlane configuration is focused on tuning QoS settings for a WLAN, but there are also changes to the entire wireless network. Here is an overview of what happens when you enable Fastlane on a WLAN.
  • The QoS profile for the WLAN is set to Platinum. Remember that the QoS profile acts as a limiter on the QoS markings upstream and downstream between the controller and the APs. To preserve Voice DSCP markings, the WLAN's QoS profile must be set to Platinum. 
  • Enables AVC on the WLAN, and maps an automatically-created AVC profile to the WLAN. More on this later. 
  • Enables Admission Control Mandatory for the Voice AC on both 802.11a and 802.11b networks. Load-based CAC is selected, with maximum bandwidth for Voice set at 50%. This differs from the default value of 75%, which is the recommendation when configuring for Cisco phones. Expedited Bandwidth is also enabled. 
  • Enables QoS Map, Trust DSCP upstream, and creates a DSCP to UP exception map. These are global settings. 
  • Changes the EDCA profile on both 802.11b and 802.11a networks to a built-in profile called Fastlane. More on this later. 
Making all these changes requires disabling both bands, so it is quite disruptive. Plan accordingly. 

For all of this to work, there has to be a way for iOS devices to tell Cisco APs that, well, they are iOS devices. This is accomplished through a tagged Information Element that is included in probe and association requests. 
Apple iOS IE

Conversely, the WLAN needs to let the iOS device know that Fastlane is enabled. This is also accomplished with a tagged IE in probe and association responses, and also beacons. 

IE that tells iOS devices Fastlane is enabled
Note that this IE decodes as Aironet, and appears in beacons even if Aironet IE is disabled on the WLAN. Apparently the value for "Aironet IE data" varies, but the OUI Type is consistent. 

Now I will expand on each of the items that Fastlane configures, starting with the AVC profile. When you enable Fastlane on a WLAN, a new AVC profile called AUTOQOS-AVC-PROFILE is created and mapped to the WLAN. This profile maps DSCP settings to well-known applications used on iOS devices. The details of the AVC profile can be found here, but a picture is worth a bunch of words.


There are many more rules in this set, some of which punish apps like Netflix and BitTorrent with very low DSCP values.

Next in the list is Call Admission Control for Voice. Enabling CAC for the Voice AC will limit client access to the Voice AC to devices that support TSPEC and iOS devices that support Fastlane. Load-based CAC is enabled for Voice on both bands, which is a little suspect. Load-based CAC will reject a lot of calls on 2.4 GHz due to the normally higher channel utilization values. I recommend being careful here if you have voice devices on 2.4 GHz that support TSPEC.

A notable difference between Fastlane CAC and the recommended CAC settings for Cisco voice is the max bandwidth setting. The recommended max bandwidth percentage for Cisco voice is 75%, and Fastlane sets it at 50%. This means that each radio will be able to admit fewer TSPEC calls. It's hard to pin an exact number on how many fewer, however.

Another CAC feature that is enabled with Fastlane is Expedited Bandwidth. Expedited Bandwidth allows a radio to admit a TSPEC call from a CCXv5-capable client that indicates the call is urgent, even if there wasn't bandwidth available to admit the call. Use of this feature requires that the Call Manager be configured appropriately to mark certain calls, like 911 calls, as urgent.

Why don't we take a 5 minute break? 

Fastlane also makes changes to global QoS settings. The QoS Map setting is set to Enabled. This adds a tagged element to Association/Re-association Response frames that tells clients what UP values should be used for sending IP traffic tagged with a certain DSCP value. Below is an image which shows the configuration of the map on the WLC, with the resulting elements included in the Association Response.

QoS Mapping
In addition to the QoS map, there is an exception list, which maps certain DSCP values to UP values outside of the ranges defined in the map. Take a close look at the map above; DSCP 46, normally used for voice packets, is mapped to UP 5, which is classified for video.  The DSCP exception list fixes this, as well as adjusting other values. Below is an image showing part of the exception list, along with the tagged element that appears in the Association Response as a result.

DSCP Exception List

Trust DSCP UpStream is also enabled. This allows the AP to copy the DSCP marking of a packet sent from a wireless client to the DSCP of the CAPWAP tunneled packet towards the WLC. At the WLC, the CAPWAP header is stripped off and the original IP packet is sent on the wire, using the original DSCP marking. For a detailed discussion of Trust DSCP UpStream, I highly recommend the BRKEWN-2000 session presentation (login required) from Cisco Live Berlin 2017, from the one and only Jerome Henry.

Remember that the AVC profile above will also be manipulating DSCP markings. In the downstream direction, when a packet arrives at the AP and is queued to be sent to a client, the AVC profile is applied first, and then the manipulated DSCP value is converted to a UP value through the DSCP exception table. For example, if a Cisco Jabber audio packet arrived at the WLC with a DSCP value of 0x0, the AUTOQOS-AVC-PROFILE would re-write the DSCP value to 46. The DSCP exception table would convert that DSCP value of 46 to a UP value of 6. In turn, the UP of 6 would became an Access Category of 3 (Voice).

The final and perhaps most impactful change Fastlane makes is setting the EDCA profile on both 802.11a and 802.11b networks to the Fastlane built-in profile. If you are unfamiliar with Wireless QoS and the role of EDCA, I recommend Andrew von Nagy's 5-part Wireless QoS series

In short, EDCA parameters determine how long a client must wait to transmit a frame depending on Access Category. Voice AC frames should wait less for access to the wireless medium than Best Effort in order for Voice frames to have a better chance at getting transmitted. The Fastlane EDCA profile sets parameters according to the latest recommendations from the IEEE. Below is a comparison of the EDCA parameters between the default (WMM), Cisco's Voice Optimized, and Fastlane.
EDCA parameters compared
There are two things that stand out to me. First, there is a big difference between Voice Optimized and Fastlane. The Voice Optimized profile is highly biased towards voice traffic, based on the values of AIFSN and CWMIN. If you are currently using the Voice Optimized profile (recommended by Cisco if you are using Cisco wireless phones), changing to Fastlane could potentially have a negative effect on your voice applications.

The second difference is the change in TXOP values from the default WMM profile and Fastlane. The TXOP value specifies a limit, in intervals of 32µs, that a client can hold access to the medium for. Why would you want to limit the amount of time, and is that necessary? It's necessary because of the possibility of large aggregated A-MPDU frames with 802.11ac clients (up to 4 MB!). Without a TXOP limit, once a client gains access to the medium it can send a single gigantic A-MPDU. If that client is at the edge of the cell, that could result in the client sending that big frame at a low data rate, consuming lots of airtime. Setting the TXOP limit requires the client to size its A-MPDU frames small enough that they can be transmitted, including the following SIFS and Block Ack, within the allotted time. This means that low data rate clients cannot monopolize airtime while sending large frames.

What impact will setting a TXOP limit for BE have on your network? In theory, TXOP limits result in smaller A-MPDUs, which means more contention for access to the medium for clients trying to transmit large amounts of data. It also means less efficiency, since there will be more airtime used for headers and block acks. In a very unscientific experiment, I transferred a 250 MB file from a laptop to a network share over a wireless network (clean channel 100, 20 MHz width, 2 spatial streams, all HT and VHT rates enabled). First I used the default WMM EDCA profile, which does not set a TXOP limit for Best Effort traffic. I then used the Fastlane EDCA profile,which specifies a TXOP limit of 79 for BE (about 2.5 ms).

With no TXOP limit, I saw A-MPDU frames with up to 60 sub-frames, with total size close to 100KB. With TXOP limit of 79, most A-MPDUs had 30 sub-frames, with total size of only about 45KB.

To compare efficiency, I graphed the number of Block Acks per second sent from the AP to the client. Here's what that looked like.

Restricting the A-MPDU size clearly results in more overhead in Block Acks. It's also clear from the graph that the data transfer without a TXOP limit finished a couple of seconds faster. We can also look at channel utilization by looking at the QBSS element in the beacons while the data transfer took place.

Channel Utilization Compared for TXOP Values
Without TXOP, peak channel utilization is a bit higher, but takes longer to ramp-up. With the TXOP limit of 79, peak utilization is lower, but steady.

My purpose with this blog was to detail all of the changes that enabling Fastlane will make to your wireless network, and what the impact of those changes might be to non-iOS devices. Hopefully the reader has enough information on Fastlane to make an informed choice on whether to enable it on a wireless network they manage.