Best Effort Fibre Channel

There’s a common misconception about Fibre Channel, in that it only operates in a lossless mode. While it’s true that for many applications, Fibre Channel is used in a totally lossless way (commonly through buffer to buffer credits and other less used mechanisms). But there is a class of service supported by most Fibre Channel switch vendors and HBAs that does allow for less care (and thus dropping) of Fibre Channel frames: Best Effort (Class 9).

There are several classes of service in Fibre Channel. All but one of them is lossless. Most of the time, Fibre Channel runs in Class 3, which provides losslessness through a buffer credit mechanism. Here are all the Fibre Channel classes of service, (although typically just Class 3, 6, and 9 are supported in modern switches):

• Class 1: Dedicated bandwidth, acknowledged end-to-end
• Class 2: Shared bandwidth, acknowledged end-to-end
• Class 3: Shared bandwidth, unacknowledged (lossless handled on a port-to-port basis)
• Class 4: Virtual circuits, dedicated bandwidth per VC, can be acknowledged or unacknowledged
• Class 5: Undefined
• Class 6: Multicast (yuck)
• Class 9: FC_BE (Best Effort). Shared bandwidth, unacknowledged, lossfull (only lossfull method)

Fibre Channel is usually lossless because it’s running the SCSI protocol as the upper layer protocol, and there’s a widely (but somewhat erroneous) belief that SCSI doesn’t tolerate lost commands well. For some applications, that’s true, loss of a SCSI command is bad. If you’re running a financial institution with an Oracle database or FICON connected mainframe to keep track of financial records, then yes, you’re going to want to run a supported lossless method (typically Class 3).

But there are situations where running Class 9 best effort Fibre Channel may be beneficial. If you’re running a web site with a message board or a Windows server, your SCSI requirements aren’t all that tough. You can lose a SCSI command here or there, and either the application will recover (loading up Battlefield 3) or you just don’t care (troll on a message board).

Class 9 is relatively new, only having been ratified by the T11 working group (under P9FOS committee) in 2005. However since it was designed specifically for existing hardware, only a software update is needed to support it, so most switch and HBA firmware from the major vendors (Cisco, Brocade, Emulex, QLogic, etc.) support it. The idea for a class of lossfull service was in fact inspired by Ethernet.

Fibre Channel, where did you learn to drop frames?

From you, Ethernet, all right? I learned it from watching you!

Turning on Fibre Channel Class 9 (FC_BE, Best Effort) is easy on a Cisco MDS:

switch# conf t 
switch(config)# no system default  
switch(config)# switchport mode F 
switch(config)# switchport class 9 
switch(config)# switchport buffer wred

The mode F turns the port into an F_Port (for an N_Port to plug into), and class 9 makes it class 9. The last command is an important one that most people forget: Turning on WRED (Weighted Random Early Detection). This is a buffer management mechanism that drops frames randomly when a buffer gets close to full. You might be familiar with this mechanism in Ethernet ports as a method to prevent tail drop (leaky bucket) so that TCP performance doesn’t suffer. We’re obviously not running TCP (although you could with IPFC), but tail drop is still bad performance-wise with SCSI.

So why would you wanto to turn on Class 9? Performance mostly. You can get a serious case of HoL (Head of Line) blocking because of congestion somewhere else in the Fabric. Class 9 enables the FIWDI (Fuck It, We’ll Drop It) mechanism when congestion occurs in a fabric.

Congestion on the fabric? Fuck it, we’ll drop the frame!

Note: So as not to pollute Google results, this was of course an April Fools joke. There’s not way anyone who loves Fibre Channel would allow lossfullness.