It’s All About The IOPS, Baby

“It’s all about the benjamins, baby” – Puff Daddy

One of the new terms network administrators are starting to wrap our brains around is the term IOPS, or I/O Operations Per Second.  Traditionally, IOPS have been of the utmost importance to data base administrators, but haven’t shown up on our radar as network admins.

So what is an IOP? It’s a transaction with the disk. How long does it take a storage client to send a command to a storage operation (write or read) and receive the information or confirmation the data has been written.  Applications like databases, desktops, and virtual desktops are fairly sensitive to IOPS.  You know your laptop is suffering from deficient IOPS when you hear the grinding noise of the laptop when you open up too many VMs or too many apps simultaneously.

Traditional hard drives (spinning rust as Ivan Pepelnjak likes to call them) have varying degrees of IOPS. Your average 7200 RPM SATA drive can perform roughly 80 IOPS. A really fast 15,000 RPM SAS drive can do around 180 IOPS (figures are from Wikipedia, and need citation). Of course, it all depends on you measure an IOP (reads, writes, how big is the read or write), but those are general figures it keep in mind.  One IO operation make take place on one part of a disk, and the next IO operating may take place on a completely different.

That may sound like not a lot, and you’re right, it isn’t.  For databases and virtual desktops, we tend to need lots of IOPS. IOPS scale almost linearly by adding more disks. Mo spindles, mo IOPS. That’s one of the primary purposes of SANs, to provide access to lots and lots of disks. Some servers will let you cram up to 16 drives in them, although most have space for 4 or 5. A storage array on a SAN can provide access to a LUN comprised of hundreds or even thousands of disks.

Single hard drives are pretty good at sequential reading and writing, which is different than IOPs. If you have a DVR at home, it depends mostly on sequential reading and writing.  A 7200 RPM can do about 50-60 Megabytes/second of sequential reads, less in writes. And that’s the absolute best case scenario: A single file located in a contiguous space on the platter (so the spindle doesn’t have to bounce around the platter getting bits of data here and there).  If the file is discontiguous, or lots of smaller files all located in various areas of the platter, the numbers will go down significantly.

What does that mean to network admins? A single 7200 RPM SATA drive can push about 400-500 Megabits per second on your network.

Bandwidth

In networking, we tend to think in terms of bandwidth.  The more the better.  We also think of things like latency and congestion, but bandwidth is usually on our minds. In storage, it’s not so much bandwidth as IOPS.

Let’s say I tell you that I have two different storage arrays for you to choose from.  One of the arrays is accessible via a 1 Gbit end-to-end FC connection.  The other array is connected via accessible via an 8 Gbit end-to-end FC connection. You know nothing of how the arrays are configured, or how many drives are in them. They both present to you a drive (LUN) with a 2 Terabyte capacity.

Which one do you chose?

The networkers in us would tend to go for the 8Gbit FC connected array.  After all, 8 Gbit FC is 8 times faster than 1 Gbit FC.  There’s no arguing that.

But is it the right choice? Maybe not.

Now you learn a little more about the array.  The LUN in the 1 Gbit storage array is comprised of 20 drives in a RAID 10 setup with 100 IOPS per drive.   That’s about 1,000 total IOPS.

The 8 Gbit FC connected array is comprised of (2) two terabyte drives at 100 IOPS per drive in a RAID 1 array (two drives, mirrored).  That’s 100 IOPs.

So while the 8 Gbit array has 8 times the bandwidth, the 1 Gbit array has 10 times the IOPS.  Also, the (2) two terabytes drives aren’t going to be able to push 800 Megabytes per second that 8 Gbit FC would provide.  They’d be lucky to push 80 Megabytes.

If you’re using those LUNs for databases or virtualization (especially desktop virtualization), you’re going to want IOPS.

Operating systems love IOPS, especially desktop operating systems. Chances are the slowest part of your computer is your spinning rust. When your computer slows to a crawl, you’ll typically hear the hard drive grinding away.

Solid State Drives

SSDs have changed the game in terms of performance. While your highest performing enterprise spinning disk can do about 180 IOPS, the SSD you get from Best Buy can do over 5,000.  (By the way, I highly recommend your next laptop have an SSD drive).

There are hybrid solutions that use combinations of SATA, SSDs, and battery-backed up RAM to provide a good mix of IOPS and economy.  There are a lot of other interesting solutions too to provide access to lots of IOPS.