Flash-Based SAN Storage and the Need for Speed

Brief SAN Storage History

Flash-based SAN storageStorage Area Network systems or SANs as they are usually referred to, started out from the need to share storage between two or more servers. This was initially done to achieve high availability. A SAN employs a method of connecting servers through a switching network (Ethernet or Fiber Channel), and the servers connect to the SAN switches using Host Bus Adaptors or Ethernet cards. The heart of the system is called the storage or disk array, which is used for block-based, file-based or object storage. The disk drives contained in the storage array are almost always the bottleneck in a SAN.

Storage arrays started as simple extensions of the technology that was available to servers when SANs were first being developed. Redundant Array of Independent Disks (RAID) combines multiple physical drives into one or more logical units for data redundancy and performance improvement. The SCSI or fiber channel-based disks that made up the arrays were the largest and fastest hard disk drives (HDDs) around at the time. In the early days of SAN technology, all vendors had access to the same brands and speeds of drives. SAN array vendors eventually started to differentiate from competitors by adding RAM cache in front of RAID arrays to achieve faster response times. Later, it was discovered that software would become the innovation factor for array vendors. Anyone can put together a RAID array with cache, but progressive vendors began using proprietary software to speed up the delivery of data to and from the network. A leader in the software side of SAN arrays over the past few decades has been NetApp. But now, EMC (Dell), 3Par (HP), Hitachi, and IBM and have also risen to the top of the pack.

Development of Flash-Based SAN Storage

Flash storage can be simply defined as electronic storage memory designed to write, read, and store data continuously without having power being applied to it. Flash storage systems differ from traditional HDD storage because there are no moving parts. A typical HDD is made up of magnetic platters that spin at extremely high speeds (10K or 15K RPM for enterprise-class devices) with a small read-right head that moves back and forth over the disks. These drives are physically limited in how fast they can access data and are also vulnerable to the heads crashing into the spinning platters. Flash-based SAN storage does not have that limitation since it’s fully electronic. Flash storage developed from a goal to have reliable storage that would fit in a device you could hold in your hand. Although it was invented in the early 1980s, its wide spread use and mass production did not come about until roughly the middle of 2000.

No-Flash, All-Flash, and Hybrid Storage Arrays

Up until about 10 years ago, the only type of SAN storage array you could buy was one that contained spinning disks. While flash memory was available at the time, it was too expensive to scale to Terabyte sizes and beyond. As the need for data storage continued to skyrocket, the price per GB of flash-based SAN storage has been going in the opposite direction. Somewhere around 2014-2015, all-flash arrays began to appear on the market. The demand to leverage flash-based SAN storage has sharply increased in recent years and is used for data requiring ultra-low latency.

So why would you use one array type over another? Although reliability may be a factor, it all comes down to speed! All-flash arrays can be up to 10x faster than spinning disk arrays. Hybrid (part flash and part spinning disk) are typically faster than pure spinning disk arrays but still cannot match the performance of all-flash drives. Software must decide what data to place in the flash memory and what should be on spinning disk. As you might expect, the algorithms that control when to put data into the flash memory don’t always get it right. An all-flash array does not need algorithms to get it right since the data is stored in memory. Two other advantages of all-flash scenarios are that it takes much less power and space than traditional drives. These are important factors when dealing with expensive data center space.

To Flash or not to Flash

So what’s the catch with flash memory? The trade-off for this significantly faster speed is cost. Using flash-based SAN arrays is still roughly 4x more expensive than spinning disk technology and this can be limiting depending on the data storage size your needs demand. Big data, for example, would benefit from an all-flash approach, but it’s often too costly to go this route. This situation will become less important as the price of flash technology continues to fall.

Flash-based SAN storage is perfect for applications with low-latency needs, and those using relatively small data volumes – when speed is critical and can justify the investment in an all-flash array. If your application requires massive amounts of data (50TB or more), then an all-flash may be price prohibitive for your budget. In this scenario, it may be better to consider a hybrid array.

One of the best use cases for all-flash technologies is a high-end ecommerce platform, where speed is king. Studies show that mobile site abandon rates are at least 50% higher for sites taking more than 3 seconds to load a page. There are many variables affecting page load times, but the major one is data access speed. All-flash storage helps eliminate this issue.

The Bottom Line

What it all comes down to when dealing with storage is using the right tool for the job. Today’s technologies all have their place. It may be hard to believe, but tape and similar legacy systems are still used for long-term, low-priority archiving. Traditional spinning drives and well-designed hybrid solutions will be around for some time to come as they provide a cost benefit over the current all-flash systems. However, when applications have an insatiable need for IOPS and put a constant crunch on system resources, flash-based SAN storage is definitely the way to go if you have the budget.

Glen Thayer

Glen Thayer has been in IT since 1988. He has a passion for solving real world challenges with the right technology solution. Glen is an AWS Certified Solutions Architect with extensive experience ranging from systems design and administration to enterprise architecture.

Latest posts by Glen Thayer (see all)