The theme song to the popular TV show, “The Big Bang Theory” condenses the evolution of the entire cosmos into a snappy one minute, 55 second pop tune. I can sum up the entire evolution of computer storage even more succinctly: Tape, Disk, Flash, and…?
It is widely held that magnetic tape was first commercially deployed as a computer data storage medium by UNIVAC in 1951. Later that same decade IBM invented rotating, random access disk storage and the Hard Drive was born.
Few products in history have enjoyed such spectacular declines in cost and size along with equally dramatic improvements in capacity and performance. Today 4TB hard drives are freely available and deliver hundreds of MB/sec bandwidth and hundreds of IOPS per drive. But every technology runs its course and today we see HDD’s rightly going the way of the dinosaur because HDDs could not evolve as fast as CPUs did.
Enter NAND flash with data access and write times orders of magnitude faster than spinning media, greater storage density and lower operating costs. Limited early production volume relative to HDDs meant much higher costs initially, but with consumer applications driving volume adoption in smartphones and laptops, flash components soon began to ride the same cost curve HDD’s have been on for almost 60 years…
Over the last decade or so, flash performance, resilience, density and –yes- cost have made the technology suitable for aggregation into system-level products with sub-millisecond latencies and millions of IOPS possible from a single all flash array. The best all flash arrays are purpose-built with consistent low latencies for optimal performance under an enterprise’s most demanding applications. All flash arrays are straight performance plays, while a small growing set of products offer more.
We are at an inflection point in computer storage: New products are emerging that combine optimized flash-based performance with data protection services and data reduction services built on a unified design at the same cost as legacy HDD-based arrays.
No one knows with any certainty which path primary storage in the enterprise will take over the next several years. The more specific the prediction, the less likely it is to materialize.
There are, however, several fundamental characteristics the next generation primary storage platform will have, whatever the specific implementation might look like. Here’s what I think those characteristics include.
The next generation of enterprise storage will be the first without moving parts. Flash is the fastest growing segment of the storage market and will very soon overtake spinning media in active data applications.
RAID was formalized in the RAID Paper by Gibson, Patterson and Katz in 1989, but vendor-specific, storage-based data protection schemes go back much further than that. And since then the range of data protection options have exploded with mirrored, local, remote, synchronized and asynchronous options. Next generation enterprise-grade primary storage platforms will marry flash performance with the complete suite of data protection services currently found on legacy HDD arrays.
The growth of data, accelerating every year - and this is before the internet of things– has given rise to data efficiency solutions including deduplication and compression. The best storage platforms of tomorrow will have low - block or byte – level deduplication and compression. Those services will be simple at the default level while control of these features for sophisticated implementations will be very granular, from the LUN level, to groups of LUNs, from an entire storage platform to groups of platforms regardless of location.
Unified and Simple
Because IT budget pressure will continue into the foreseeable future, so will pressure on storage suppliers continue to deliver the simplest most powerful solutions possible. Complex solutions, where vendors cobble together components from different product lines are evolutionary dead ends. Solutions built on unified code bases, optimizing purpose-built hardware architectures that are easily managed will become de facto standards in the next generation of mainstream enterprise storage.
The impact of this next generation storage platform will be to accelerate the consolidation we are already seeing in data center storage. Footprint, power and HVAC cost reductions driven by flash performance and densities. As flash based platforms move into primary storage roles, operating expense savings will increase. Payback will happen sooner.
Primary storage platforms in the enterprise will coalesce around Flash. They will have robust data protection features, be easy to run, manage and maintain, and they will not cost more. Market dynamics and the underlying physics will drive cost out of storage for the next several years the same way it has over the last 60 years.
So the future of enterprise storage will be flash-based, with the complete suite of data protection and data reduction services IT professionals are used to seeing, and it will be built on a unified code base, tightly integrated with a rational, robust hardware architecture, all at a cost in line with current HDD array prices. All of this will be managed through a simple management console to keep IT HR costs in line.
Users will be served data faster than they have ever been. SLA’s will be easier to meet. Storage management simplified. Data center costs reduced. The future of primary storage has never looked better.