By Ben Kilburg, Senior Solutions Architect – East
At Chesapeake Systems, we like to say that our passion for the bleeding edge of technology helps to keep our clients on the cutting edge. That motto fuels the primary purpose of our work – to help you use technology to your advantage. Sometimes that means you need the latest and greatest products in the marketplace. Just as often, you don’t.
A current trend that we’re starting to see become part of the architecture for possible shared storage solutions is NVMe (Non-Volatile Memory Express). NVMe is a new storage protocol designed to provide direct data transfer between central processing units and SSDs using a computer’s PCIe (Peripheral Component Interconnect Express) bus. It offers an alternative to SATA (Serial Advanced Technology Attachment) and SAS (Serial Attached SCSI) protocols and was designed to address the bottlenecks inherent in these previous technologies, unlocking the full potential of solid-state media. Its benefits include higher input/output operations per second (IOPs), gobsmacking throughput, and greatly reduced latency. The specifications for these drives report them to be roughly 20 times faster than traditional spinning HDDs (Hard Disk Drives) are today.
Naturally, people are talking about NVMe, especially after IBC 2019 where Quantum and Western Digital, to name just two, showed off their solutions.
It’s the hot new thing. But do you really need it?
If your media creation environment needs extremely low latency and fast access to large amounts of data, then NVMe should certainly be considered. A couple of examples would be if you’re working in a collaborative environment with multiple streams of uncompressed or lightly compressed 8K or 4K video in use per workstation, or if you are orchestrating an event with a large number of concurrent ingest or playout feeds, such as an eSports competition.
Additionally, VFX houses with large numbers of real-time or non-real time renders might profit from every conceivable advantage to make shots available as fast as possible. There are some non-video workflows that also need extremely low latency and extremely fast performance as well. If I were trying to master global finances via high-frequency trading, build the perfect human via genomic research, or profit from understanding the human condition via real-time big data analytics, you bet I’d want to build a fire breathing NVMe monster.
In our corner of the industry, unsurprisingly, we are seeing organizations dealing with large amounts of data as quickly as possible interested in NVMe, such as big media conglomerates ingesting a lot of high-resolution media and networks acquiring shows at the highest resolution so they can future-proof content. These organizations may still be delivering primarily in HD but they are archiving 4K files for a time in the future when viewers may expect higher resolution as a matter of course.
Furthermore, Quantum, whose F-Series NVMe storage arrays received two industry awards during the IBC 2019 show, may be considered at the forefront of NVMe. Their F-Series storage, designed specifically for high-end video workflows, uses 24 NVMe drives per chassis and provides users with 25 gigabytes of aggregate bandwidth to work from simultaneously.
But outside this super high-end usage, most of our customers really don’t need a shared storage NVMe solution yet.
Most production and post environments currently don’t require extremely low latency or high IOPs because video playback is about large streams of sequential data. They’re ingesting and working with video on large, centralized, shared storage volumes that are using dozens if not hundreds of hard disk drives, which allows them to retain petabytes of information with great performance. Currently, that’s still the best bang for the buck, and the best solution for most use cases right now because compared to a petabyte of HDDs, a petabyte of NVMe is exponentially more expensive.
That’s not to say that NVMe doesn’t have its place in your workflow today — it’s just a matter of the scale of adoption. People are moving towards solid-state technology wherever it is affordable to do so. For example, I wouldn’t buy a new workstation or laptop without one. On a smaller scale where NVMe’s reliability and performance truly shine, it can make all the difference in the world to a freelancer or editor working remotely from internal or direct-attached storage. In fact, desktop workstations and laptops equipped with NVMe storage outperform some of the SAN volumes we built for customers five years ago and certainly weigh hundreds of pounds less.
Additionally, a hybrid approach to shared storage — where some storage vendors provide a layer of NVMe cache on top of traditional HDDs — could be commonplace in the near future. This solution could provide the best of both worlds with the speed of NVMe drives and the capacity and price of HDDs, so long as the software controlling the data flows between the drives works transparently and seamlessly.
In conclusion, we believe that customers at the top end – like studios, networks, and VFX houses – will use large NVMe shared storage volumes first. Additionally, the single, independent editor can make NVMe the cornerstone of their business by buying a machine equipped with a 1-4TB NVMe drive or attaching this storage to their machine via Thunderbolt 3. This option is cost-effective and offers 10 times the speed and performance of previous configurations.
Mid-tier media entities that comprise the majority of content producers and asset owners, who are currently working with compressed 4K or mezzanine HD video, might still consider the greater capacity and average performance provided by HDD storage solutions versus a smaller capacity NVMe storage solution with weapons-grade performance, as a better fiscal choice. You don’t need a supercar if a nice SUV will get the job done.
NVMe is in their future, too, as the industry inevitably moves to 4K, 8K, HDR, and whatever other astounding and immersive new technologies are waiting in the wings ready to blow our collective minds. Right now, the decision to explore NVMe really boils down to IOPs and latency.
Chesapeake Systems is ready to help if you are considering NVMe. As your creative IT resource, we know time is money. Let us help you find the right solution for your business.