Modern LTO: mLogic Revolutionizes Long-term Storage
History of LTO
LTO tape is a data storage format our clients have been using for years for longer-term preservation of media files, and it continues to be as relevant as ever to the needs of video and rich media producers. The tapes are relatively inexpensive in terms of the price per terabyte they offer, relative to hard drives, and are similarly dense – they are a small, light media format that is easy to store and transport. Of course what makes LTO (which stands for linear tape open) particularly appealing is its characteristic data preservation capability. While certainly not infallible, LTO tapes are, as a technology, better at preserving data for longer periods without risk to the data being corrupted over time, or the media itself failing, than hard drives. It is what the tapes are designed for, after all.
We tell our clients that it’s always best to archive data onto a pair of LTO tapes than a single tape, in case you do experience a media failure. With that said, if kept in the proper environmental conditions, LTO tapes can work perfectly fine after being kept on the shelf for ten or more years. This is totally unlike, say, a pile of desktop hard drives sitting on a shelf, going unused for years at a time. Hard drives are particularly prone to failure after longer periods of inactivity, and any hard drive that’s been “shelved” is going to have very high odds of failing if you try to fire it up for the first time in five years, hoping to pull some files off for a new project. There is a high chance the hard drive will fail to spin up, and the read/write head may literally have stuck to the platter due to lubricants congealing over time. Hard drives are made to spin, and work better when they are spun up regularly.
A Different Way
We at Chesapeake Systems have long sought a more “approachable” entry into LTO technology, and a few ventures over the years have tried to bring a less expensive desktop implementation of the LTO tape drive to market, with less than total success. Most LTO tape systems come in the form of larger “library” units, which often feature a handful of LTO drives to read from and write to multiple tapes at once. Tapes are stored in these libraries by the dozens, hundreds, or even thousands, and automated robotics (controlled by one of a handful of tape archive software packages, such as Archiware P5 Archive or Quantum’s Storage Manager) handle the job of moving tapes between inactive “slots” and the actual tape drives which the tapes need to be inserted into in order for their content to be accessible, or for the tape to be available to have more data written to it. Libraries can fill half a server rack, be the size of one or more server racks, or even fill an entire data center. These sophisticated tape library systems are fantastic, but out of the budget for smaller shops. Some folks also look to LTO as a useful format for, say, sending data from a shoot back to the post-production operation, or to send a completed program from post over to a broadcaster for delivery (it is quite common these days for broadcasters to request LTFS-formatted LTO tapes as a file-based delivery master format replacement for the video tapes of yore). For these more basic data transport scenarios, a large tape library system managed by third-party archive management middleware is just too “big” a solution.
Tabletop LTO drives have been a reality for a number of years, but have not been embraced by many media producers due to, I believe, the complexity of their setup and use. First, you need to connect the LTO drive to your computer system, and that up until recently has required putting a SAS (serial-attached SCSI) PCIe card into your desktop computer, and then connecting the interface card to the tabletop drive. That’s a practically old-school SCSI level of annoyance for today’s users, many of whom have “come up” in the era of USB, Firewire and Thunderbolt interface standards. Many media producers are on the Mac platform, and no Mac has shipped for at least 3 years that even offers an internal PCIe slot that could accommodate a SAS card.
Additionally, the software used to control these tabletop LTO drives has been a mixed bag, with some people using third-party archive software to control even a single drive tabletop LTO system, while others used LTO drive control software that was more “native” to the drive itself, and available directly from the manufacturer for use under either macOS or Windows. These simplified software packages, which typically format LTO tapes as LTFS (linear tape file system), were the most promising approach we’d yet seen to controlling tabletop LTO drives. LTFS is a neat format – it involves keeping an index of all the files on a tape immediately available on the tape itself, which can be read by a system, thus allowing it to provide a list of files on the tape to an operating system. With the right software, the operating system can be essentially “tricked” into thinking of the LTO tape as a “drive,” and can display it as a file path in Mac’s Finder or Windows’ Windows Explorer filesystem navigation applications. Files can be dragged onto the tape “drive” that appears to be mounted to copy data onto it, and dragged off to copy data off. Essentially a desktop hard drive-like workflow, with a data tape. You can’t play a file directly off of the tape, because that requires actually pulling data from tape back to a file system, which the tape is not (it is only masquerading as one, via the LTO tape drive software). All in all this has worked well, but again, these days, most media producers don’t have a ton of use for a tape drive solution that requires a PCIe SAS card.
New Solution
We became aware of a solution to this problem last year that we finally had the chance to test out toward the end of 2016 that I would like to share. We learned about a new manufacturer of tabletop LTO drives, aimed at media professionals, called mLogic, and they were kind enough to send one of their tabletop LTO drives to us to test this past autumn. mLogic’s tabletop drives were similar to others we’d worked with in the past that used SAS to connect to a host system, with one major difference: They feature a native Thunderbolt interface! It seemed like the right solution for a customer we were considering it for, but we wanted to test it out in the shop to make sure it would work as expected. I spoke with Roger Mabon, mLogic’s CEO, and he was willing to send us a loaner unit to evaluate for our prospective customer, and for our customer base in general. What we found was encouraging.
Like the tabletop LTO drives that use SAS interfaces, mLogic’s drives use Mac or Windows software that allow LTO tapes to be formatted as LTFS, and basically treated like a mounted filesystem, even though they are really not, and the same limitations I described above apply – you cannot work with data directly from the tape, as it must first be copied back to a real filesystem, which your applications are designed to work off of.
You plug the drive in via a single Thunderbolt cable, which of course couldn’t be easier. mLogic’s Thunderbolt 2 devices can work with the latest Thunderbolt 3-equipped Mac and Windows machines, so long as you use a Thunderbolt 3 to Thunderbolt 2 adapter cable – no big deal. You install the Mac or Windows software, pop in a tape, format it, tell your computer how you’d like to pseudo-mount the tape as a volume or directory, and there it appears, ready for data. Pop a tape with data into another computer with an mLogic drive (or nearly any setup that can read LTFS-formatted LTO tapes), and you will be able to see your files and pull them off to disk. It really is that easy.
I asked Roger at mLogic what inspired him to release their line of products (they have a few models, including a rackmount dual-drive unit that allows you to clone data to two tapes at once – which can also be done by daisy-chaining two of the single drive desktop drives together, and enabling the feature in their software). Roger reminded me of something that had totally skipped my mind – he was the guy who founded G-Technology! Who in the world of media hasn’t used G-Tech drives at some point over the years? Chesapeake had sold probably thousands of them, and I told Roger I had missed this connection. He went on to tell me that when he would visit various production shops, and see stacks of his G-RAID drives on shelves or in closets, it would kill him, because he knew that it was not a reliable long-term data storage medium, as this is not what desktop hard drives are for. “So I decided to do something about it,” Roger said, and he founded mLogic to improve access to the much more appropriate longer-term storage technology of LTO. I thought this anecdote was hilarious, as I have had the experience of terror when seeing such stacks of desktop drives. For Roger it must have been quite visceral, as they were more often than not, stacks of his own drives! I have to give credit to the guy for really seeing the need, and doing something about it.
“We’ve never before had such a compelling desktop archive, and even desktop archive plus MAM, solution in our portfolio until now.”
A workflow we will be implementing soon using the mLogic drive will also involve another piece of software, the MAM CatDV, and specifically its “archive plugin” that is built into some versions of the desktop CatDV client application. CatDV’s archive option was originally developed for another tabletop LTO solution from a number of years ago that never achieved its promise, but we found that it works very well with the mLogic drive and its software and LTFS-formatted LTO tapes. The archive plugin basically takes an asset or assets from CatDV, and while keeping their metadata (and proxy, if you generated one) on disk, basically shuffles the full resolution assets back and forth between their original disk path as tracked by CatDV, and essentially any other external “file path” location. The good news is, LTFS tapes that are mimicking a file path, when using the mLogic software, work fine as a destination for these archive operations! And of course, the CatDV archive plugin is used to restore the data as well. This is a nice extra layer, again for people who want to tag assets with searchable rich metadata, and perhaps keep viewing proxies available online, even when the bigger original media files are offline on tape. What’s even more interesting is, even a purely desktop implementation of CatDV (with no back-end database residing on a separate server) should be able to handle this workflow with an mLogic drive. This brings MAM down to a truly desktop technology, with an all-in cost of less than $10K for hardware, software, and some time for setup and basic training. Even better, if your needs expand, metadata captured to the desktop version of CatDV can be migrated to the full database version of the software down the road. We’ve never before had such a compelling desktop archive, and even desktop archive plus MAM, solution in our portfolio until now. We plan to make sure to let our customers know about the mLogic drives, and that it’s never been nearly so easy, or cheap, to embrace the fantastic medium of LTFS-formatted LTO tapes. A tabletop LTO-7 drive from mLogic, which can hold roughly 6TB of data per tape, runs around $5K. If this is something that’s interesting to you and you’d like to learn more, please be in touch!
