Read about them here

The post 12TB Hard Drives on the Horizon appeared first on WalterBiscardi.com.

We store everything, and I mean EVERYTHING, on bare hard drives.  Yes, I know what you’re saying.  “Walt, hard drives die!”  Yes, they absolutely 100% do.  That’s why everything is stored on both a Master and Clone with the clones stored off-site.  When either the Master or Clone dies, we purchase a new drive and make a brand new Clone.

Organization: First off, every drive gets assigned a 7 digit number by our Media Management Specialist such as 0002372.   Why 7 digits?  So we can store up to 9,999,999 items before we run out of numbers.   This applies to ALL media that’s stored in our library including Tape, DVD, Blu-Ray, CD, Hard Drives and anything else that goes into the library.   It’s all managed through a VERY simple Filemaker Pro database we started about 6 years ago and it still works quite well.  We tried using CatDV for a while, but it was just too confusing and cumbersome so we switched back to Filemaker and have kept running it since.  Kelly can pretty much find anything in the shop within 5 minutes so it’s still working well for us.   Moving forward, the next thing we will test out is Axle as soon as we get our new series rolling which will be reality series style so it will involve a LOT of media per episode.

Storage Unit: We’ve been running “Tray Less” drives from WeibeTech for all four years we’ve been doing this system.  That company was purchased by CRU-Dataport a few years ago, but the units still remain.  You can get 1, 2, 4 and 8 bay units. Here’s a picture of the RTX410-3QJ which is essentially the latest version of the 4 bay models we run in the shop and we also run a couple of the single drive units.  This is a JBOD unit with four individual connections for the four drive trays running standard SATA drives.  What this means is that all four drives will show up as individual units instead of the entire box showing up as a “RAID.”   So we can copy data to two Masters and two Clones simultaneously if we want.  And you don’t have to use all four slots to use the unit.  1, 2, 3 or all slots can be used at any given time.

The “Tray Less” designation means the drives literally slip in and out of the drive bays like the old floppy disks.  Open the front door, slide the drive in, close the door and start your data transfer.

We have these connected to a Mac and we manually control all data transfer.  So our Media Management specialist will load the appropriate drive, confirm the data that needs to be archived off our server and manually drag it onto the drive for the transfer.  Then she’ll copy from Master to Clone and then the Clone will go off site.

We reverse the process to put the data back onto the server.  It’s VERY simple and VERY low tech.

I’ve sent these units out into the field and when we get the series, this will be how we’ll transfer, clone and ship media from the field.   The master footage will be copied from the Camera cards to three drives simultaneously in the RTX unit.  Drives 1 and 2 will be the Master and Clone with Drive 3 the Backup Clone.   Drives 1 and 2 will stay with the Production Team until they return to Atlanta while the Backup Clone will be shipped back to Atlanta at the end of each production day.

Since this unit uses off the shelf SATA drives with nothing proprietary we can pick up additional hard drives pretty much anywhere on our travels around the US and around the world as needed or just have drives overnighted to wherever our crews are.

Hard Drives: We tend to purchase Western Digital and Hitatchi drives as they are generally the most plentiful around and have the best price vs. performance.  Remember we don’t need top speed / top performance because we’re not editing with these drives, just storing them on the shelf.   We purchase from a myriad of locations such as TapeOnline.com, Fry’s Electronics, Best Buy even Staples on occasion when they have a sale.   So far the largest drive we’ve purchased is a 3TB model, but we tend to stick to 2TB as they seem to be just the right size to hold most of our projects.   The price point has also drastically dropped for the 2TB so they’re a good buy for the size.

Storing the Media: WeibeTech, now CRU Dataport also sells the “Drivebox” which is an antistatic box to store the bare drives.  Literally looks like small VHS box and the drives store very securely in these boxes which take up very little room.

You can see below the DriveBoxes sitting on IKEA Benno DVD towers.  I like these units because they’re very compact footprints with very short shelves that don’t sag under the weight of the drives.  We can store 40 drives on a single Benno unit so with 2TB drives, that’s 80TB of backup data in a very small footprint.  Considering it takes about 4 of these units side by side to equal a “normal” bookshelf, that’s 320TB in a nice compact footprint.  The taller boxes are the original version of the DriveBox, they’ve gotten shorter in the last two years.   We make sure to spin up all the drives every four months minimum to make sure the drives are ok.   If any shows any sort of falter, we replace it immediately.

Fail Safe Storage: Now if you want to go all out “Fail Safe” secure backup, then you start looking into LTO units which are just on the transition phase from LTO-5 to LTO-6.  The plus side is that the tapes are guaranteed to last for 50 years.  Yes, I said tape, still the most stable thing to put on a shelf.  The downside is they only hold 1.5TB maximum for media.  Yes you can store 3TB of compressed data, but you should never compress video data.   The storage units themselves generally start around $2000 and quickly go up from there.   I’ve been considering a switch to LTO because the LTO-5 tapes are only $33 per 1.5TB.  BUT all the LTO units are proprietary per manufacturer so once I commit to a manufacturer, then I’m stuck with them for a while, unlike these bare hard drives which I can put into any SATA drive unit and a Mac or PC will read them.   That is changing now with LTO-6 and a new format to write to, but it’s still on the “bleeding edge” for me so for the time being, we continue to go with the hard drives.

I expect that by mid 2014 we will begin transitioning over to LTO for long term off site archive storage and continue using hard drives as on site archive storage.   In fact, an LTO solution will probably be my main focus at the NAB show in April 2014.

There you go, a quick look at how we’ve been archiving our data at BCM for about 4 or 5 years now and it’s worked quite well for everything we’ve been doing including the documentaries, episodics and corporate projects.

The post Archiving Data appeared first on WalterBiscardi.com.