Luminous Landscape Forum

Equipment & Techniques => Motion & Video => Topic started by: Morgan_Moore on March 19, 2011, 02:45:23 am

Title: Storing Heaps of Data (Mar 11)
Post by: Morgan_Moore on March 19, 2011, 02:45:23 am
Ive put the date in the title becuase this info may go out of date!

I use a few years old Mac. and some (accounts/letter) PCs on a network the accounts are back up seperately and not of concern to this thread

I have been managing my data - still photos for years in a simple manner..

Date based external HDs (USB connection type cheap ones)

ie I have two HDs maybe Jan10_A and Jan10_B (so far I have 20X2 - 40 HDs)

I store the Bs at an ofsite location

I also store client jpg client delivers somewhere else (as well)

Should I need a picture from Setp09 I just get the HD out of my box plug it an and copy the picture(s) to my main drive and do the work

Just like my negatives! - filed in date folders - 'Jan-Jun 1996' etc

I have a database of what I shot when so easily know the date to look for

I feel disconection from the mains and web (my box of HDs) provides a strong 'firewall'

I feel the probability of both A and B failing to boot at the same time is low (should A or B fail I would imediately create a C)

Not very technical but it seems to work

----

This is begining to be a PITA now I am shooting a lot of video

-I cannot edit video smoothly when it is on a USB HD
-copying a video project onto my regular HD may take hours
-I cannot put Files of over 4GB onto my External HDs

-----
I am not a fan of Raid or NAS systems because they are too complex and dont seem to deal with the concept of

"I have infinite DATA and it will never all be 'online' at once"

They are not fire/burglar/coffee spill/flood proof
All data is in one location
They are too complex for me

---------

I have a fairly poor internet connection - so I dont think clouds are there for me yet

---------

SO being a non nerd - certainly not up for reading computer forums I  heard the mention of the word DROBO and thought 'this sounds right'

-edit speed FW connection
-lots of cheap drives
-possiblity of handling over 4gb files

I then watched the DROBO intro video.. http://www.drobo.com/resources/drobodemo.php

And was immediately put off, the girl mentions that 'drobo backs up your data automatically'

No good - Im convinced that will get in a mess, say I have 4 1TB discs plugged in but 30TB worth of disks near by

All I want is the chuck some HDs in a slot and they appear as external drives on the desktop (or better still on the network)

Ill handle the data management myself (just like I handle exposure myself thanks!)

Any thoughts on a system in the DROBO price bracket - maybe I can get a drobo and just turn the auto stuff off ?

If I am going for 'bare' HDs I feel when offline they shoudl be stored in plastic containers (coffee spill proof) - source of these?

Thanks

SMM





Title: Re: Storing Heaps of Data (Mar 11)
Post by: Christopher Sanderson on March 19, 2011, 10:37:59 am
Here is what I do for video.

Since hard drives are relatively cheap, a system with multiple hot-swappable drive bays makes the most sense. I have a Sonnet Fusion (http://eshop.macsales.com/item/Sonnet%20Technology/FUSD4Q2TB/) and multiple trays. This is connected by eSATA cable (but will become obsolete when an equivalent Thunderbolt system become available in the next few months...)

I store ALL original video on HDs named ORIGINAL; I have an A version on-site and a B version off-site, they get rotated regularly.

I transcode the originals (to Apple Pro Res) for editing. These files exist on two separate (redundant) mirrored RAIDs set up in two of the external Sonnet drive bays and two of the MacPro's internal drive bays.

Once the project is finished, it is exported in ProRes to a set of A & B drives named FINAL. Project files and key graphics are archived on a separate drive. The editing ProRes files are then deleted.

I also have a Voyager Hard Drive Dock (http://eshop.macsales.com/shop/NewerTech/Voyager/Hard_Drive_Dock) to quickly copy files onto a storage Hard Drive. This avoids the 4 minutes required to screw a drive into a tray.  :D

So far this system has worked pretty well.
Title: Re: Storing Heaps of Data (Mar 11)
Post by: alain on March 19, 2011, 02:54:36 pm
...
I also have a Voyager Hard Drive Dock (http://eshop.macsales.com/shop/NewerTech/Voyager/Hard_Drive_Dock) to quickly copy files onto a storage Hard Drive. This avoids the 4 minutes required to screw a drive into a tray.  :D
...

In Europe the same looking hard drive docks are sold under the sharkoon quickport label.  They also sell silicone protection sleeves to put the HDD in to get a minimal protection (named : Quick Protect).

I do like them very well for the easy off working.
Title: Re: Storing Heaps of Data (Mar 11)
Post by: Jim Pascoe on March 21, 2011, 06:31:37 am
Sam

Regarding the DROBO - I have one and it has performed well for me.  Four 1TB drives installed which gives me about 2.7gb of usable space.  This just appears on the desktop as one big drive and I know that if one of the drives fail I can replace it and not lose any data.  I have also read of potential problems and pitfalls with this solution - but for me it has worked well.

Jim
Title: Re: Storing Heaps of Data (Mar 11)
Post by: Christoph C. Feldhaim on March 21, 2011, 07:54:08 am
On my PC I use A Mirror Raid.
For backup I use a similar system but from Iomega, 4x1TB in a small NAS box as RAID 5.
Easy setup, reliable.
Additionally I have an online backup service (backblaze).
I feel safe and spoiled with that solution.
Title: Re: Storing Heaps of Data (Mar 11)
Post by: Morgan_Moore on March 21, 2011, 02:06:03 pm
Thanks everyone, the second two solutions dont seem to work for 20Tb and fast rising quantities of data

Chris, Ill go through your solution in more detail

SMM