Topic: Cloud Backups (Read 8485 times)

Craig Arnold · « **on:** October 30, 2009, 07:46:35 am »

I would love to use S3, and I only have around 1TB of images at the moment.

But that's $150 per month.

An external 1TB USB drive costs a lot less than that to buy and can be "sneakernetted" to a friend's house or the office. Not as good, but a whole lot cheaper, though drives do need to be checked periodically.

Maybe S3 is a solution for the portfolio shots however. For many or even most of us we would probably be reasonably happy if we could save our best 100 images from a fire. At an average of 20Mb per image that's only around 2Gb or $0.30 per month - much better!

I think there is a market for online archives though, as opposed to online always-available storage, at a fraction of the cost. If my house burned down and my offsite USB drive wouldn't fire up, I would happily wait for a week to get my files back from an online archive service.

Jeremy Payne · « **Reply #1 on:** October 30, 2009, 07:50:49 am »

Quote from: peripatetic

I think there is a market for online archives though, as opposed to online always-available storage, at a fraction of the cost. If my house burned down and my offsite USB drive wouldn't fire up, I would happily wait for a week to get my files back from an online archive service.

I agree - I don't want online 'backup' ... I want online archives.

I have working files on RAID 0, backups locally on RAID 5 and archives on a disk in the closet. I want the archive off-site and managed.

lobsterboy · « **Reply #2 on:** October 30, 2009, 09:34:06 am »

I looked at S3 and came to the conclusion that it was way too expensive for my needs.

In the end I opted for BackBlaze: $5 a month or $50 a year for unlimited storage.

As with all these things, initial upload can take months but thats about the only downside - it just works.
The advantage over sneakernet is you don't have to remember to do a backup & transport the disk every few months.

fike · « **Reply #3 on:** October 30, 2009, 09:41:05 am »

That was an interesting article, but agree that to store 1TB of data at Amazon is too expensive. For that price, I can buy lots of hard drives for local backups. I generally keep two backup disk sets using 1TB or 1.5TB drives. One backup package is stored at my cabin in the mountains, while the other is kept in a hot-swap SATA bay on the front of my PC, continuously backing up my main data drive that is in RAID 1. Every month or two, I swap disk sets.

All of my data is simultaneously on 3 or 4 drives. If my house burns down, the most I could lose would be a month of data. If my house burns down, honestly a couple of weeks of photos will be a minor issue. (OCD pros will disagree with me.)

Here is the setup:

* Active Internal 1TB Drive: Archive of RAW files from prior years (2002, 2003....2008)
* Active Internal RAID 1 Array of 1.5TB Drives: Current year RAW archive, working directories, and Final Copy directories for all years.
* Removable 1TB Hard Drive in Drive-bay: Daily backup of Archive of RAW files and Current Year
* Removable 1TB Bare Drive stored offsite: Less than 30 day old Backup of Archive of RAW Files and Current Year

Using Amazon for Final Copies may be an interesting idea. I will need to look into that. Otherwise, I think I am covered. All data is on a minimum of three drives at all times. One of those copies is offsite.

I have considered NAS systems like Drobo, but I am not enthusiastic about proprietary systems like that, and I don't see how it would be an improvement over what I am doing.

sergio · « **Reply #4 on:** October 30, 2009, 10:29:13 am »

What I do for my backups is pretty "primitive". I use LR as my database, and put all my files into numbered hds and then carbon copy them into another disk the same size and put in a safe. Works like a charm. But, Geoff's article made me paranoid. What can I do to make sure that my backup is Ok? Technology definitely DOESN'T make life simpler. I find shooting digital liberating in some aspects, but not in others. Converting my images into a zillion 1s and 0s doesn't make it easier.

DarkPenguin · « **Reply #5 on:** October 30, 2009, 11:39:09 am »

I use Mozy for my backups. But with the connection I have it takes a while to back things up. If I wasn't a photo packrat this wouldn't be a problem.

I also backup to an extra internal HD and several external HDs.

Edit: As an aside a metadata aware backup application would be great. I really only need to backup images with a rating of 2 stars and above.

soboyle · « **Reply #6 on:** October 30, 2009, 12:40:36 pm »

$8K raid setup, or $300 monthly at S3 for my 2 TB?
This solution is for somone in a different tax bracket than I'm in.
I need some creative solutions for regular folks. Currently I keep 2 off-site external drives, one full backup, one just raw and jpg backup. my home computer is no raid, regular SATA hard drives, plus an external USB drive backup.
Who can afford to keep updating systems with costs like those mentioned in the article?
Not me.

One option I have not heard mentioned anywhere, you may already have a lot of online storage space that you are already paying for. If you have a website, you likely have gigabytes of unused storage on your account, simply ftp files to a directory on your account, possibly encrypting them before uploading. Might be a good solution for 5 star images.

TimG · « **Reply #7 on:** October 30, 2009, 01:45:51 pm »

Quote from: soboyle

$8K raid setup, or $300 monthly at S3 for my 2 TB?
This solution is for somone in a different tax bracket than I'm in.

You mean not everyone on this forum shoots with a Linhof 679CS + P45, and they're not *gasp* well-heeled (former) executives at Fortune 500 companies? LOL.

Shoot! And to think, I thought I needed at least $50k in gear just to call myself a photographer.

pschefz · « **Reply #8 on:** October 30, 2009, 02:39:13 pm »

after years of messing around with mozy, elephant trunk and others i finally found crashplan....and it really works....the biggestt advantage is that it not only allows you to put everything in the cloud, but also to sync the data between remote computers.....
i have a fios with 20mbps upload and crashplan actually uses most of it....all others cap the upload speed...mozy with 140kbp is a joke....
i have 2TB in the cloud right now and keep adding....price is 50$/year unlimited storage, no cap on upload and the backup software is rocksolid in 10.6.1.....can't beat that....

John.Murray · « **Reply #9 on:** October 30, 2009, 06:32:48 pm »

We recently configured a similar promise cabinet for a medical imaging application (we used SAS) - I can attest to the build quality and ease of use. Only downside is noise, these things are LOUD.

DarkPenguin · « **Reply #10 on:** October 30, 2009, 06:49:36 pm »

Quote from: pschefz

after years of messing around with mozy, elephant trunk and others i finally found crashplan....and it really works....the biggestt advantage is that it not only allows you to put everything in the cloud, but also to sync the data between remote computers.....
i have a fios with 20mbps upload and crashplan actually uses most of it....all others cap the upload speed...mozy with 140kbp is a joke....
i have 2TB in the cloud right now and keep adding....price is 50$/year unlimited storage, no cap on upload and the backup software is rocksolid in 10.6.1.....can't beat that....

I get up to 700kbps with Mozy. What speed are you getting to crashplan?

robertvine · « **Reply #11 on:** October 30, 2009, 07:00:22 pm »

I found a much easier solution to RAID. What the article fails to mention is that when you want to add more storage to the array, you need to destroy all the data (so really you need to move it all to another system). I bought a Windows Home Server, it is not a true RAID system but it uses a software solution to ensure that two copies of you data is stored on separate drives and you can simply add a new drive to the system and add it to the pool in order to increase your storage space. It was very easy to set up and has a great interface plus lots of other useful features. I started with a 640gb and 400gb drive and have since added 500gb and 1tb drives to the pool. With a gigabit network switch the file transfer from the WHS is as fast as keeping everything on the internal drive of my Mac.

I only keep about 90gb of files on Amazon S3, about 60gb of photos and 30gb of videos - it's a good solution if you don't store massive amounts of data but it could get expensive if I needed more space. Hopefully prices will come down over time.

pschefz · « **Reply #12 on:** October 30, 2009, 08:02:16 pm »

Quote from: DarkPenguin

I get up to 700kbps with Mozy. What speed are you getting to crashplan?

when i tried mozy the cheap version had a capped upload speed, the business solution is not capped but much more expensive....crashplan does not cap the upload....i get almost 1mb/sec upload....they also have an option to send them a harddrive with data to avoid the initial upload.....
i have about 2TB images, some video, my itunes and my aperture library (which was a problem with several services...) no problem at all, everything gets updated every day and new shoots get uploaded overnight.....

i had looked at them before but they did not offer online backup....only computer to computer....which is great as well btw....and i might put a raid box somewhere else just to make sure....this is easier of course because the initial backup can be done local, the box moved and crashplan takes care of the updates....

i am in no way affiliated but it just works....

i almost signed up with photoshelter (1TB for 1000$/year which also includes a lot of stuff i don't need, website,...) i tried that and the upload speed was insane...i got 2mb/sec....but the price was too much and crashplan really works for me....

bobrapp · « **Reply #13 on:** October 31, 2009, 12:36:53 am »

This topic has me considering going back to film for important work. At least for medium format. I probably have some 20, 30 or more tb in positives and negatives (not including my fathers Kodachromes from the late 1930s or 616/626/120/sheet films from earlier times both here and in Europe)!

A very long hmmmmmm.

I would not want to pass my storage to a cloud - it might rain!

pschefz · « **Reply #14 on:** October 31, 2009, 01:42:53 am »

Quote from: bobrapp

This topic has me considering going back to film for important work. At least for medium format. I probably have some 20, 30 or more tb in positives and negatives (not including my fathers Kodachromes from the late 1930s or 616/626/120/sheet films from earlier times both here and in Europe)!

A very long hmmmmmm.

I would not want to pass my storage to a cloud - it might rain!

my film is actually much harder to file and keep safe...takes up a ton of room and is (obviously) always only in one place...fire, water, theft....and just storage (humidity, temperature,...) who can afford an actual proper film archive?

Rowat · « **Reply #15 on:** October 31, 2009, 11:06:57 am »

I am interested in other users' experiences with Crashplan & Backblaze. Amazon's S3 is amazing, and you know the the company is not going anywhere; but again, pricey, if you are dealing with large amounts of data. 4TB would cost $600/month just for the storage fees.

So, have other users out there had good success with multi-terabyte amounts of data and online services?

Andrew.

feppe · « **Reply #16 on:** October 31, 2009, 10:22:24 pm »

That article was cursory at best, ill-informed at worst. Not to mention missing the price bracket for most people.

The article seemed to imply that RAID is a backup solution. It's not. RAID should be used for uptime and/or speed (depending on the mode). Sure there's redundancy, but that should not be confused with reliability. Your RAID card is toast, your data is likely toast. User error? Same result. Theft, fire, water damage? Toast.

S3 and comparable services are a rich man's offsite solution. Most of us have terabytes of data to backup, and using those services costs hundreds of dollars per month. Just put your data on external hard disks or tape, and take it to the office or a friend.

As for Windows Home Server, that's an even worse backup solution. It shipped with a crippling data corruption bug which MS took quite a while to acknowledge, and several months to fix. It appears to be stable now, but again, it's not backup as it shares all of the shortcomings of RAID for backup use.

Depending on the value of the data and level of paranoia, you should have at least two different backups, at least one of them off-site. You should test that you can actually recover the data periodically; CRC check is not enough, actually try to recover the data. And you should occasionally take snapshots of your data, instead of only using incremental backups.

John.Murray · « **Reply #17 on:** October 31, 2009, 11:40:23 pm »

I didn't see any implication that RAID was being used as backup in the article. You really should look at the Promise cabinet; it goes far beyond any consumer RAID solution. The individual unit I worked with contains 4 independant SAS channels connected via two separate controllers. Failure of up to 3 of the cannels will slow down data transfer but will not affect operations:

http://www.promise.com/upload/datasheet/1_...0s_DS112807.pdf

Windows home server is just another implementation of dynamic disk capabilities starting Server 2003 (with a *nice* interface) - like any RAID solution, there are bad, better and best practices:

http://support.microsoft.com/kb/816307

I agree with your conclusion regarding Amazon's data backup pricing, but one point missed, is reliability: these folks know how to warehouse data! I probably wouldn't store *every* file in my possession there, but certainly the inportant ones....

atwort · « **Reply #18 on:** November 01, 2009, 06:12:18 am »

I don't see the need for a RAID,you cannot keep adding new drives, they are all kept running and so more likely to fail, if you want to work between 2 locations you have to cart the whole thing around and all disks must be replaced when full and so are a pain to bring back online unless you have another cabinet.
Drives are getting larger and cheaper, I use incremental daily back up onto 2 internal drives for live work which then gets copied onto 2 drives in a JBOD SATA sonnet cabinet and stored 1 at home and 1 in the studio. iView media pro is my thumbnail database so an image is easily found, the catalogued off line drive can be quickly located and slotted into the cabinet.

All these spinning discs in Raids, failing, burning up power, noisy, hot, no need.

fike · « **Reply #19 on:** November 01, 2009, 07:58:48 am »

I think there is a lot of misinformation about RAID. Earlier in this thread someone said that if your RAID card goes bad, your data is unrecoverable. That isn't true of all implementations. Another person said that you need to tear down and recreate your RAID array if you want to add a drive for increasing size. That, too, isn't true for all implementations. It is true that older RAID systems had both these faults, but these issues have been addressed.

RAID is a practical every-day solution for redundancy so that your backups don't have to be done hourly. If you want to work from a laptop, they aren't practical, but doing all your photo work from a laptop isn't a very practical solution anyway (i'm not even going to go into it).

One suggestion, that I haven't heard mentioned is to make sure to replace hard drives before they get too old. It is kind of wasteful, but I rarely have trouble finding someone to take and use my old drives for USB drives or something similar. Another use for old hard drives is to put them into RAID 0 as a fast scratch disk. I would say that I rarely keep a data drive more than 12-14 months.

I agree that the Amazon service pricing is a bit out of reach for most of. I am not sure that whether I need to use it in addition to a decent backup and recovery plan that includes offsite storage.

Author Topic: Cloud Backups (Read 8485 times)

Jeremy Payne

DarkPenguin

DarkPenguin