Pages: [1]   Go Down

Author Topic: Best way to backup lightroom photos?  (Read 2815 times)

N_N_Nikmat

  • Newbie
  • *
  • Offline Offline
  • Posts: 30
Best way to backup lightroom photos?
« on: February 03, 2017, 02:25:03 AM »

I have been pondering this question for a little while now and would like to tap into the collective wisdom available here.

I am using lightroom on an iMac and currently backup to a direct attached USB 3 hard drive via the automated Time Machine in Mac OS.  This happens automatically in the background.

Is this okay?  Can anyone suggest improvements to this?

My primary concern is having a reliable backup of my photos.
Logged

Tony Jay

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2856
Re: Best way to backup lightroom photos?
« Reply #1 on: February 03, 2017, 02:53:28 AM »

Redundancy is the key!
More that one back-up is prudent and have at least one back-up offsite (one could use the cloud but I just use another EHD that is rotated every now and then).

Also you need to back up your catalog (may be you are already doing this) onto those same external drives.

It may be worth asking whether you are a passionate amateur or have professional leanings?
(I don't think that this is a useful discriminator as to value of images but it can play into resources available.)

It is a big topic and another thread in "Computers and peripherals" is dealing with the various hardware and software options for back-up and archiving: http://forum.luminous-landscape.com/index.php?topic=116168.msg957323;boardseen#new
Tony Jay
Logged

rdonson

  • Sr. Member
  • ****
  • Online Online
  • Posts: 2564
Re: Best way to backup lightroom photos?
« Reply #2 on: February 03, 2017, 12:33:10 PM »

I do regular backups to external drives as well.  Hopefully soon I'll be adding cloud storage backup from BackBlaze.

After losing a lot of Lr settings and such one time I decided there was more to backing things up so I could easiy get Lr back up and running.  I now also use TPG LR BACKUP  https://photographers-toolbox.com/products/mdawson/tpglrbackup/.

I may be a bit paranoid as I have two onsite Drobos with one a copy of the other and another external WD drive that clones my Mac HD with CCC.  The Drobos have all my photos.  I just hate rebuilding Humpty Dumpty when it happens so I look for ways to get back together as quickly as possible.
Logged
Regards,
Ron

N_N_Nikmat

  • Newbie
  • *
  • Offline Offline
  • Posts: 30
Re: Best way to backup lightroom photos?
« Reply #3 on: February 04, 2017, 12:01:02 AM »

Thanks for your help and suggestions.  I'll look into the drobo data solution.  That TPG LR BACKUP looks good too.

I found this youtube video helpful for a DAS backup.
Logged

Hoggy

  • Full Member
  • ***
  • Online Online
  • Posts: 162
  • Never take life, or anything in it, too seriously.
Re: Best way to backup lightroom photos?
« Reply #4 on: February 04, 2017, 12:58:37 AM »

I now also use TPG LR BACKUP  https://photographers-toolbox.com/products/mdawson/tpglrbackup/.

Yeah..  I'm STILL pissed that the backups are automatically zipped now, making that plugin only useful for the presets folder!   >:(   It was nice having that plugin do the zipping in the background.

Depending on the OP's resources..  (Let's just say that, for me, free/cheap is not only nice - it's a necessity.)

I have the catalog backups going to a folder that is synced to one of my Google Drive accounts (15GB free).  This also allows for a level of deletion protection on top of some versioning (for other files).

For the images..  Pretty much all my images are DNG with no embedded previews, and only some with fast load data.  So between 2 catalogs, it tops out at almost 50GB.  I would say, for tertiary, but it ends up being a primary for me..  I make running backups to BD-RE50's.  I also need to get off my butt and mail my sister a set of BD-r25's for off-site.  I'm thinking of making a second set for storage in my car, as well - but obviously I won't expect those to last as long.  It should also be noted that I don't get out to shoot as much as I'd like, so they don't build up that fast.  But if I'd make a bunch of really good photos (as opposed to polishing turds :) ), I would surely get out my 2 hard drives as well.
The issue being that I have those hard drives ready to go with me in case of a fire..  And they have power bricks, while the BD writer doesn't..  So the hard drives end up being a PITA to hook up.

So depending on how prolific a shooter you are, a BD-writer may be quite beneficial.
« Last Edit: February 04, 2017, 03:41:30 AM by Hoggy »
Logged
^^^ Many line feeds are refusing to stay in my posted messages on this site, for some reason. ^^^
Cams: Pentax K-3, K-30 & Canon G7X, S100
Firm supporter of DNG, throwing away originals.
It's the hash, man..  That good hash!

BobShaw

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1001
    • Aspiration Images
Re: Best way to backup lightroom photos?
« Reply #5 on: February 04, 2017, 02:35:12 AM »

I am using lightroom on an iMac and currently backup to a direct attached USB 3 hard drive via the automated Time Machine in Mac OS.  This happens automatically in the background.
Is this okay?  Can anyone suggest improvements to this?
You are doing everything and all that you need to do.
You do need to have multiple drives though. At least 3 and rotate them regularly. Ideally have drives big enough to go back at least a year.
Also backup up everything including OS, applications, users etc so that you can rebuild an entire machine
Logged
Website - http://AspirationImages.com
Blog - http://AspirationImages.com/blog
Landscape, Portrait, Product Photography - Sydney, Australia

bassman51

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 83
Re: Best way to backup lightroom photos?
« Reply #6 on: February 04, 2017, 05:30:50 PM »

Very timely from my perspective. My backup strategy was (MacOS):

- primary catalog on internal HD, backed up by LR to Dropbox and Time Machine to a pair of external drives
- primary image storage on external HD, local backup via Time Machine as above, offsite backup to CrashPlan. 
- Additional copy of imported files on alternate external HD, backed up to Time Machine and CrashPlan
- Additional copy of Selects Raws to Dropbox, and JPEGs to Flickr

Last Tuesday morning the external drive housing the primary image storage failed - refused to power up at all.  I immediately ordered a new hard drive, and went to Time Machine to retrieve the local copy.  For reasons I still don't understand, that drive was missing from Time Machine (probably user error).  However, CrashPlan appeared to have all 1.3TB of data, so I began restoring to my internal HD. 

By early afternoon, the new external drive arrived and was installed, and I redirected the restores to that location. 

By Thursday evening, all 1.3TB were restored from CrashPlan and LR was repointed at their new location.

By Saturday, CrashPlan finished the backup of the new HD, probably by noting that the new files being backed up were already at their site.  Time Machine now appears to have all of the image files as well. 

So ...
- multiple layers of backup, using multiple technologies.  Technologies fail, users make errors, etc.
- onsite and offsite - your onsite copies could burn, you offsite provider could fail
- check that the backups exist and you can actually restore by changing or deleting a primary file and recovering it from each backup

Logged
Life is what happens when you're busy making other plans.

N_N_Nikmat

  • Newbie
  • *
  • Offline Offline
  • Posts: 30
Re: Best way to backup lightroom photos?
« Reply #7 on: February 04, 2017, 05:59:57 PM »

Interesting information and stories - thanks.

It seems everyone is doing something different!  However the key seems to be redundancy.

I have only about 150Gb of photos (all of which comfortably fit on internal drive) to backup so for now for me:
1. Continue automated Time Machine whole drive back up
2. Dropbox sync of Lightroom configuration files via TPG LR BACKUP
3. Remote backup (via rotated occasional direct attached storage of photos only)

The other component seems to occasionally check the integrity of the backups.
Logged

Eric Brody

  • Full Member
  • ***
  • Offline Offline
  • Posts: 235
    • http://www.ericbrodyphoto.com
Re: Best way to backup lightroom photos?
« Reply #8 on: February 05, 2017, 01:02:14 PM »

It doesn't appear anyone has mentioned the most important issue. You need a BOOTABLE clone. Many people use Carbon Copy Coner or Super Duper. I use CCC.  Time machine has its uses but it's not really a bootable clone & it's had it's problems, see diglloyd's blog recently. Ideally you'll have multiple backups including at least one off site. The cloud is fine but slow & you're at the mercy of that company's business plan. They go belly up, your files go belly up. As others have said redundancy is the key. The probability of a single drive failure over time is 100%, it goes down with each drive added. At some point the probability of multiple simultaneous drive failures (short of a nuclear blast) becomes low enough that you can relax a bit, so long as you keep backing up regularly. I'm an enthusiast & back up every night, twice, with a schedule on CCC. Good luck, we all need it these days.
Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1834
    • The Last Word
Re: Best way to backup lightroom photos?
« Reply #9 on: February 05, 2017, 01:59:14 PM »

Very timely from my perspective. My backup strategy was (MacOS):

- primary catalog on internal HD, backed up by LR to Dropbox and Time Machine to a pair of external drives
- primary image storage on external HD, local backup via Time Machine as above, offsite backup to CrashPlan. 
- Additional copy of imported files on alternate external HD, backed up to Time Machine and CrashPlan
- Additional copy of Selects Raws to Dropbox, and JPEGs to Flickr

Last Tuesday morning the external drive housing the primary image storage failed - refused to power up at all.  I immediately ordered a new hard drive, and went to Time Machine to retrieve the local copy.  For reasons I still don't understand, that drive was missing from Time Machine (probably user error).  However, CrashPlan appeared to have all 1.3TB of data, so I began restoring to my internal HD. 

By early afternoon, the new external drive arrived and was installed, and I redirected the restores to that location. 

By Thursday evening, all 1.3TB were restored from CrashPlan and LR was repointed at their new location.

By Saturday, CrashPlan finished the backup of the new HD, probably by noting that the new files being backed up were already at their site.  Time Machine now appears to have all of the image files as well. 

So ...
- multiple layers of backup, using multiple technologies.  Technologies fail, users make errors, etc.
- onsite and offsite - your onsite copies could burn, you offsite provider could fail
- check that the backups exist and you can actually restore by changing or deleting a primary file and recovering it from each backup

If I did the math right, that's an average download rate of 60Mb/s. Pretty impressive. I never got anything near that out of CrashPlan.

Jim

N_N_Nikmat

  • Newbie
  • *
  • Offline Offline
  • Posts: 30
Re: Best way to backup lightroom photos?
« Reply #10 on: February 05, 2017, 04:55:02 PM »

The other problem with cloud services here in Australia is upload speeds are seriously limited on ADSL which is what most people have.

I get a max of about 1Mbit/s upstream which limits the practical usefulness of these services.
Logged

BobShaw

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1001
    • Aspiration Images
Re: Best way to backup lightroom photos?
« Reply #11 on: February 06, 2017, 12:52:03 AM »

... However the key seems to be redundancy.

I have only about 150Gb of photos (all of which comfortably fit on internal drive) to backup so for now for me:
1. Continue automated Time Machine whole drive back up
2. Dropbox sync of Lightroom configuration files via TPG LR BACKUP
3. Remote backup (via rotated occasional direct attached storage of photos only)
The other component seems to occasionally check the integrity of the backups.
In answer to the first part - NO, Redundancy is not the key.
People on the Internet unfortunately use the word "redundancy" like they know what it means. Redundancy provides no backup at all. Redundancy is the R in RAID and its prime purpose is to allow a disk array to keep working if one disk fails. That allows the credit card payments to keep coming in the middle of the night, but is generally totally useless to a photographer. Redundant systems have the SAME data. That sounds good until you realise that you two bad copies or two deleted copies.

Backup on the other hand provides multiple copies of different data, being the data you had yesterday, the data the day before, the data the year before. That means that if you delete or corrupt the file today then you can go back (back up) to the copy you took yesterday. Time Machine does that brilliantly so keep doing that. Ignore comments about it not being bootable. In practice it is using CMD-R on any machine, not just the one it was made on.

If you only have 150G today then if you continue in photography then very soon you will have more data than will fit on an internal drive. I suggest that you move to an external drive sooner than later.
Points 2 and 3 if you live in Australia you can forget. I am on NBN which is much faster than you and it would take 5 weeks per TB to restore over the Internet. Just use multiple drives and keep one at a relatives.

Yes, check integrity. I scan the folder structure of the backup and look for random changes I know that I have made. Hardly perfect but seems to work.

Cheers.
Logged
Website - http://AspirationImages.com
Blog - http://AspirationImages.com/blog
Landscape, Portrait, Product Photography - Sydney, Australia

Hoggy

  • Full Member
  • ***
  • Online Online
  • Posts: 162
  • Never take life, or anything in it, too seriously.
Re: Best way to backup lightroom photos?
« Reply #12 on: February 06, 2017, 03:25:14 AM »

Points 2 and 3 if you live in Australia you can forget. I am on NBN which is much faster than you and it would take 5 weeks per TB to restore over the Internet. Just use multiple drives and keep one at a relatives.

Although if he uses smart previews, and backs those up as well - then one could restore those first, and still be able to continue working on them.  Meanwhile, he could restore the full files that he might need/want right away - and let the rest of the restore work in the background.  ... slow, but something to consider..


Quote
Yes, check integrity. I scan the folder structure of the backup and look for random changes I know that I have made. Hardly perfect but seems to work.

I don't mean to start a debate about 'to DNG or not'..  But as my sig states, that integrity checking built into DNG is very important to me - it has already saved several files that could've been lost forever, as corrupted copies propagated through several backup sets.  I now always do a 'validate DNG Files' before doing a full backup.

But do note that if your camera records native DNG's, that most if not all, DO NOT put the integrity hash in there.  And that LR used to 'reconvert' them when choosing 'copy as DNG', but no longer does so..  So I need to now do it after import in order to get that hash in there..
Logged
^^^ Many line feeds are refusing to stay in my posted messages on this site, for some reason. ^^^
Cams: Pentax K-3, K-30 & Canon G7X, S100
Firm supporter of DNG, throwing away originals.
It's the hash, man..  That good hash!

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1834
    • The Last Word
Re: Best way to backup lightroom photos?
« Reply #13 on: February 06, 2017, 11:18:04 AM »

If you only have 150G today then if you continue in photography then very soon you will have more data than will fit on an internal drive. I suggest that you move to an external drive sooner than later.

150 GB is less than 2% of the capacity of current large internal disk drives. In the past, I have found that the growth in single drive storage capacity pretty much paces the growth of my oeuvre. That may not continue with the transition to SSD (an oxymoron to be sure), but at 2% today, it can lag a lot and he'll still be able to fit the whole thing on one disk.

Jim

BobShaw

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1001
    • Aspiration Images
Re: Best way to backup lightroom photos?
« Reply #14 on: February 06, 2017, 05:04:36 PM »

150 GB is less than 2% of the capacity of current large internal disk drives. ... That may not continue with the transition to SSD
I moved to SSD with my last computer purchase 4 years ago. I would not buy a computer without SSD for the OS. It would be like having a V8 car with 4 spark plugs removed.
The next step is to retrofit my MBP.

Also if your data is on your internal drive then you can't use a different computer which is important to me.
These days with 32G or larger cards and 60MB images you will soon fill an internal. I have 4TB of data. Not all photos but documents, installers etc as well.
Logged
Website - http://AspirationImages.com
Blog - http://AspirationImages.com/blog
Landscape, Portrait, Product Photography - Sydney, Australia

N_N_Nikmat

  • Newbie
  • *
  • Offline Offline
  • Posts: 30
Re: Best way to backup lightroom photos?
« Reply #15 on: February 06, 2017, 05:54:07 PM »

Well for now I'm down to this........

1. Automated Time Machine backups (to permanently attached external usb drive)
2. Occasional (say monthly) backups of photos only (to a couple of portable usb drives one of which will be kept off site and these can be rotated)
3. Occasionally check integrity of backups

Now I'm feeling safe from everything except cosmic ray induced electromagnetic showers  :o
Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1834
    • The Last Word
Re: Best way to backup lightroom photos?
« Reply #16 on: February 06, 2017, 05:54:29 PM »

I moved to SSD with my last computer purchase 4 years ago. I would not buy a computer without SSD for the OS. It would be like having a V8 car with 4 spark plugs removed.
The next step is to retrofit my MBP.

Also if your data is on your internal drive then you can't use a different computer which is important to me.
These days with 32G or larger cards and 60MB images you will soon fill an internal. I have 4TB of data. Not all photos but documents, installers etc as well.

I use two workstations for image editing, and synch the drives if I want them the same. I use 1 and 2 TB SSDs for OS and apps, but use triple 8TB internal drives for data storage, with quad 8 TB and mirrored SSDs externally in addition.

Jim

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1834
    • The Last Word
Re: Best way to backup lightroom photos?
« Reply #17 on: February 06, 2017, 06:01:09 PM »

Well for now I'm down to this........

1. Automated Time Machine backups (to permanently attached external usb drive)
2. Occasional (say monthly) backups of photos only (to a couple of portable usb drives one of which will be kept off site and these can be rotated)
3. Occasionally check integrity of backups

Now I'm feeling safe from everything except cosmic ray induced electromagnetic showers  :o

Here's what I do.

1. Local backups to two sets of multiple 8 TB drives over USB3 using GoodSync.
1a. Monthly trips to the bank to swap 8TB drives with backup information on them.
2. Backups to Synology NAS boxes using GoodSynch and Vice Versa.
3. Server side triple replication of Synology data with Vice Versa.
4. Cloud backup of all data but images to AWS S3.

If I were you, I'd have at least two on-site backup copies, ideally backed up with different software.

Jim

bassman51

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 83
Re: Best way to backup lightroom photos?
« Reply #18 on: February 06, 2017, 08:29:14 PM »

Jim, your math is correct.  CrashPlan reported about 54M/sec during the restore.   Frankly, I was amazed, as I have a 50M/s plan from Cablevision and rarely experience that speed, let alone consistently over days.   

Rather than redundancy, my approach exphasizes diversity: diversity of location, diversity of hardware, diversity of software.  The only commonality is the user, Me.  In my case last week, I had a hardware problem with the primary drive and a software (or configuration - same thing) problem with Time Machine.  Since CrashPlan was unrelated in anyway to either of the former, it survived and saved my butt. 

As of today, all three - the new primary dive, the onsite Time Machine backups, and the offsite CrashPlan backups seem to be ok.  But a new problem has arisen, with the CrashPlan app on my Mac refusing to run without crashing.   I've opened a trouble ticket with the vendor.  I suspect - but can't prove on my own - that there's a queue overflow associated with the 3TB+ backup it's trying to synchronize. In the meantime it appears to be running the backup itself in the background.   
Logged
Life is what happens when you're busy making other plans.

ericbowles

  • Newbie
  • *
  • Offline Offline
  • Posts: 28
    • Bowles Images
Re: Best way to backup lightroom photos?
« Reply #19 on: February 07, 2017, 04:51:19 PM »

There is a serious move right now that is going to change the way we think about storage and backups.

There are several issues involved.  You want to have a place to store your files - and versions if needed.  You need a backup for those files - and usually an offsite backup as well.

Your configuration usually needs to provide for versioning just to be sure you don't overwrite or discard something critical.  In the case of Lightroom, you need to backup your catalog and protect your catalog at least as well as your images with multiple backups, off site protection, and versions.

Hard drives on computers are increasingly moving to solid state drives.  They are a lot smaller, and increasingly it is hard to find anything larger than 500 GB.  The cost ramps up quickly as you move above 250GB.  Computers are getting smaller and lighter, and large internal spinning drives with related fans don't fit that model.  SSD is cooler, smaller, faster, and has better battery life. 

The assumption is that you are using external storage - generally involving the cloud.  Cloud storage will continue to improve but also be more specialized.  There are challenges with cloud services.  Soem of them don't permit adding only incremental files - you need to upload everything.  Most are relatively slow limited by data speeds at the provider.  And many cloud services are poor when it comes to downloading or recovering just a few files.  A large download representing a total recovery could take months with some services.

For now, we still need to rely on external local storage.  There are a number of configurations - stand alone drives, RAID, etc.  The key is to have multiple copies of the image files that can be accessed and used.  At a minimum, you should have two backup copies onsite plus an offsite copy.  For critical files, I'd add a cloud copy that includes critical files and your Lightroom catalog backup.

Without question, your computer will eventually fail, and external drives will fail.  My experience is three computer hard drive failures and two hard drive failures over the last 10 years. 

The weakest point in most backup plans is the person.  Deleting files, losing files, not being able to find files, writing over files - they all happen.  You might have a great backup plan but not be diligent about executing it.
Logged
Eric Bowles
Nature Photography
Workshops and Instruction
Pages: [1]   Go Up