Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: KevinA on November 19, 2015, 03:47:29 am
-
New to LR after Aperture.
I am dedicating a large drive to LR, I will be adding identical drives for backup. Which software for backing up the drives are preferred? I was using the Aperture vaults, very simple so not easy to get it wrong.
I'm hoping there is something easy and simple. Suggestions please, thanks.
-
Carbon Copy Cloner
https://bombich.com
-
Carbon Copy Cloner
https://bombich.com
+1
-
I am assuming you are on a Mac so TimeMachine is all you need.
-
I am assuming you are on a Mac so TimeMachine is all you need.
Yes, and no. Certainly Time Machine can back up your images, but it is in a compressed format that in the event of a problem you need to reconstruct the files and folders from the TM archive.
Conversely, if you use Carbon Copy Cloner, Super Duper or other such offerings that create a cloned copy of the original drive ... In the event of a drive failure, you can simply point Lr to the cloned drive and continue working without breaking stride. Lr can't directly read or access a TM backup, so you would be down, until you reconstruct the desired file/folder structure. Which is no big deal for a small library, but if your assets number in the tens of thousands, that can be a lot of valuable time expended.
I even keep a CCC clone of my startup drive, as well as a TM backup for extra protection, but for my 12TB image/video archives, I only use CCC.
-
+1 to what ButchM wrote. TM is great and has saved my butt but it's not something you want to use alone. Like him, I do not include the dedicated drive I keep all LR data on for any Time Machine backups. Time Machine isn't all you need. CCC, SuperDuper or ChronoSync are all better for cloning all that data (including presets stored with the catalog) to another dedicated drive.
-
Yes, and no. Certainly Time Machine can back up your images, but it is in a compressed format that in the event of a problem you need to reconstruct the files and folders from the TM archive.
I'm not clear what you mean by this. Time Machine backups are not "compressed." They're just the files. However, there are some UNIX "tricks" used (specifically, hard links), to keep files from being stored every single time they're backed up, if they don't change. But this doesn't involve compression.
The easiest way to restore from Time Machine is just to use the menu icon and restore to a specific point in time. If for some reason you don't want to do this, you can navigate to the backup location and go to the folder called "latest" which will point you to the directory that looks exactly like the last backup you made. You could copy that directory to the restore location and you're good to go.
Carbon Copy Cloner (or its nearly identical analogue, SuperDuper!) are conceptually simpler in that they're just a straight "clone" of your drive. The big downside are that they're mainly manual processes, and so it's highly likely people using them may have to revert to a backup that's days or weeks old. "Do it every night" is a lot of extra work over Time Machine, for no true benefit other than simplicity if you don't want to learn.
I'd also recommend an offsite backup. On OS X, I've found BackBlaze to be quite good.
-
I'm not clear what you mean by this. Time Machine backups are not "compressed." They're just the files. However, there are some UNIX "tricks" used (specifically, hard links), to keep files from being stored every single time they're backed up, if they don't change. But this doesn't involve compression.
It's kind of a PITA to find data to recover from TM depending on how far back it goes. That's also the beauty unlike a clone, you can go back multiple 'backup's' and get data that would be over written using a product like SuperDuper. So that begs the question. If you have one image you've edited 20 times in the last 2 weeks, do you want or need each iteration? If so, TM is useful but of course will take a lot more HD space for all of this. On the other hand, you have all your LR data on a dedicated drive that you clone after you work on important files or like me, automatically every night, is that sufficient (answer, yes). I can clone that drive to as many others as I wish, rotate them in differing locations etc. TM isn't the same process. So when folks say 'just use TM' it appears they don't understand the significant differences between and ramifications of it's backup schema versus a clone. You can have both. You can have one. But consider the differences, they are extreme.
-
The big downside are that they're mainly manual processes, and so it's highly likely people using them may have to revert to a backup that's days or weeks old. "Do it every night" is a lot of extra work over Time Machine, for no true benefit other than simplicity if you don't want to learn.
No extra work whatsoever. CCC can (and does) run as a background process, schedule anything from monthly to hourly intervals absolutely NO manual intervention required - set it and forget it - together with 'safety net' incremental file retrieval (if you need it).
If the OP is serious about Lr backup, then I suggest also looking at TPG Lr Backup.
http://www.photographers-toolbox.com/products/mdawson/tpglrbackup/
http://thephotogeek.com/quick-easy-lightroom-backup/
-
Is Carbon Copy Cloner better to use than Super Duper? If so why. At the moment I manually backup using SuperDuper.
-
Is Carbon Copy Cloner better to use than Super Duper? If so why. At the moment I manually backup using SuperDuper.
I can't comment on Super Duper, I've never used it. But I can attest to CCC, both the program and the support you'll get from Mike Bombich which is nothing short of stellar. In it's most simple format just point to two disks, one source, one backup and forget about it.
If you're manually backing up - then yes, it is definitely better.
There's a free 30-day trial, and frankly, it's worth checking out.
-
I can't comment on Super Duper, I've never used it.
And I can't fully comment on CCC because it's been years since I used it. Free in the old days. It did the job as does SuperDuper. I like SuperDuper because it's so damn simple, and inexpensive. But I switched for my daily backups to ChronoSync because it has more options including data verification. It's a lot more 'difficult' (involved) to setup than SuperDuper because it offers more options that may or may not be important to you. In the end, any of these products will get the job done pretty well. I still use SuperDuper for quick and simply disk cloning but ChronoSync for daily unattended backup's with verification. I have no reason to suggest you move away from SuperDuper unless you require more options.
-
... but ChronoSync for daily unattended backup's with verification.
... with CCC verification option enabled, " CCC will calculate an MD5 checksum of every file on the source and every corresponding file on the destination, then uses these MD5 checksums to determine if a file should be copied and verifies the integrity of the files on your destination volume before files are copied "
https://bombich.com/kb/ccc3/advanced-settings
Looks as though these two are very similar.
-
I'm not clear what you mean by this. Time Machine backups are not "compressed." They're just the files. However, there are some UNIX "tricks" used (specifically, hard links), to keep files from being stored every single time they're backed up, if they don't change. But this doesn't involve compression.
Maybe not 'compression' in the truest sense ... but how do you explain that a TM backup can contain more data than the actual volume of the drive it is saved to?
The big downside are that they're mainly manual processes, and so it's highly likely people using them may have to revert to a backup that's days or weeks old. "Do it every night" is a lot of extra work over Time Machine, for no true benefit other than simplicity if you don't want to learn.
It's no 'extra work' at all. Although all my CC backups are capable of being invoked manually, all of my CCC backups are scheduled and performed automatically at the time and days I specify. Nothing could be simpler ... the setup for scheduling is quite easy.
I also like the feature that when I do my periodic backup to my drives that are stored off site ... CCC recognizes the drive is online and automatically performs the update/backup when the drive is inserted into the dock.
Nothing could be simpler.
-
Is Carbon Copy Cloner better to use than Super Duper? If so why. At the moment I manually backup using SuperDuper.
I can't comment on Super Duper either. Haven't looked at it in recent years. I only mentioned it as I had looked at it previously and it was the only otters option I was aware of to consider for the chore.
I, like many others, have been using CCC for a very long time because it was a complete and full-featured options that was also free. It worked so well and Mike's customer service is second to none that when he started charging, it was a no-brainer to send some currency his way. No regrets.
-
New to LR after Aperture.
I am dedicating a large drive to LR, I will be adding identical drives for backup. Which software for backing up the drives are preferred? I was using the Aperture vaults, very simple so not easy to get it wrong.
I'm hoping there is something easy and simple. Suggestions please, thanks.
crashplan from code42 (can do local backup too - free, not only to cloud)
-
Yes, and no. Certainly Time Machine can back up your images, but it is in a compressed format that in the event of a problem you need to reconstruct the files and folders from the TM archive.
Not correct at all. For individual files or folders it is drag and drop from the "latest" or however far back you need to go. Also not only does it back up images, it can back up documents, settings, users, OS, etc and restore onto a different hardware machine.
-
Not correct at all. For individual files or folders it is drag and drop from the "latest" or however far back you need to go. Also not only does it back up images, it can back up documents, settings, users, OS, etc and restore onto a different hardware machine.
It comes down to scale and the time it can take to restore from a backup.
Sure ... when restoring or reverting to a single previous iteration of an image, TM can be handy. But what do you do in the event of a catastrophic failure where you lose everything on the drive in question due to hardware failure?
CCC backs up all files on the drive as well ... it doesn't care what type individual files are ... CCC, Super Duper creates complete exact duplicates of a drive ... even a bootable startup drive ... can you boot from your TM backup drive? Nope. That's why I use both options for my startup drives. TM for recovering inadvertently deleted or corrupted files ... and a CCC clone, so I can continue working without any delay.
What if you need to 'drag-n-drop' 3TB of images from your TM backup because the working drive failed? How long does it take to restore that much data? With a clone, you simply mount the drive, point Lr to the backup drive drive and you are instantly up and running and can rebuild the failed drive at your convenience.
With TM, that recovery period is much longer until filled folder structure is restored.
-
Another reason to have at least one backup that is not physically connected to your system: ransomware that encrypts files on attached drives. See Brian Krebs, “Ransomware Now Gunning for Your Web Sites” http://krebsonsecurity.com/2015/11/ransomware-now-gunning-for-your-web-sites/#more-32761
-
Maybe not 'compression' in the truest sense ...
Time machine doesn't use "compression" in any sense.
but how do you explain that a TM backup can contain more data than the actual volume of the drive it is saved to?
It can't. If you think it can, you don't understand the concept of a Unix hard link.
Jeremy
-
It can't. If you think it can, you don't understand the concept of a Unix hard link.
Jeremy
Perhaps ... I don't understand. Once again ... explain how, for example that the machine I am typing from at this very moment, the TM backup for the startup drive dates back to Sept. 20, 2015. The startup drive is only a 500GB HD. The TM backup drive is likewise a 500GB HD.
Yet, If I were to go back and restore all available files that are available from my TM backup since Sept20, the total volume of data would certainly exceed 500GB. Especially when you consider all the shared files that have come and cone in my workgroup Dropbox folder over the past two months far exceeds the maximum uncompressed storage volume available to this computer.
If there is no 'compression' occurring ... then how is it possible to retain this amount of data? Not to mention multiple iterations of the same file can cumulatively measure above and beyond the space required to store the most recent version of said file.
So please, explain away, I'm always willing to learn.
-
New to LR after Aperture.
I am dedicating a large drive to LR, I will be adding identical drives for backup. Which software for backing up the drives are preferred? I was using the Aperture vaults, very simple so not easy to get it wrong.
I'm hoping there is something easy and simple. Suggestions please, thanks.
Kevin,
Let me explain my total backup procedure so we are not just addressing part of the problem.
My system setup is as follows: One machine which is a MacBook Pro (MBP) with 1TB internal drive. On that drive are my documents, e-mails, applications. The Lightroom catalog and previews are stored here as well. The most recent photos from shoot are there as well. Older photos as well as documents etc. is stored on a single external LaCie Thunderbolt connected drive (3TB in total and mirrored (aka. Raid-1)). I regularly make Lightroom backups which is not really backup as such, but more a consistent copy of the catalog provided that testing integrity before backup is checked. I delete all but the latest backup from the backup folder. The backups are backed up on the TM backups.
I'm travelling a lot doing my photo workshops and the backup strategy is taking this into consideration. The backups are split between the MBP (and the external drive.
The MBP is backed up in my office on two external USB 3.0 drives (2 and 3 TB capacity) using Time Machine (TM). I daily (or sometimes every second day) attach the drives to my machine and run the TM backups. There are alternatives to TM. I have used TM since 2009 without a problem so continue to use it.
The external LaCie drive is backed up on two USB 3.0 drives (not the same as the TM drives). I use ChronoSync http://www.econtechnologies.com to synchronize folders on the LaCie drive to each of the two other drives and I have templates (synchronizer documents) for doing this. There are other solutions available that does the same thing.
For disaster backup (a backup independent of all other backups and which take in consideration physical disasters like a fire and theft in my home) I use Backblaze https://www.backblaze.com. Here my MBP and the external LaCie drive is backed up. There are other solutions available. When I return from a trip it takes about a day before all is backed up again. I have a 100Mbps symmetrical internet connection based on fiber into the building. Backblaze can use the entire bandwidth with 10 backup threads configured with no throttling.
For travel I bring two 2TB portable drives for TM backup. During my trip I will every day do the TM backup on the two drives. During my travel the online backup is disabled as typically the WiFi systems in hotels does not have capacity to backup the possibly more than 100GB that I shoot during a trip.
If during a trip I loose everything, I can go back to the TM backups in my office and get a new MBP and restore everything as it was when I left for the travel. If I loose the MBP only I can restore one of the two portable TM backups. The two backups should be kept separate so not both are lost in case of theft.
Why two of every backup? The reason is simple: I do not like having only one copy left of the primary copy is lost. If only one copy is left after failure this one could also fail before restore on a new drive or MBP.
Clearly it is possible to do something similar with different products and also using Windows, of course.
-
If there is no 'compression' occurring ... then how is it possible to retain this amount of data? Not to mention multiple iterations of the same file can cumulatively measure above and beyond the space required to store the most recent version of said file.
http://icanhelpyourmac.com/tip_archive/understanding_apples_time_machine/index.html (http://icanhelpyourmac.com/tip_archive/understanding_apples_time_machine/index.html)
An important fact to remember is that a file is not "backed up" unless it is stored on two hard drives at the same time. If you delete the original file off your computer's internal HD it is no longer backed up. Yes, the file does still exist on the Time Machine HD, but if the Time Machine HD would fail in any manner there is no longer a copy on your Internal HD. In this way, your Time Machine HD is also a Storage HD, but don't rely on it as such.With the above sentences in mind it is important to remember that since Time Machine will never delete the final copy of any file this is created on your Time Machine HD, your Time Machine HD may fill up very quickly unless steps are taken to reduce the number or size of files that are created, renamed or moved. For instance... if you create a file called "A.doc" on your desktop Time Machine will create a file called "A.doc" on your Time Machine HD. If you simply rename this file to "A1.doc" Time Machine will now permanently store a copy called "A.doc" and another file called "A1.doc".
This is not a problem when it comes to Word files or any text file in general as their sizes are small enough you likely would not notice the backup HD space being affected. Conversely, if you are working with an Adobe Photoshop file (.psd) that is 300 MB in size you Time Machine HD will likely fill up very quickly. Each hour you work on the file and alter it Time Machine will back up that hour's version of the 300 MB file. Thusly if it takes 4 hours to complete the file, Time Machine could have created well over 1 GB worth of backups. My suggestion is to use a folder called "In Progress" and another folder called "Archive". When you work on files save them only into the "In Progress" folder. Only once complete should the file be moved to the "Archive" folder. The reason this can solve the problem is because you can then simply delete the "In Progress" folder from your Time Machine HD. Doing so will delete all files and all copies of files from within the "In Progress" folder and then the next hour Time Machine will recreate the "In Progress" folder with it's new contents.
Here's why using TM on image files might not be such a good idea.
-
http://icanhelpyourmac.com/tip_archive/understanding_apples_time_machine/index.html (http://icanhelpyourmac.com/tip_archive/understanding_apples_time_machine/index.html)
Here's why using TM on image files might not be such a good idea.
Yes, but only a problem if the TM drive is constantly connected. Or just have a drive big enough so it's not really a problem. If the amount of data being backed up by TM is say 500GB, a 2-3TB TM drive would not fill up fast even with PSD files being updated and hourly backups are consolidated after 24 hours, daily backups for a month and weekly backups older than a month until the volume runs full and the oldest backup gets deleted. But, of course, as you point out, one need to be aware of this.
-
Yes, but only a problem if the TM drive is constantly connected.
Sure but not having it connected kind of defeats it's purpose of 'backing up' all kinds of data while you work. I've got email going all the time, TM is really slick for recovering them if I delete some mail and change my mind.
Or just have a drive big enough so it's not really a problem.
An easy answer but not always possible. And with TM, eventually something has to get deleted.
If the amount of data being backed up by TM is say 500GB, a 2-3TB TM drive would not fill up fast even with PSD files being updated and hourly backups are consolidated after 24 hours, daily backups for a month and weekly backups older than a month until the volume runs full and the oldest backup gets deleted. But, of course, as you point out, one need to be aware of this.
Maybe not but the point I attempted to make is to 'dismiss' the idea that TM is just like other backup's, it's not and that TM is adequate for all backup needs, it's not. The URL points out how many of the files copied are not a backup any longer as only one copy exists.
I love TM, I encourage people to use it. It's not the same as a daily clone using the products discussed. Use both systems or more, but understand that TM and cloning are not remotely the same schema.
-
Sure but not having it connected kind of defeats it's purpose of 'backing up' all kinds of data while you work. I've got email going all the time, TM is really slick for recovering them if I delete some mail and change my mind.
An easy answer but not always possible. And with TM, eventually something has to get deleted. Maybe not but the point I attempted to make is to 'dismiss' the idea that TM is just like other backup's, it's not and that TM is adequate for all backup needs, it's not. The URL points out how many of the files copied are not a backup any longer as only one copy exists.
I love TM, I encourage people to use it. It's not the same as a daily clone using the products discussed. Use both systems or more, but understand that TM and cloning are not remotely the same schema.
The recommendation not to use TM because of frequently updated large files is my point. Either you don't have the TM connected all the time or you have a large enough TM backup drive and have two of them is my recommendation. Even if you had 20 1GB PSD files being updated every hour for a working day 10 hours you would have at the most 10*20GB=200GB of data which gets down to 20GB after 24 hours. This should be clear for those who use TM and get this disrecommendation of it's use.
I do agree it's important to understand it's use and limitations. The thing about deleted files goes away with two TM drives connected which would be a good idea anyway. Never have only one copy of your data.
-
The recommendation not to use TM because of frequently updated large files is my point. Either you don't have the TM connected all the time or you have a large enough TM backup drive and have two of them is my recommendation.
3rd option, what I use: storing all my image data on a separate drive and tell TM not to touch it. Then tell the cloning software (and for me CrashPlan) to back it up every night. TM is dedicated to my boot disk and all those important files. Large image files are not touched but backed up at least once a day. As such, my 2TB TM drive has files going back 2 years. IF I included big images files, that wouldn't be the case. Now one can ask, do you need to go back 2+ years to find some old file? I've gone back many months but no, not two years. But that data is available if I had to. It wouldn't if I included big image files. But whatever works for you.
-
3rd option, what I use: storing all my image data on a separate drive and tell TM not to touch it. Then tell the cloning software (and for me CrashPlan) to back it up every night. TM is dedicated to my boot disk and all those important files. Large image files are not touched but backed up at least once a day. As such, my 2TB TM drive has files going back 2 years. IF I included big images files, that wouldn't be the case. Now one can ask, do you need to go back 2+ years to find some old file? I've gone back many months but no, not two years. But that data is available if I had to. It wouldn't if I included big image files. But whatever works for you.
Well, one can argue about that, but the consolidation to one file per day after 24 hours avoids most if not all of the problem. Anyway one could just buy a bigger drive. But whatever works is the important part and know how it works.
-
I'm hoping there is something easy and simple ...
A couple of pages on and me thinks the OP has done a runner ... :)
As we've touched upon RAID's I'd point out that for those on OS X, Apple has stripped RAID support from their disk utility on El Capitan. OWC have SoftRAID , which they've now produced in a LITE version ($34). Well worth the investment for two reasons: (1) they're is constant real-time monitoring of all drives on the system in the menu bar (via an LED style bulb) which will alert you to any persistent errors in writing to disk - usually ample warning before a disk fails, and (2) real time logging of hours each individual disk's hours of use, i/o errors and reallocated sectors.
-
Well, one can argue about that, but the consolidation to one file per day after 24 hours avoids most if not all of the problem. Anyway one could just buy a bigger drive. But whatever works is the important part and know how it works.
If an argument is about the consolidation of all image files, LR database etc on one drive, I'll point out some of the advantages:
These are probably the largest files you'll create so as you continue to shoot and edit iterations, having the biggest drive you can find and consolidating them in one place makes sense. Why clog up disk space with MS word files?
Cloning this one drive to as many drives as you desire means you can pick one up, take it on location and have all your data with you. As you add images from location, it's a snap to clone the original drive, backing it up and ensuring the desktop now has the new files.
With the LR catalog on this drive, anyone with a copy of the application, Mac or Windows can access all the photo's as they also reside on this drive.
As I stated, it's super easy to turn off TM for this one drive.
Multiple cloned drives can be rotated. Off site, in a fireproof safe, on your desktop machine and for location work.
Cloning with the applications mentioned is fast, only new or altered data is written.
Off hand, none of the above is possible or easy IF the backup schema is solely TM.
-
A couple of pages on and me thinks the OP has done a runner ... :)
As we've touched upon RAID's I'd point out that for those on OS X, Apple has stripped RAID support from their disk utility on El Capitan.
There are mixed views as to whether Apple 'stripped ' RAID support from DU ... or if it hasn't been added yet. Disk Utility was completely rewritten for El Cap and the RAID portion may come back at a later date as that project moves along.
I reference this point that the RAID function still exists in El Cap as it can be invoked via Terminal ... it just isn't available in the Disk Utility UI.
-
Thanks all. At present I'm running Aperture and LR while I make the transition.
I already back up my computers HD with TM, I didn't fancy using TM for the library. I don't store much on the laptop, not even the LR catalog as its a flash drive and space is not great, I wouldn't buy with such little space again (250gb).
My LR dedicated thunderbolt drive also holds the catalog, I will use one of the mentioned programs to make two clones of the LR drive. One backup is never enough and if I do get a major problem, I NEVER plug in a backup until I know what the problem is. I'm thinking if the catalog is on the same drive as the images whichever computer I use the catalog is staying consistent .
Also a large selection of my work gets uploaded to my site as high res jpg, so its 2 1/2 backups in reality I have.
Thanks again.
-
Also I have my library insured for quite a lot incase for some reason everything vanished....in fact if that happened I would take the money and run :-)
-
Perhaps ... I don't understand. Once again ... explain how, for example that the machine I am typing from at this very moment, the TM backup for the startup drive dates back to Sept. 20, 2015. The startup drive is only a 500GB HD. The TM backup drive is likewise a 500GB HD.
Yet, If I were to go back and restore all available files that are available from my TM backup since Sept20, the total volume of data would certainly exceed 500GB. Especially when you consider all the shared files that have come and cone in my workgroup Dropbox folder over the past two months far exceeds the maximum uncompressed storage volume available to this computer.
If there is no 'compression' occurring ... then how is it possible to retain this amount of data? Not to mention multiple iterations of the same file can cumulatively measure above and beyond the space required to store the most recent version of said file.
So please, explain away, I'm always willing to learn.
The link that Andrew posted is quite helpful, if a bit superficial. The crucial thing is that when it makes what is apparently an image of the entire disk, what it is actually doing is duplicating only the directory structure of the disk and those files which have changed.
Every file has an entry in at least one directory which points at it. It also has a reference count, which is the number of directories in which it appears. For an existing, unchanged file, all Time Machine has to do is to create an entry in a new directory and increment the reference count. However large the file, that takes only a few bytes. Some changes had to be made to the Unix file system to allow hard links to point at directories, and a good deal of care was needed to avoid circular directory references.
When a file is deleted, its reference count is decremented and if it is now zero, the space the file occupies is returned to the free list.
That also a superficial explanation, but I hope it helps. I used to have a link to a very comprehensive article, but I can't find it now.
Jeremy
-
I may have missed something as I read all the interesting and useful posts in this thread. The OP said he wanted to back up Lightroom. As he is coming from Aperture, about which I know absolutely nothing, I suspect but am not certain he recognizes the difference between the LR catalog (database) and the actual image files which can be located anywhere one wishes. It is critical to back up the image files as well as the catalog. I keep my LR catalog on the SSD of my Mac Pro but keep the image files, the actual photographs are on a different drive in an OWC Thunderbay array. I use TM and CCC for the SSD and use CCC for the image files, including an off site backup. Lots of great information about backup strategies in this thread, thanks to all.
-
The link that Andrew posted is quite helpful, if a bit superficial. The crucial thing is that when it makes what is apparently an image of the entire disk, what it is actually doing is duplicating only the directory structure of the disk and those files which have changed.
Every file has an entry in at least one directory which points at it. It also has a reference count, which is the number of directories in which it appears. For an existing, unchanged file, all Time Machine has to do is to create an entry in a new directory and increment the reference count. However large the file, that takes only a few bytes. Some changes had to be made to the Unix file system to allow hard links to point at directories, and a good deal of care was needed to avoid circular directory references.
When a file is deleted, its reference count is decremented and if it is now zero, the space the file occupies is returned to the free list.
That also a superficial explanation, but I hope it helps. I used to have a link to a very comprehensive article, but I can't find it now.
Jeremy
That doesn't explain how my TM backup actually contains more data than the drive can store ... even though my backup TM drive is the same size as the drive being backed up.
-
That doesn't explain how my TM backup actually contains more data than the drive can store ... even though my backup TM drive is the same size as the drive being backed up.
It doesn't. It can't. Why do you think that it does?
Jeremy
-
Time Machine looks to me to be very similar if not identical to the standard rsnapshot/rsync mechanism (http://rsnapshot.org/) available on many Linux and UNIX systems. As is so often the case, Apple seems to have taken an open-source tool, modified it to work within Apple's version of the UNIX operating system, and made it easier to use.
(I use rsnapshot on a pair of Linux systems as my second-level back-up, after Time Machine, for my images and Lightroom catalogs. I have a script that copies any changes in the image and catalog hierarchies to a general-purpose Linux file server once a day—the location I use for LR imports is updated once a minute—and rsnapshot backs up the file server to a dedicated Linux back-up server once an hour.)
-
Eric, I'm well aware of LR stupid none baqckup solution. If they can create an option to backup the catalog why can't they come up with an integrated back up for the important things.
There are many things in LR I prefer the dam side of it isn't one. Clunky to say the least.
-
If they can create an option to backup the catalog why can't they come up with an integrated back up for the important things.
Such as ?
1- Create a catalog backup directory
2- Create a sub-directory 'lr-config' ( for all your configuration files, dcp, presets, etc etc , of which there will be plenty)
3- Dig deep and invest in donation ware :
http://www.photographers-toolbox.com/products/mdawson/tpglrbackup/
http://thephotogeek.com/quick-easy-lightroom-backup/
All your Lr program files will be in one place - up to you how and where you back up your image files.
-
There are mixed views as to whether Apple 'stripped ' RAID support from DU ... or if it hasn't been added yet. Disk Utility was completely rewritten for El Cap and the RAID portion may come back at a later date as that project moves along.
Possible, but I doubt it.
More likely that Apple have seen the way ahead, much as they did when they stopped installing internal dvd drives. With the advent of distributed computing, fast SSD's, efficient backup programs - the need for a RAID-0 or RAID-1 is no longer mainstream. Users who want/need RAID with TB2 speeds will more likely go for RAID-4 or 5 setups , most with hardware controllers and sophisticated , not to say potentially troublesome, software which is beyond the scope of the OS. Apple approved Pegasus RAIDs being a recent example.
-
Eric, I'm well aware of LR stupid none baqckup solution. If they can create an option to backup the catalog why can't they come up with an integrated back up for the important things.
There are many things in LR I prefer the dam side of it isn't one. Clunky to say the least.
The backup of the catalog in Lightroom is (as I mentioned in my post earlier on explaining my total backup method) is NOT a backup in the sense of a backup. You need to backup your entire system including the catalog and image files. The word backup is bad term for what it really is: A consistent copy of the Lightroom catalog (=database) at a given point in time.
-
Thanks all. At present I'm running Aperture and LR while I make the transition.
I already back up my computers HD with TM, I didn't fancy using TM for the library. I don't store much on the laptop, not even the LR catalog as its a flash drive and space is not great, I wouldn't buy with such little space again (250gb).
My LR dedicated thunderbolt drive also holds the catalog, I will use one of the mentioned programs to make two clones of the LR drive. One backup is never enough and if I do get a major problem, I NEVER plug in a backup until I know what the problem is. I'm thinking if the catalog is on the same drive as the images whichever computer I use the catalog is staying consistent .
Also a large selection of my work gets uploaded to my site as high res jpg, so its 2 1/2 backups in reality I have.
Thanks again.
If your Lightroom environment (catalog and image files) gets used on several computers it's a good idea to have the catalog on the same drive as the image files. I overcame this problem by having only one machine powerful and a large enough SSD of 1TB. For me this was the best solution since I travel a lot and want to take my machine with me with all essential information and the latest photos which I can work on the road when time allows. I do also bring a copy of my image files on two 2.5" USB 3.0 drives if I need access to older files. I have always a top level folder on each drive that holds lots of shoot folders underneath so I can switch the top level folder within Lightroom to point to a different drive. I will usually not modify any content of the image folders on the drives with the copy, but I could synchronize back to the main folder back in office if I needed to make modifications.
-
It doesn't. It can't. Why do you think that it does?
Jeremy
Told you before. My TM backup goes back 2 months ... If I restored files from the past that have been moved/deleted in that span ... the total data restored would far exceed the total capacity of my startup drive.
-
Another thing to consider when backing up Lr is your presets, configuration files and plugins. If something goes south recreating our Lr environment can be a significant pain in the backside.
A tool to consider in your backup arsenal is one that's dead simple to run (automatic) and easy to set up. It's a Lr plugin.
http://www.photographers-toolbox.com/products/mdawson/tpglrbackup/
-
Told you before. My TM backup goes back 2 months ... If I restored files from the past that have been moved/deleted in that span ... the total data restored would far exceed the total capacity of my startup drive.
There's no "quart into pin pot" magic, Butch. Time Machine doesn't use any form of compression and the disk simply can't hold more data than there is space on the disk.
Time Machine looks to me to be very similar if not identical to the standard rsnapshot/rsync mechanism (http://rsnapshot.org/) available on many Linux and UNIX systems. As is so often the case, Apple seems to have taken an open-source tool, modified it to work within Apple's version of the UNIX operating system, and made it easier to use.
No, Chris, it's completely different from rsync. I've found the rather more technical article on the workings of Time Machine that I mentioned earlier: it's here (http://arstechnica.com/apple/2007/10/mac-os-x-10-5/14/). The technical bit starts about half way down. It's old but I don't think the fundamentals of Time Machine have changed. If you want a user-friendly GUI on top of rsync (and God knows most of us need something to make that program's options intelligible), try Carbon Copy Cloner.
Jeremy
-
No, Chris, it's completely different from rsync. I've found the rather more technical article on the workings of Time Machine that I mentioned earlier: it's here (http://arstechnica.com/apple/2007/10/mac-os-x-10-5/14/). The technical bit starts about half way down. It's old but I don't think the fundamentals of Time Machine have changed.
The Ars Technica description sure reads like a description of rsnapshot to me, with the exception of the directory-link hack. (As you probably are aware, rsnapshot uses rsync as a transport to update remote files, but provides quite different functionality.) And when I browse the filing hierarchies created by Time Machine and rsnapshot, respectively, the directory links maintained by the former are the only distinction I can detect. Moreover, they would probably be easy to add to rsnapshot—it's just a Perl script, after all, albeit a fairly elaborate one—if Linux and mainstream UNIX filesystems supported hard links to directories the way OS X does. It's also suggestive that rsnapshot was first publicly documented in 2004 and Apple introduced Time Machine in 2007; post hoc of course doesn't imply propter hoc, but I doubt Apple independently invented that particular wheel. I don't know for certain that OS X uses the rsnapshot Perl script, per se. The Apple developers may have come up with their own implementation. But as far as I can tell both the techniques employed and the resultant back-up hierarchies are identical, modulo the directory-link feature of the Apple filesystem.
-
The Ars Technica description sure reads like a description of rsnapshot to me, with the exception of the directory-link hack. (As you probably are aware, rsnapshot uses rsync as a transport to update remote files, but provides quite different functionality.) And when I browse the filing hierarchies created by Time Machine and rsnapshot, respectively, the directory links maintained by the former are the only distinction I can detect. Moreover, they would probably be easy to add to rsnapshot—it's just a Perl script, after all, albeit a fairly elaborate one—if Linux and mainstream UNIX filesystems supported hard links to directories the way OS X does. It's also suggestive that rsnapshot was first publicly documented in 2004 and Apple introduced Time Machine in 2007; post hoc of course doesn't imply propter hoc, but I doubt Apple independently invented that particular wheel. I don't know for certain that OS X uses the rsnapshot Perl script, per se. The Apple developers may have come up with their own implementation. But as far as I can tell both the techniques employed and the resultant back-up hierarchies are identical, modulo the directory-link feature of the Apple filesystem.
I didn't comment on rsnapshot. It doesn't really matter which utility is used under the skin to move bytes from one disk to another, or even which utility lies on top. The directory-link feature is hugely important to TM and is the reason why it would be impossible to re-create it on any current non-Apple Unix system. It couldn't be added to rsnapshot because it's not a feature of the application doing the copying: it needs to be supported by the filesystem and for very good reasons (they're dangerous) you simply can't create hard links to directories in non-Apple Unix.
(http://imgs.xkcd.com/comics/porn_folder.png)
Jeremy
-
Told you before. My TM backup goes back 2 months ... If I restored files from the past that have been moved/deleted in that span ... the total data restored would far exceed the total capacity of my startup drive.
No, it wouldn't. This is nonsensical.
What is the size of the most recent backup on Time Machine? That's all you need to look at, and it's all that would be restored.
-
Either you don't have the TM connected all the time or you have a large enough TM backup drive and have two of them is my recommendation.
FYI about not having TM drive connected at all times:
http://www.macworld.com/article/3007573/operating-systems/what-to-do-when-mobile-time-machine-backups-linger-and-fill-storage-space.html
After reading the case of the missing El Capitan hard disk space and following all the advice in that column, Jim Williams still had an inexplicable 300GB that had no reason to exist. He ran a disk analysis program that showed that a hidden Unix directory named .MobileBackups.trash What is it and can he get rid of it?
This is a side effect of Time Machine, of all things. When one of your Time Machine targets is a drive that isn’t currently connected to your Mac, the backup system will continue to generate system snapshots up until all but 20 percent of drive storage is filled. After that point, Time Machine starts to delete snapshots and is more aggressive if you have very little storage available (less than 10 percent of drive capacity or less than 5GB).
-
In case anyone is interested I went the CCC route.
Thanks everyone for the help.
-
A couple of pages on and me thinks the OP has done a runner ... :)
As we've touched upon RAID's I'd point out that for those on OS X, Apple has stripped RAID support from their disk utility on El Capitan. OWC have SoftRAID , which they've now produced in a LITE version ($34). Well worth the investment for two reasons: (1) they're is constant real-time monitoring of all drives on the system in the menu bar (via an LED style bulb) which will alert you to any persistent errors in writing to disk - usually ample warning before a disk fails, and (2) real time logging of hours each individual disk's hours of use, i/o errors and reallocated sectors.
Do you use the Lite or the full version of SoftRaid? Any major benefits to the full version for the majority of photographers?
-
Like some others above, Carbon Copy Cloner for backing up everything, plus Time Machine for a supplementary copy of the start-up disk. Wish Time Machine were an inclusive rather than exclusive process for selecting the drives to back up, but I guess Apple is trying to accommodate the least sophisticated users, which is understandable.
-
In case anyone is interested I went the CCC route.
Thanks everyone for the help.
Pleased to see that at least one thread proved helpful to the OP!
Did you also have a look at TPG LrBackup for all you config files and settings ?
-
Do you use the Lite or the full version of SoftRaid? Any major benefits to the full version for the majority of photographers?
David -
SoftRAID Lite.
The 'lite' version covers any number of RAIDs 1 and 0.
There's no need for the full version unless you're intending to move to a RAID 4, 5 or 10 setup.
The SMART reporting, "hours-of-use" monitor, block level health analysis and predictive failure are, I believe, the same in both full and lite versions.
OWC tout their superior and faster (to Apple) RAID driver. That may well be so, I haven't checked it, 'cos I'm still running Apple RAID, with Soft RAID doing the monitoring. There's also a cost difference $34 v $175.
You can convert Apple RAID volumes to SoftRAID but, obviously, not SoftRAID to Apple if you change your mind in the future. Also, you can convert Apple mirrored sets into single volumes , not at all sure if you can do that with SoftRAID. Moot points, but not, if like me, you're thinking of replacing my RAID-0 setup with an SSD (or two) and then backed up to RAID-1 mirror.
SSD's and TRIM are all supported under SoftRAID Lite, so unless you're operating in an 'enterprise' environment, RAID-0 and 1's are all a photographer is likely to need, particularly with storage capacities and speeds expanding all the time.
One small, but thoroughly irritating point – I bought my copy just over a month ago, only to then discover that OWC are unable (read: unwilling) to issue licence codes via email. They insist on sending a CD package out by post. Not too much of a problem if you're in the USA, but for delivery to the EU it took just under a month. And then all I had to do was type in the code for my trial copy to be validated, no need to reinstall. OWC, if you're reading this: it's almost 2016 …
All best wishes for 2016 – to all on LuLa.
-
Yea, in my dealings w/ OWC, I've found them similarly 'rigid' about a number of support and service issues. Makes me think 2-3x before I press 'Add to Cart'.