How do you backup your files?

[fike]

I actually maintain three versions of all my files too.  I use a RAID 1 array for redundancy.  When I make my monthly backups, I am actually getting the third copy for moving offsite.

I'll have to take a look at syncback.

[dwdallam]

I actually maintain three versions of all my files too.  I use a RAID 1 array for redundancy.  When I make my monthly backups, I am actually getting the third copy for moving offsite.

I'll have to take a look at syncback.

Please don't think that RAID 1 gives you a backup. Prima-facia  it would seem like "yeah I have backup in RAID 1," and you do in real terms. But in practicality a backup really means something that is away and secure from your original copy, such as your removable disk, or a server running RAID where the disk is populated from a different power source on a different server structure, or "real" redundancy.

So if you have a simple single disk mechanical failure, your RAID 1 will recover that since identical information is written to both disks. However, if you lose a disk due to electrical malfunction, such as a power supply or outside power fluctuation, it will most likely take out both disks. This happened to a friend of mine where his entire computer roasted in less than a second due to a power surge the power surge protector didn't protect. It is safer than a single drive for sure, but for different reasons than being safe like a removable disk.

For that reason, I bought a hot swap SATA drive docking station. You plug bare drives into it with out any housing enclosure. When I'm done backing up, I pull the drive out and put it upside down in the carriage, safe from electrical attacks. The technology for the docking station is just an extension of server farm technology to a single docking station. Check it out. It takes 2.5" and 3.5" drives. Cheap too.

http://www.overclockersclub.com/reviews/th...take_blacx_hdd/

Check out page two where you will see a lot more images of this unit. EVERYONE is using these now. If you Google: Thermaltake BlacX you'll see Fred Miranda pop up, all sorts of other photography sites, computer enthusiast sites, lots more. I think they cost 35.00 US. I've been waiting for this technology for years. BTW the link to that page is an older review when the BLacX didn't have eSATA, but only USB. The new ones all have USB and eSATA capability.

Really though check out SyuncBack. It really is the shinola.

[OldRoy]

Not sure that I can contribute anything terribly useful here but I've been using Robocopy for donkeys' years and have not had any problems that weren't attributable to setup stupidity on my part. Whether it deals with the xmp issue is another question however - I'll have to take a look. But it has switches that cover most of the configurations that I've ever needed - including in a mission-critical (ugh) production environment. I'll have a look at syncback though!

On eSata, a word of warning that may only apply to a dotard like myself. If your internal drives are configured as "dynamic disk", the addition of any new drive, including one mounted via the external eSata interface (which is just Sata on a long wire) your entire ntldr configuration wil be instantly ****erred resulting in a non-booting system... Of course the dynamic disc mode was an error on my part whilst building this pc. I'd been playing about with various RAID modes and got sick of re-installing the os, and left the hardware in this configuration. Bad mistake. I managed to find a fix but it's horrible.

I'm still using eSata for my secondary backup external drive (in a caddy as described above) but I leave the drive switched off and only switch it on after booting and when I run the secondary backup script.

Roy

[dwdallam]

Not sure that I can contribute anything terribly useful here but I've been using Robocopy for donkeys' years and have not had any problems that weren't attributable to setup stupidity on my part. Whether it deals with the xmp issue is another question however - I'll have to take a look. But it has switches that cover most of the configurations that I've ever needed - including in a mission-critical (ugh) production environment. I'll have a look at syncback though!

On eSata, a word of warning that may only apply to a dotard like myself. If your internal drives are configured as "dynamic disk", the addition of any new drive, including one mounted via the external eSata interface (which is just Sata on a long wire) your entire ntldr configuration wil be instantly ****erred resulting in a non-booting system... Of course the dynamic disc mode was an error on my part whilst building this pc. I'd been playing about with various RAID modes and got sick of re-installing the os, and left the hardware in this configuration. Bad mistake. I managed to find a fix but it's horrible.

I'm still using eSata for my secondary backup external drive (in a caddy as described above) but I leave the drive switched off and only switch it on after booting and when I run the secondary backup script.

Roy

You should pull the drive out of the caddy so it interrupts the flow of electricity in surge conditions. Just because the switch is off doesn't mean there isn't a direct wire connection to your drive. Electricity can arc aross switches, especially those very small distance between contacts on a little push button on of switch in our docking stations.

[Nick Rains]

For those that do use Synch Toy, as I do, you need to look at the 'Contribute' option, not 'Synch' or others - this is where people are going wrong. 'Contribute' copies files one way only, no deletions at all. It simply copies files that are new or changed. If you delete a file locally, it does not delete it on the backup drive- it's one way only.

"Contribute will add changes from the left folder to the right folder. New and updated files are copied left to right. Renames on the left are repeated on the right. No deletions."

Check out the help files in Synch Toy and look for 'Contribute'. Read the other 4 methods and you'll see where people are going unstuck.

SynchToy is limited but does what it says it does. It's no use for backing up systems but for copying directories of images to a second location on a regular basis it's very effective. it runs at midnight every night and I have yet to have any issues - touch wood!

[dwdallam]

For those that do use Synch Toy, as I do, you need to look at the 'Contribute' option, not 'Synch' or others - this is where people are going wrong. 'Contribute' copies files one way only, no deletions at all. It simply copies files that are new or changed. If you delete a file locally, it does not delete it on the backup drive- it's one way only.

"Contribute will add changes from the left folder to the right folder. New and updated files are copied left to right. Renames on the left are repeated on the right. No deletions."

Check out the help files in Synch Toy and look for 'Contribute'. Read the other 4 methods and you'll see where people are going unstuck.

SynchToy is limited but does what it says it does. It's no use for backing up systems but for copying directories of images to a second location on a regular basis it's very effective. it runs at midnight every night and I have yet to have any issues - touch wood!

Which means that if you already have source and backup files and want to run SyncToy to recreate the profile, will need to delete the entire back up first, then run SynchToy. I've done some extensive research into what SynchToy will do and what it will not, and it will not do the things I and others have described here, such as seeing updated xmp files, which it cannot see.

On the other hand, SyncBack does it all. If you have files on the backup side and files on the Source side and you want to create a new profile and then run the job, deleting any files on the backup side that are not on the Source side any longer, while copying/overwriting new and updated files--SynchBack does that perfectly. ST can't, period. Go to MS's forum on SyncToy or look for articles on the MS website--and others--and you will see these two concerns are very real.  SyncToy also leaves its stupid databases lying all over your drives in every directory, Source and Destination that you create. SyncBack keeps them all in one place, like it should.

Compared to SyncBack--a free program--SyncToy is not a toy, it's a joke.

[fike]

Please don't think that RAID 1 gives you a backup. Prima-facia  it would seem like "yeah I have backup in RAID 1," and you do in real terms. But in practicality a backup really means something that is away and secure from your original copy, such as your removable disk, or a server running RAID where the disk is populated from a different power source on a different server structure, or "real" redundancy.

So if you have a simple single disk mechanical failure, your RAID 1 will recover that since identical information is written to both disks. However, if you lose a disk due to electrical malfunction, such as a power supply or outside power fluctuation, it will most likely take out both disks. This happened to a friend of mine where his entire computer roasted in less than a second due to a power surge the power surge protector didn't protect. It is safer than a single drive for sure, but for different reasons than being safe like a removable disk.

For that reason, I bought a hot swap SATA drive docking station. You plug bare drives into it with out any housing enclosure. When I'm done backing up, I pull the drive out and put it upside down in the carriage, safe from electrical attacks. The technology for the docking station is just an extension of server farm technology to a single docking station. Check it out. It takes 2.5" and 3.5" drives. Cheap too.

http://www.overclockersclub.com/reviews/th...take_blacx_hdd/

Check out page two where you will see a lot more images of this unit. EVERYONE is using these now. If you Google: Thermaltake BlacX you'll see Fred Miranda pop up, all sorts of other photography sites, computer enthusiast sites, lots more. I think they cost 35.00 US. I've been waiting for this technology for years. BTW the link to that page is an older review when the BLacX didn't have eSATA, but only USB. The new ones all have USB and eSATA capability.

Really though check out SyuncBack. It really is the shinola.

You are right about the RAID limitations.  To mitigate that possibility, in addition to the RAID, I have invested in pretty decent uninterruptable power supply for my PC unit.  

I took a look at the synchback webpage and it seems to have all the same features as the genie-soft product I used.  I particularly liked the Genie Soft product because its compressed files were in standard zip format.  I liked that because if the software manufacturer went out of business, I would always have the zip files.  The problem I had was that I kept getting Delayed Write Failures on my very large files (larger than 1GB).  I also tried Acronis Drive Image and one other program; they all had the same problems.  Any tool that compresses the file takes so long to finish the compression of these very large files that it seems that the hard drive (or OS) times-out and makes the backup fail. I expect SyncBack would have the same limitation.

So, now I backup manually.  

That hard drive dock is cool.  I have almost bought it a few times.  I have a similar gadget that is even more flexible.  On one USB cable it has data connectors and power for PATA 2.5 and 3.5, and SATA.  With this cable I can plug-in any bare drive I want.  This is nice because I still have some decent sized PATA drive laying around to use for backup.

[fike]

This Hard Drive docking station enables eSATA and USB.  eSATA is much faster than USB2.  I might need one of thee.

eSATA HD Dock

[dwdallam]

This Hard Drive docking station enables eSATA and USB.  eSATA is much faster than USB2.  I might need one of thee.

eSATA HD Dock

Fike,

That's the one I was talking about. It's the BlacX. It also uses less CPU and is slightly faster than it's closest competitors.

As far as backing up and compressing, there are several ways to get around your problems, which is a limitation of the compression algorithms (if I remember correctly).

(1) Tell windows to compress your backup drive. This is probably the best choice.

(2) Use TrueCryp and create a volume on your backup drive, either partial or full. The entire drive will be encrypted and compressed. Once you mount the drive, copying, backing up, etc., will work like a regular drive, all transparent. Plus all of your information is encrypted. TrueCryp is Open Source and won't be going broke  You can also use TruCrypt as a boot level security tool, meaning your entire hard drive is compressed and encrypted at the boot level. TrueCrypt writes to your disk MBR anbd sets up a boot level password login before the OS is booted. This is overkill.

(3) Use most any ISO creation software and backup your files to the ISO, which you mount as any other drive. The ISO would reside on your backup drive. You can create ISO's that are compressed. The only problem with that approach is that all of your files are in ONE container. if the container gets corrupted, you lose it all. I don't know how likely that is though, and I've never seen an ISO "go bad." ISOs images are really cool. I could probably come up with several even better options, other than (1), which is your easiest and most workable bet.

Once you get your backup work flow down, you will feel a lot of unknown stress disappear from your life. At least I do now that I started using SyncBack. I can't tell you all how much that reduced my stress level.

Again, thank much for passing SyncBack on to me.

[Nick Rains]

On the other hand, SyncBack does it all. If you have files on the backup side and files on the Source side and you want to create a new profile and then run the job, deleting any files on the backup side that are not on the Source side any longer, while copying/overwriting new and updated files--SynchBack does that perfectly. ST can't, period.

OK it won't do that although I'm not exactly sure why you want to - but no matter, it's your workflow not mine and it obviously works for you.

OTOH, ignoring the issue of little files let lying around , it does exactly what I want it to, which is to make sure all the files on my internal storage drive are copied out to a second location if they are changed or added to. I don't want to ever automate the deletion of any files - too risky. I'll do that as a separate task, if I ever need to.

I'll check out Synchbabk in the meantime...

[dwdallam]

OK it won't do that although I'm not exactly sure why you want to - but no matter, it's your workflow not mine and it obviously works for you.

OTOH, ignoring the issue of little files let lying around , it does exactly what I want it to, which is to make sure all the files on my internal storage drive are copied out to a second location if they are changed or added to. I don't want to ever automate the deletion of any files - too risky. I'll do that as a separate task, if I ever need to.

I'll check out Synchbabk in the meantime...

Nick,

It won't copy your xmp files or any files like that because it cannot see that the files have changed. I have thousands of hours locked up in xmp files. I sure don't want to lose that time.

[Ronny Nilsen]

OK it won't do that although I'm not exactly sure why you want to - but no matter, it's your workflow not mine and it obviously works for you.

OTOH, ignoring the issue of little files let lying around , it does exactly what I want it to, which is to make sure all the files on my internal storage drive are copied out to a second location if they are changed or added to. I don't want to ever automate the deletion of any files - too risky. I'll do that as a separate task, if I ever need to.

I'll check out Synchbabk in the meantime...

I use both methods with SyncBack.  

I use external cabinets with eSata drives, and have one drive that is synced (new ad and modified files are copied, and deleted files are deleted) and one drive where new and and modified files are copied (and old versions preserved) but no delete. If the main drive fails I just swap it with the synced drive and is back in operation in 5 min. And if I make an error and delete or modify files I can get them back from the backup drive.

Ronny

[Nick Rains]

Nick,

It won't copy your xmp files or any files like that because it cannot see that the files have changed. I have thousands of hours locked up in xmp files. I sure don't want to lose that time.

OK, I see.

I use DNG files almost exclusively so I don't have the same problem. One advantage of DNGs I guess. If I used sidecar files or other xmp stuff I'd be in trouble.

[jjj]

It won't copy your xmp files or any files like that because it cannot see that the files have changed. I have thousands of hours locked up in xmp files. I sure don't want to lose that time.

To repeat myself, try Clone 2.1. That can tell if a file has been modified in several ways. I've used it when working away with a PC laptop and need to keep my 2 ext HDs in sync.
It's a very neat and powerful, if basic looking bit of software.

[PeterAit]

Two questions related to this thread:

1) What exactly is stored in an XMP file?

2) Why doesn't SyncToy pick up changes to these files?

THanks,

Peter

[Pelao]

I run all my images off an external drive, via FW800. When the images are downloaded to the main external drive ( using Lightroom) they are copied right away to a second drive using Chronosync. Only then do I erase images from the memory card.

Once I have sorted the images, and deleted the duds, the main drive is copied to the second drive again, thus eliminating the duds there too.

At this stage I burn copies of the originals to Archival Gold DVDs.

So at any time my Lightroom library is on 2 drives, and originals are on archival quality DVDs. ( Archival Gold DVDs)

I will soon add another drive to which I will back up the library once a week and keep off-site.

The final stage is an online backup of my favourite images.

Simple, effective and working like a dream for years (before Lightroom it was a main image folder).

[nubins]

For my Mac backups I use NTI Shadow or Retrospect for software.

My main focus when designing my backup system was not how to do it but what to use. There are so many methods and solutions out there and many work very well. I wanted two backups for redundancy and online storage in case something happens to both hard drives like an electrical surge or fire.

So here is what I came up with:

For hard drives I wanted redundancy so I looked for a RAID 1 mirrored hard drive enclosure so every time I backed up the data is stored on two drives. I ended up looking for an SAN enclosure that can always be accessible on my network regardless of what computer I am using. There are also cheaper enclosures out there that connect via USB/Firewire/eSATA that are only local hard drives but have RAID 1 mirrors built in.

SAN network hard drive setup (this is what I use):
http://www.dlink.com/products/?pid=509
Seagate Barracuda 7200 RPM drives

local hard drive setup:
http://www.tigerdirect.com/applications/se...ygtCjCVRqCjCVRq

For online storage I use iDrive. For $49.95 a year you can have one personal storage account for 150 GB. If you need more storage you can add up to a total of 5 150 GB accounts at $49.95 each per year. Not bad for 750 GB of storage compared with any other online storage service. I think you can store all or at least most of your keeper RAW files with 750 GB. You can purchase more storage but the cost goes up significantly since they consider it a business account after 750 GB.

http://www.idrive.com/

I've been using this system for a year with no faults or crashes yet. Worse case scenerio if my place burned down at least I have my network backup as well.

I hope this helps,

Jeremy

[Pelao]

For hard drives I wanted redundancy so I looked for a RAID 1 mirrored hard drive enclosure so every time I backed up the data is stored on two drives. I ended up looking for an SAN enclosure that can always be accessible on my network regardless of what computer I am using. There are also cheaper enclosures out there that connect via USB/Firewire/eSATA that are only local hard drives but have RAID 1 mirrors built in.

One thing to note:

Yes, a raid means that your data is on two drives. But note that should one of the drives become corrupted for any reason, the corruption is copied to the second drive automatically. It's healthier to manage the copying so that it only happens when you have confirmed the data on the first drive is healthy.

[dalethorn]

One thing to note:
Yes, a raid means that your data is on two drives. But note that should one of the drives become corrupted for any reason, the corruption is copied to the second drive automatically. It's healthier to manage the copying so that it only happens when you have confirmed the data on the first drive is healthy.

One (just one) of the major problems people have with backing up to DVD's is finding something when you need it.  For that, you need to catalog the DVD's, i.e. each DVD and what's on it.  We assume nothing can go wrong with an archive DVD, or do we make two?  Back to the catalog.  How do you describe what's on the DVD's?  A general description like "Bob's wedding - 12/15/2008?"  Or how about "Landscape shoot - redwood forest - 1/13/2009?"  Sometimes you need to find something by a specific thing in an image that isn't in the catalog.  I've found it necessary to file a collection/archive set in its own folder, and within each folder each image has a filename that describes the main point or object of the image, or possibly that plus a secondary subject/object in the image.  Then, archiving all of those on a hard drive instead of diskettes, I can find precisely what I need quickly with a foldername/filename search, without having to maintain a hand-written catalog.

I make database software for process manufacturers, and what do you suppose their most requested (and least realized) issue is?  To be able to easily get to all of the data they entered into their database, quickly and painlessly.  Not happening yet.

[GiorgioNiro]

I have grown so annoyed and tired with the vagaries of these tools, that I have gone back to manually copying my data to a second, external hard drive.

1) Backup an entire year(s) on a hard drive.  Store offsite. (two hard drives in rotation)
2) Every month backup all of current year on a hard drive. Store offsite. (two hard drives in rotation)
3) At beginning of new year, make backup of all previous years on a new hard drive and store offsite.

I am risking the last month of data, but it makes more sense that way.  All the backup programs had problems with my huge panoramic source files (1GB is my largest, but 800MB is very common).  I also had problems with write protect failures in these huge file transfers to my WD Mybook 500GB.  Now I just move it manually, it is slightly annoying every year, and a very minor task monthly.

Me Too!
And this Hard Drive accessory makes it easy!
http://eshop.macsales.com/item/Newer%20Tec...ogy/FWU2ES2HDK/