Luminous Landscape Forum
Equipment & Techniques => Computers & Peripherals => Topic started by: tonysiciliano1 on July 03, 2018, 02:10:40 pm
-
I have 6 TB of photos that (in addition to my backups onto external hard drives) I would like to back up online using something like Backblaze. The problem is my upload max speed is 10-15 mbps so I might die before I get the photos all uploaded. Anybody know of any workarounds for this problem? I'm wondering if you can send your files on a hard drive to any sites like Backblaze. Suggestions?
-
I have 6 TB of photos that (in addition to my backups onto external hard drives) I would like to back up online using something like Backblaze. The problem is my upload max speed is 10-15 mbps so I might die before I get the photos all uploaded. Anybody know of any workarounds for this problem? I'm wondering if you can send your files on a hard drive to any sites like Backblaze.
You can, at least in the case of BackBlaze:
https://www.backblaze.com/blog/introducing-backblazes-rapid-ingest-service-fireball/
https://www.backblaze.com/blog/cost-data-of-transfer-cloud-storage/
Jim
-
Thanks for the info. I checked BackBlaze rapid ingest service fireball service. It costs $550 to rent the device to put your data on and mail back to them. That would make sense if you had a lot of data, but for 6 TB it's too expensive. Oh well, a nice thought....
-
Thanks for the info. I checked BackBlaze rapid ingest service fireball service. It costs $550 to rent the device to put your data on and mail back to them. That would make sense if you had a lot of data, but for 6 TB it's too expensive. Oh well, a nice thought....
How much would it cost you if you lost your data?
-
Thanks for the info. I checked BackBlaze rapid ingest service fireball service. It costs $550 to rent the device to put your data on and mail back to them. That would make sense if you had a lot of data, but for 6 TB it's too expensive. Oh well, a nice thought....
Many of us would think 6TB was quite a lot of data! Anyway, the purpose of backup is ability to restore, not just to provide peace of mind from the though that "it's somewhere else as well". How long would it take you to download 6TB form an online backup if you lost it?
Jeremy
-
I have 6 TB of photos that (in addition to my backups onto external hard drives) I would like to back up online using something like Backblaze. The problem is my upload max speed is 10-15 mbps so I might die before I get the photos all uploaded. Anybody know of any workarounds for this problem? I'm wondering if you can send your files on a hard drive to any sites like Backblaze. Suggestions?
My upload speed is 18mbps and it took me 5 weeks to backup 6TB of photos to Backblaze.
Re. restoring data. Backblaze will send you the data on a hard drive.
-
Honestly, 6TB is nothing - as I do a 30TB replication between 2 NAS's ;) The trick is in Backblaze exclude some folders - as in anything shoot more than 6 months old. Once it has the most recent stuff uploaded, then start removing the exclusions working from newest to oldest. That way the most important, valuable shots get uploaded first.
Or head down to the library - or any other place that has faster upload speeds.
https://www.nbcbayarea.com/news/local/San-Francisco-Public-Library-Nations-Fastest-Internet-Access-311586421.html
-
My upload speed is 18mbps and it took me 5 weeks to backup 6TB of photos to Backblaze.
I have somewhat more than that, and it took a couple of weeks to do the upload over a 50 Mb/s ATT IPFlex connection (theoretical time for 6TB is 11.1 days). Backblaze supported that transfer speed: I was getting 6+ MB/s speed reports from GoodSync, and everything else, including HD Netflix streaming, worked just fine during the upload. However, every 24 or 36 hours the Backblaze server would stop responding and I'd have to re-click "Analyze and Synch" in GoodSync.
Jim
-
Why do you need online? I assume you are worried that a fire or tornado or burglary will happen? Why not back up onto hard disks yourself and make sure the disks are secure? I use our safe, which will survive pretty much anything. Or, a safe deposit box at a bank.
-
Why do you need online? I assume you are worried that a fire or tornado or burglary will happen? Why not back up onto hard disks yourself and make sure the disks are secure? I use our safe, which will survive pretty much anything. Or, a safe deposit box at a bank.
I do that, too. Can't have too many copies...
Jim
-
Does your bank say that your safe deposit box is waterproof or fireproof? Is it insured?
-
Just keep your computer on and let it run. I uploaded about 2.5tb to black blaze that way.
Sent from my iPhone using Tapatalk
-
I do that, too. Can't have too many copies...
That is the key. I have a PO Box and just keep one in there also. Safes are good but "fireproof" safes are only fireproof for documents which burn at a much higher temp than hard drives melt.
To me in Australia on line systems are useless. It would take 6 months to upload my data
-
Just because it will take a “long” is no reason to abandon the idea. Make a complete backup (or two) on external drives, and store them in two different locations. Then start backing up new data you create and files you change today and going forward at Backblaze. This will rapidly reach stasis, and you will have all of your data offsite. Then start adding the older data to Backblaze. Eventually, it will finish.
-
If you backed up two external drives and kept one in your house and one in your neighbor's or relatives, what would be the odds both houses would burn down at the same time. Is the cloud any more secure?
-
If you backed up two external drives and kept one in your house and one in your neighbor's or relatives, what would be the odds both houses would burn down at the same time. Is the cloud any more secure?
Prior to 9/11, there were companies in the WTC that had their backup in the other tower ...
-
I created my own Cloud backup using two QNAP NAS's. I had both onsite for the initial data load and then packed up the second and installed it at my son's home. Initial cost is higher but I have no monthly costs and I control everything with the hardware and network connections. Using RAID10 I would need two hard drives in the NAS to fail to lose any data. Oh, and my son is 10 hour drive away so I'm pretty sure that no fire or flood will hit both homes at the same time.
I know it's not what you were asking but I contemplated the same before I went this route.
-
For offsite, the appeal of Backblaze is that you don't have to think about it or arrange anything. Set it off and away it goes. 50 bucks a year is hard to beat cost wise. You haven't had online backup for the past few months, at least you will know over the next few months it will be taken care of.
-
If you backed up two external drives and kept one in your house and one in your neighbor's or relatives, what would be the odds both houses would burn down at the same time. Is the cloud any more secure?
Sadly, any backup process which requires manual intervention everyday is likely to fail because we are lazy. The beauty of a cloud-like solution - whether it’s a commercial cloud or one you set up yourself - is that it runs every day (or hour) with no intervention.
-
For offsite, the appeal of Backblaze is that you don't have to think about it or arrange anything.
For any backup system, I recommend periodic trial restores. Set and forget is not a good strategy, IMO.
Jim
-
I never advocated forget...
And agreed always do trial restores. There's always World Backup Day (http://www.worldbackupday.com/en/) which we use to test each of ours.
-
I never advocated forget...
And agreed always do trial restores. There's always World Backup Day (http://www.worldbackupday.com/en/) which we use to test each of ours.
Maybe I got the wrong idea when you wrote: "...don't have to think about."
Jim
-
You did.
My comparison was with managing offsite drives in other premises.
-
Sadly, any backup process which requires manual intervention everyday is likely to fail because we are lazy. The beauty of a cloud-like solution - whether it’s a commercial cloud or one you set up yourself - is that it runs every day (or hour) with no intervention.
Yes... if the process relies in a human remembering to do something... it is going to fail.
http://dgpfotografia.com
-
I never advocated forget...
And agreed always do trial restores. There's always World Backup Day (http://www.worldbackupday.com/en/) which we use to test each of ours.
Sadly, I’ve restored from Crashplan more than once. Happily, it succeeded each time.
-
The one truth... drive failure is inevitable, whether through internal or human failure. There's simply the question of how will you restore.
I've been there more than once. It was interesting to me restoring my MacBook Pro last year how much of my data lies in online services now. Dropbox and iCloud hold all of my documents, my photographs and video and other media all on Backblaze, in addition to local drives. I had not done a clean install of MacOS for well over a decade, simply upgrading as I went so I took the opportunity to not do a simple Time Machine restore and instead do a clean install and let the files download from iCloud and Dropbox. The main pain point was dealing with automation which I had to reconstruct.
-
Sadly, I’ve restored from Crashplan more than once. Happily, it succeeded each time.
My experiences with Crashplan were less satisfactory. Upload speeds were fastish, at about 15 Mb/s, at first, then incredibly slow as the backup continued. After 5 months, about halfway through a 6 TB upload, the app hung and couldn't be restarted. Uploading more data than that in a couple of weeks with Backblaze was a relief.
Jim
-
I also gave up on trying to upload 6TB to Crashplan - it was just too slow. I switched to Backblaze and my upload speeds were ten times faster.
-
I set up my own offsite "cloud" backup.
I have close to 8TB I want an offsite backup of and what I did was buy a cheap external 8TB drive and used SyncToy (free download from Microsoft (https://www.microsoft.com/en-us/download/details.aspx?id=15155)) to make a copy of all that data right from my PC to the external drive.
Then I took that drive to work and connected to my computer there and set up another free app called Resilio Sync (https://www.resilio.com/platforms/desktop/). Sync uses Bit Torrent technology to sync folders across the Internet. The free version will do the trick but I upgraded for a few nice features like bandwidth throttling.
Since I had copied all my data already, it just had to index everything (took a couple of days) and now it just copies any new files or changes I make at home to the offsite drive at my office. Works great and no monthly fees!
You could also set this up at a friend's or relative's house if you have someone who could host it for you and you don't have an office outside of your home.
-
Matt, thanks for the link - resilio looks interesting. I have a similar setup, backing up to my mother's iMac nightly using ChronoSync as well as using Dropbox and Time Machine. Ever since I lost three weeks' work when I was an undergraduate, I've been obsessional about backups. My setup provides that within 24 hours (because I have set ChronoSync to run at 0030), anything I have done is stored on three computers' internal drives, three TM backups and in Dropbox. That satisfies even my neuroticism.
Jeremy
-
Matt's setup works, as long as folks are aware that any file overwrites & corruption on the source computer are transmitted automatically to the offsite one. Something that allows versioning on the remote side - like anything overwritten or changed in the previous 14 days is recoverable would be huge. Also, alerting as to a sync that didn't happen.
-
Matt's setup works, as long as folks are aware that any file overwrites & corruption on the source computer are transmitted automatically to the offsite one. Something that allows versioning on the remote side - like anything overwritten or changed in the previous 14 days is recoverable would be huge. Also, alerting as to a sync that didn't happen.
Like Jeremy I'm also a bit of a backup fanatic. I haven't lost any significant data but working in IT have seen it happen to others many times. I hate telling people their data will need to be restored from backup when I know they probably don't have one.
What I outlined was my offsite backup plan which is one component of my whole approach. I also have a home server running the open source (free) application URBackup. That backs up all the computers in my household on an ongoing basis, keeping multiple versions of changed files as well as full images to quickly recover from a drive failure. The offsite is a second line of defense in case of fire, flood or other catastrophic situation.
8)
-
Like Jeremy I'm also a bit of a backup fanatic. I haven't lost any significant data but working in IT have seen it happen to others many times. I hate telling people their data will need to be restored from backup when I know they probably don't have one.
What I outlined was my offsite backup plan which is one component of my whole approach. I also have a home server running the open source (free) application URBackup. That backs up all the computers in my household on an ongoing basis, keeping multiple versions of changed files as well as full images to quickly recover from a drive failure. The offsite is a second line of defense in case of fire, flood or other catastrophic situation.
8)
Defense in depth - nicely done!
-
Matt's setup works, as long as folks are aware that any file overwrites & corruption on the source computer are transmitted automatically to the offsite one. Something that allows versioning on the remote side - like anything overwritten or changed in the previous 14 days is recoverable would be huge. Also, alerting as to a sync that didn't happen.
And setting the maximum change that will propagate over...
Jim
-
Matt's setup works, as long as folks are aware that any file overwrites & corruption on the source computer are transmitted automatically to the offsite one. Something that allows versioning on the remote side - like anything overwritten or changed in the previous 14 days is recoverable would be huge. Also, alerting as to a sync that didn't happen.
You're right about the utility of versioning, and that's something that ChronoSync doesn't offer (it will move deleted files to a temporary folder for a while, but that's obviously not quite the same thing). Time Machine does versioning, of course, and since there's a TM drive attached to my mother's computer (the remote backup machine) I get versioning that way. It's tricky to use unless I go round to her flat myself, of course, but it's only a mile away.
CS sends me an email after each run, listing files which have been copied or deleted; and it will email me about any sync which hasn't worked. Since I get those emails daily, I notice if they don't appear.
Like Jeremy I'm also a bit of a backup fanatic. I haven't lost any significant data but working in IT have seen it happen to others many times. I hate telling people their data will need to be restored from backup when I know they probably don't have one.
What I outlined was my offsite backup plan which is one component of my whole approach. I also have a home server running the open source (free) application URBackup. That backs up all the computers in my household on an ongoing basis, keeping multiple versions of changed files as well as full images to quickly recover from a drive failure. The offsite is a second line of defense in case of fire, flood or other catastrophic situation.
Thanks for that link, Matt. I'll investigate.
Jeremy
-
If you use Dropbox Plus you can enable versioning for up to 1 year.
For Crashplan users you can set quite detailed versioning, including backup frequency from every 15 minutes to once a day, then versioning allows you to set for one week, after one week, after 90 days, after a year, and deleted files with various timings (or in the case of deletions including "never").