Luminous Landscape Forum
Equipment & Techniques => Digital Asset Management => Topic started by: calindustries on April 14, 2016, 10:40:14 am
-
Anyone have an off-site auto-backup alternative to Crashplan? I've been running it for a few years and there are no issues other than it's insanely slow to upload files. I'm NEVER caught up, like never. Always months behind. I have a fast upload speed from my ISP but it seems to be really bottlenecked on the Crashplan end of things. I'm not a network engineer so something simple to use and forget about unless needed is what I'm looking for. Not even a big amount of storage needed, maybe about 5TB or so.
Thanks,
Craig
-
I'm not sure another service is going to be any faster, sounds like your upload speeds. I use CrashPlan and have no such issues but am likely not uploading as much data. Got 2.4 gigs up there....
-
Anyone have an off-site auto-backup alternative to Crashplan? I've been running it for a few years and there are no issues other than it's insanely slow to upload files. I'm NEVER caught up, like never. Always months behind. I have a fast upload speed from my ISP but it seems to be really bottlenecked on the Crashplan end of things. I'm not a network engineer so something simple to use and forget about unless needed is what I'm looking for. Not even a big amount of storage needed, maybe about 5TB or so.
Thanks,
Craig
How fast is your upload speed? Have you discussed with CP to determine if bottleneck truly is on their side and any possible resolution?
-
Upload speed is 50Mbps. They won't say they "bottleneck" but they say they can't match my speed.
-
Anyone have an off-site auto-backup alternative to Crashplan? I've been running it for a few years and there are no issues other than it's insanely slow to upload files. I'm NEVER caught up, like never. Always months behind. I have a fast upload speed from my ISP but it seems to be really bottlenecked on the Crashplan end of things. I'm not a network engineer so something simple to use and forget about unless needed is what I'm looking for. Not even a big amount of storage needed, maybe about 5TB or so.
Thanks,
Craig
my upload speed on a genuinely new files to be added is 300-500KB/sec up to Crashplan, I am on 75mbps FIOS ...
-
Got 2.4 gigs up there....
surely you mean Tb.
-
Why not set up your own personal Crashplan server at a friend or relatives house. That way you control any bottlenecking and you can manually preload the hard disks.
-
I'd have to learn how. Guess I can look into it
-
surely you mean Tb.
Sorry; good catch. I've got 651 gig's up on CrashPlan.
Did a test, upload speeds here are 12MB per second, nothing earth shattering.
-
I have no experience with crashplan, right now I'm using Backblaze, with a 30 Mbps upstream connection (300 Mbps downstream). Backblaze estimates that I can backup around 100GB of data per day...
Backblaze has a trial period, maybe you can try it to see if I works for you... With a few day you can easily see if it is going to work for you or not...
http://dgpfotografia.com
-
Thanks for the tip. I've been testing Backblaze for the last 24 hours and I've managed to upload 140Gb. This is much faster than my tests of Crashplan, which only managed 25Gb/day.
I have 5Tb to backup, so I'm looking at 1 month with Backblaze vs. 6 months with Crashplan. I'm less worried about the length of time for the initial backup, than whether the backup process will keep up with me when I'm making a lot of large files (e.g. preparing print files for an exhibition).
I'm based in Europe (London), and Backblaze allows multi-threading in its backup process. This apparently speeds things up for those users who are a long distance from the Backblaze data-centre in California - https://www.backblaze.com/blog/backblaze-online-backup-4/
So far, it seems to be working for me. However, time will tell - Backblaze seems to backup smaller files first (text files and jpegs) so it hasn't yet tackled any of my raw files and psds. Perhaps these will bog it down. (Crashplan, more sensibly, gives priority to your most recent files.)
(Connection. BT Infinity. 60 Mbps down, 18 Mbps up.)
-
Have you considered Amazon's own S3 service? You'll need an ftp client, of course, but the storage is dirt cheap if you don't mind waiting for download access. I've not checked it for upload speed, though.
Jeremy
-
I may have done the maths wrong, but it seems quite expensive to store 5Tb - $35 a month, for the cheapest Glacier service (vs. $5 a month for Crashplan or Backblaze) - https://aws.amazon.com/s3/pricing/
-
Backblaze is great.
Sent from my SM-N900V using Tapatalk
-
There are some settings in CrashPlan that throttle upload speeds. You can tell it to not use more than (say) 25% of your bandwidth, or 10% of your cpu. Make sure you aren't hurting yourself this way.
-
So far, it seems to be working for me. However, time will tell - Backblaze seems to backup smaller files first (text files and jpegs) so it hasn't yet tackled any of my raw files and psds. Perhaps these will bog it down. (Crashplan, more sensibly, gives priority to your most recent files.)
Yes, Backblaze uses the strategy of uploading first always the smaller files, that if your more important files are videos could be a bit unproductive...
-
I may have done the maths wrong, but it seems quite expensive to store 5Tb - $35 a month, for the cheapest Glacier service (vs. $5 a month for Crashplan or Backblaze) - https://aws.amazon.com/s3/pricing/
Your math is correct, Amazon s3 or glacier are more expensive if you want to storage several TBs of data. I'm using Glacier right now for the backups of my web server... It is cheaper this way, but we are talking here of a few hundred megabytes of data...
-
European users might consider Hubic ( a project from https://en.wikipedia.org/wiki/OVH ) = 10 TB for Eu50/year
-
European users might consider Hubic ( a project from https://en.wikipedia.org/wiki/OVH ) = 10 TB for Eu50/year
Thanks, interesting, I was not aware of this and I use OVH as my hosting provider for my webpages...
-
Thanks, interesting, I was not aware of this and I use OVH as my hosting provider for my webpages...
if you try - please share how is upload/download from Europe
-
I may have done the maths wrong, but it seems quite expensive to store 5Tb - $35 a month, for the cheapest Glacier service (vs. $5 a month for Crashplan or Backblaze) - https://aws.amazon.com/s3/pricing/
You're right. I'd not considered the volume of data you want to keep. Apologies.
Jeremy
-
Backblaze continues to impress. It has now backed up all my jpegs and has moved onto my raw files. No sign of a slow down - in fact, it has speeded up a bit. It is now doing 208Gb a day - that's over 8X faster than Crashplan (in my tests). Overall, I expect it to back up my 5Tb in 24 days (total).
-
In Backblaze's preferences you can click a link 'How long will my first backup take?' - this takes you to a webpage which attempts to estimate how long your backup will take. Each time I go there it says something like 'Freak Out... your data will be backed up in 300 days'. As far as I can tell, this is a false estimate that assumes you haven't made various changes in Backblaze's 'Performance' panel (i.e. switched off Automatic Throttle, and turned on Threading).
-
Upload speed is 50Mbps. They won't say they "bottleneck" but they say they can't match my speed.
This is an inordinately tardy reply, but on the off-chance it's useful to you or anyone else struggling with this problem: if Crashplan isn't using the available bandwidth, it is almost certainly due to the "Deduplication" setting on the client. If you have de-dupe enabled your backups become CPU limited (and will cheerfully eat all the available CPU in the process).
Switching off de-dupe should allow you to reach your line speed. There is no effect on your backups, other than them taking a bit more storage at Crashplan's end of things.