I've been hard at work typing this up, so read to get a bit of an idea how I believe photographers and storage line up. It's about 70% but I keep seeing incorrect or baseless articles come out and I needed to speak out.
https://medium.com/@PNWMF/storage-systems-and-methods-for-photographers-86e04f940013#.uzejsolsi
Joe, I think this is great.
A few thoughts in general first.
You might want to back up a bit and talk about the difference between backup and archiving, like I did in this (unfortunately, by now somewhat dated) article:
http://blog.kasson.com/how-to/backing-up-photographic-images/You should also talk about off-site vs on-site storage, and storage hierarchies (maybe using another name to make it easy on the less technical).
Now, some specific comments:
You say: “Direct Attach can run at a much faster speed than your wifi or wired network can (in most cases).”
You might want to amplify that. Specifically, I’ve found that Synology Rack NAS boxes attached with a single 1Gb/s Ethernet are generally as fast, and often faster, that USB3 drives. External USB3 drive enclosures sometimes provide much slower transfer speeds than my NAS connections, which run at wire speed. Even though USB3 wire speed is five times higher, I’ve never seen much improvement over NAS speeds. In addition, it seems that many USB3 implementations share bandwidth, so that other USB3 transfers can slow down (of course, that’s true if you only have a single Ethernet connection). 10Gb/s Ethernet is finally dropping in price, but that’s still beyond the scope of your intended audience, I think, as is Ethernet bonding. It is a surprise to me that the more complicated protocol stack for Ethernet and either IP or Windows Domain networking runs faster than the leaner USB3 DASD protocol, but that seems to be the case. Maybe more time to optimize the code? Maybe hardware restrictions in USB3 implementations?
You say: “Returning to the backup topic, most cloud backup platforms works fine with DAS disks, but not for NAS setups.”
I think you’re talking about the cloud backup services that use their own client software. I don’t use those. I’ve tried many, and found they are generally opaque, inflexible, and buggy. For cloud backup (and for on-site backup), I prefer a third part backup app like GoodSync, although that limits you to cloud backup services that don’t require their own apps, like Amazon S3 or Dropbox.
When you talk about Wi-Fi, you talk about it coming from a router that supports it. That’s not how I do it, and there are probably others that think like me. I have a router connected to my ISP. The LAN side of that router is connected to the WAN side of a SonicWall firewall, the LAN side of the firewall is connected, via Ethernet switches, to wireless access points distributed about the house. This allows for more reliable Wi-Fi and greater coverage than I could possible get if I bought a router with built-in Wi-Fi.
You talk about enterprise SAS drives (which I agree are overkill) and (by implication) consumer SATA drives, but you don’t mention my preferred choice for spinning in arrays, which is enterprise SATA drives. I’ve gotten no failures since I started to switch over to them about two years ago (I have about 35 drives running, so the sample size is small), and I’m always nervous during rebuilds, even with RAID 6 NAS boxes. On the remaining commercial drive arrays, I’ve configured hot spares, so if I don’t get to the server room for a week or two to notice a failure, the array has already rebuilt itself. I really should configure email alerts.
I like what you’ve said about striping. I used to be a big fan of three striped 8TB Helium disks for first-line photo storage. I’m beginning to think about SSD for that, but at present, that’s a lot of disks. Your remarks about software RAID also intrigue me. I’ll have to look at that. I’ve been using PCIe RAID controllers for the striping. I don’t use RAID5 and 6 on workstations, but aren’t there write speed problems with software versions of those, since the software has no NV RAM at its disposal? Or do the hardware RAIDs use NV RAM anymore?
You say: “if you have an Adaptec RAID card and it throws a fit and dies, you have to get a similar Adaptec RAID card to read the contents of the disk.” I don’t understand this. Why not just put in whatever the heck you want and restore from a backup? If the RAID controller dies, your array is out of service anyway, and will be for several days unless you have a spare controller on-site.
Anyway, that’s my two cents. I’m looking forward to reading the complete paper.
Jim