Pages: [1] 2 3   Go Down

Author Topic: Storage systems and methods for photographers  (Read 18859 times)

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1365
Storage systems and methods for photographers
« on: January 31, 2017, 03:22:45 pm »

I've been hard at work typing this up, so read to get a bit of an idea how I believe photographers and storage line up.  It's about 70% but I keep seeing incorrect or baseless articles come out and I needed to speak out.

https://medium.com/@PNWMF/storage-systems-and-methods-for-photographers-86e04f940013#.uzejsolsi
Logged
t: @PNWMF

rdonson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3263
Re: Storage systems and methods for photographers
« Reply #1 on: January 31, 2017, 04:30:43 pm »

A good start, Joe!  I'm sure many people will benefit from this when you've finished.
Logged
Regards,
Ron

NancyP

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2513
Re: Storage systems and methods for photographers
« Reply #2 on: January 31, 2017, 04:45:39 pm »

Thanks!
Logged

davidgp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 758
    • davidgp fotografia
Re: Storage systems and methods for photographers
« Reply #3 on: February 01, 2017, 02:33:49 am »

Hi Joe,

I just did a quick read to your article. When you are talking about the internal drive you said that all drives are SATA now, not exactly truth now a days. You can install PCIe mvne bus SSD drives now, maybe connected directly to a PCIe slot or to an m.2 connector (Apple uses instead a proprietary connector but it is just basically a PCIe drive).

If someone is buying a new PC (for macs if you select opt he SSD option it will be already one of this drives), it is better to buy one of these drives, such as a Samsung EVO 960... It will be around 3x faster than the SATA equivalent and just a bit more expensive.

Regards,

David


Enviado desde mi iPad utilizando Tapatalk

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1365
Re: Storage systems and methods for photographers
« Reply #4 on: February 01, 2017, 11:39:21 am »

Hi Joe,

I just did a quick read to your article. When you are talking about the internal drive you said that all drives are SATA now, not exactly truth now a days. You can install PCIe mvne bus SSD drives now, maybe connected directly to a PCIe slot or to an m.2 connector (Apple uses instead a proprietary connector but it is just basically a PCIe drive).

If someone is buying a new PC (for macs if you select opt he SSD option it will be already one of this drives), it is better to buy one of these drives, such as a Samsung EVO 960... It will be around 3x faster than the SATA equivalent and just a bit more expensive.

Regards,

David


Enviado desde mi iPad utilizando Tapatalk

Hey David,

Yep, NVMe drives are a whole different ball of wax, which I should state a bit clearer.  The price premium is totally worth it - I really wish they'd do some sort of slot much like the RAM on iMacs where you can easily upgrade to a larger size.  The speed and capacity of a 2280 drive is stunning.

Thanks for the feedback, I've got lots more to write, but I'm pulled many different directions.  I need to figure out how to do chapters or such on Medium.

-Joe
Logged
t: @PNWMF

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Storage systems and methods for photographers
« Reply #5 on: February 01, 2017, 11:47:21 am »

I've been hard at work typing this up, so read to get a bit of an idea how I believe photographers and storage line up.  It's about 70% but I keep seeing incorrect or baseless articles come out and I needed to speak out.

https://medium.com/@PNWMF/storage-systems-and-methods-for-photographers-86e04f940013#.uzejsolsi

Joe, I think this is great.

A few thoughts in general first.

You might want to back up a bit and talk about the difference between backup and archiving, like I did in this (unfortunately, by now somewhat dated) article:

http://blog.kasson.com/how-to/backing-up-photographic-images/

You should also talk about off-site vs on-site storage, and storage hierarchies (maybe using another name to make it easy on the less technical).

Now, some specific comments:

You say: “Direct Attach can run at a much faster speed than your wifi or wired network can (in most cases).”

You might want to amplify that. Specifically, I’ve found that Synology Rack NAS boxes attached with a single 1Gb/s Ethernet are generally as fast, and often faster, that USB3 drives. External USB3 drive enclosures sometimes provide much slower transfer speeds than my NAS connections, which run at wire speed. Even though USB3 wire speed is five times higher, I’ve never seen much improvement over NAS speeds. In addition, it seems that many USB3 implementations share bandwidth, so that other USB3 transfers can slow down (of course, that’s true if you only have a single Ethernet connection). 10Gb/s Ethernet is finally dropping in price, but that’s still beyond the scope of your intended audience, I think, as is Ethernet bonding. It is a surprise to me that the more complicated protocol stack for Ethernet and either IP or Windows Domain networking runs faster than the leaner USB3 DASD protocol, but that seems to be the case. Maybe more time to optimize the code? Maybe hardware restrictions in USB3 implementations?

You say: “Returning to the backup topic, most cloud backup platforms works fine with DAS disks, but not for NAS setups.”

I think you’re talking about the cloud backup services that use their own client software. I don’t use those. I’ve tried many, and found they are generally opaque, inflexible, and buggy. For cloud backup (and for on-site backup), I prefer a third part backup app like GoodSync, although that limits you to cloud backup services that don’t require their own apps, like Amazon S3 or Dropbox.

When you talk about Wi-Fi, you talk about it coming from a router that supports it. That’s not how I do it, and there are probably others that think like me. I have a router connected to my ISP. The LAN side of that router is connected to the WAN side of a SonicWall firewall, the LAN side of the firewall is connected, via Ethernet switches, to wireless access points distributed about the house. This allows for more reliable Wi-Fi and greater coverage than I could possible get if I bought a router with built-in Wi-Fi.

You talk about enterprise SAS drives (which I agree are overkill) and (by implication) consumer SATA drives, but you don’t mention my preferred choice for spinning in arrays, which is enterprise SATA drives. I’ve gotten no failures since I started to switch over to them about two years ago (I have about 35 drives running, so the sample size is small), and I’m always nervous during rebuilds, even with RAID 6 NAS boxes. On the remaining commercial drive arrays, I’ve configured hot spares, so if I don’t get to the server room for a week or two to notice a failure, the array has already rebuilt itself. I really should configure email alerts.

I like what you’ve said about striping. I used to be a big fan of three striped 8TB Helium disks for first-line photo storage. I’m beginning to think about SSD for that, but at present, that’s a lot of disks. Your remarks about software RAID also intrigue me. I’ll have to look at that. I’ve been using PCIe RAID controllers for the striping. I don’t use RAID5 and 6 on workstations, but aren’t there write speed problems with software versions of those, since the software has no NV RAM at its disposal? Or do the hardware RAIDs use NV RAM anymore?

You say: “if you have an Adaptec RAID card and it throws a fit and dies, you have to get a similar Adaptec RAID card to read the contents of the disk.” I don’t understand this. Why not just put in whatever the heck you want and restore from a backup? If the RAID controller dies, your array is out of service anyway, and will be for several days unless you have a spare controller on-site.

Anyway, that’s my two cents. I’m looking forward to reading the complete paper.

Jim





Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1365
Re: Storage systems and methods for photographers
« Reply #6 on: February 01, 2017, 04:12:13 pm »

Thanks for the great feedback Jim.  I need to go into how separating your active work from your archive work is critical, mostly because RTO on the archive can be a week or two, while the active work is hours at worst.  I also need to talk about how to scale a system when you outgrow a NAS and need to either buy the next bigger size or hope their drive expansion works as promised.

As for enterprise SATA drives, I've seen some interesting info come out of Backblaze: https://www.backblaze.com/blog/hard-drive-benchmark-stats-2016/

I'm bugging OWC to give us a way to mix & match drives in their ThurnderBay product - mixing SSDs and drives of different sizes would make setup for end users much easier.  Then again, I'd love an 8 bay setup, mix in a few SSDs with some spinning rust and such.  Highpoint has one, but it's an external TB-SAS and then SAS expanders inside.

There's lots to talk about, and how tech keeps changing.  There are upsides to every decision, and there is an associated cost, and what we keep trying to do is maximize the value per dollar we put in.  The way some other blogs are talking about storage and needing RAID and all this stuff, it's making blanket assumptions.  Greatest example of non-RAID user is BCooter and his/their stacks of Lacie rugged external drives.
Logged
t: @PNWMF

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Storage systems and methods for photographers
« Reply #7 on: February 01, 2017, 04:43:39 pm »

Thanks for the great feedback Jim.  I need to go into how separating your active work from your archive work is critical, mostly because RTO on the archive can be a week or two, while the active work is hours at worst.  I also need to talk about how to scale a system when you outgrow a NAS and need to either buy the next bigger size or hope their drive expansion works as promised.

As for enterprise SATA drives, I've seen some interesting info come out of Backblaze: https://www.backblaze.com/blog/hard-drive-benchmark-stats-2016/

I'm bugging OWC to give us a way to mix & match drives in their ThurnderBay product - mixing SSDs and drives of different sizes would make setup for end users much easier.  Then again, I'd love an 8 bay setup, mix in a few SSDs with some spinning rust and such.  Highpoint has one, but it's an external TB-SAS and then SAS expanders inside.

There's lots to talk about, and how tech keeps changing.  There are upsides to every decision, and there is an associated cost, and what we keep trying to do is maximize the value per dollar we put in.  The way some other blogs are talking about storage and needing RAID and all this stuff, it's making blanket assumptions.  Greatest example of non-RAID user is BCooter and his/their stacks of Lacie rugged external drives.

Joe, there's another point that can't be emphasized too much: if you're not doing trial restores every so often (to backup, but direct attached, stores), the you don't know that your backup solution is working. I had an example of that occur to me recently, although not in a workstation environment. I was setting up three test WordPress instances. I created fresh instances, loaded UpDraft, and tried to restore from my Dropbox account. The restores failed with "corrupt zip" messages. I looked at the files on Dropbox, and they were indeed corrupt.  I restored from an ftp copy, and it worked. Then I told UpDraft to write the live site files to its own storage servers instead of Dropbox. I created new fresh instances and restored to them with UpDarft using the backup copies on its own servers. That worked. So somehow the files were being corrupted as UpDraft was writing them to Dropbox.

Let's say that I never did what amounted to a trial restore. And let's say that I didn't have ftp'd copies of all the files stored locally thanks to a GoodSync script. Now let's say I'd lost the live site. I'd have been SOL.

And there's a psychological reason to do trial restores. You can practice when you're not in a panic, and you'll know what to do when you have to do a restore in anger. In work environments in the past, I can't tell you how many times I've seen a bad situation made much worse by dumb mistakes caused by IT folks who were frantic and not thinking clearly.

Automated backup is wonderful, but you then have the possibility of having wholesale errors instead of the retail ones you get with less automation. Like having someone copy a corrupt set of files over your backup copy. Sounds impossible, doesn't it? Just have two ftp clients open simultaneously and get confused which is which, and it can happen.

Jim

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1365
Re: Storage systems and methods for photographers
« Reply #8 on: February 01, 2017, 05:32:01 pm »

if you're not doing trial restores every so often (to backup, but direct attached, stores), the you don't know that your backup solution is working.

...
Jim

+1000 - it's part of why I like sync rather than backup, partly because I don't have to find the license & install the software to recover a file, but I can literally just plug it in and browse the data at random.  Organization is the base of any workflow, the more organized you can be, the better.
Logged
t: @PNWMF

degrub

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1947
Re: Storage systems and methods for photographers
« Reply #9 on: February 01, 2017, 06:05:17 pm »

that is correct - sync is not backup. Once a file is corrupted on the sync media, the change can be picked up on the next sync. or vice versa depending on the trigger.
Frank
Logged

scyth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
Re: Storage systems and methods for photographers
« Reply #10 on: February 01, 2017, 06:12:58 pm »

Logged

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Storage systems and methods for photographers
« Reply #11 on: February 01, 2017, 06:18:42 pm »

+1000 - it's part of why I like sync rather than backup, partly because I don't have to find the license & install the software to recover a file, but I can literally just plug it in and browse the data at random.  Organization is the base of any workflow, the more organized you can be, the better.

I'm with you, except maybe for the terminology. In the apps that I use for backup, GoodSync and Vice Versa, the difference between synch and backup is just a checkbox when you set up each script. In either case, the backed up files are browsable and restorable randomly, using any program you please. The difference is that with synch, changes on the backup dise are propagated to the live side, which I consider dangerous.

But just looking at the files on the backup is not enough; you need to establish that you can restore them, and that they're not corrupted after the restore, partly so you know all that works, and partly to train yourself so you know what to do when you do it for real. Sounds like you agree with that, too.

OBTW, one of the big problems with doing backups by disk images is that you can hardly ever get anyone to do a trial restore on top of their perfectly good workstation.

Jim

Jim

Tony Jay

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2965
Re: Storage systems and methods for photographers
« Reply #12 on: February 01, 2017, 08:17:55 pm »

I would definitely encourage you to complete the task!

I think that this particular aspect of digital asset management is probably the Achilles heel of a lot of photographers.
Perhaps even a couple of example back-up/archiving workflows could be constructed with relevant hardware and software components along with the relative pros and cons of each approach could be considered.

I appreciate the work that has already gone into explaining the various options and configurations that are practically available however to those who are perhaps not on the level of an IT professional it is a bit like presenting the occasional mechanic with a truck full of spare parts - put together correctly one gets a lamborghini - but to the occasional mechanic they are destined to forever remain a pile of spare parts!

I have so frequently come across individuals who fail to get the "big picture".
In a different, but related context, I recently had dealings with a photographer who run into trouble converting images to DNG. Her reasoning for doing this was to get rid of the annoying XMP side-car files accompanying her raw images. She was voicing concerns about how she would now have to "manually" update the previews in the DNG files to get accurate previews (this despite the fact that she only used Lightroom and had no intention of using any other software for viewing images). When I pointed out the mechanisms for doing this in Lightroom (as well as the point that it was irrelevant for her purposes) she primly pointed out that she did not want to write metadata back to file anyway. When I asked her what the origin of the XMP side-car files were, I got the strange answer that this was necessary to ensure that her edits were saved! When I explained that every edit, including the history was automatically saved into the catalog, it was clear that she had no idea.

I think this story illustrates a common problem in digital asset management where individuals are using the "tools" without really understanding the bigger picture and how things logically interrelate.
Even more interesting was that I discovered that her back-up and archiving regime was pretty good - in complete contradistinction to her application of digital asset management within Lightroom.
(I had fears that, given the first part of the story related above, that the back-up strategy would be a mess - but it was not the case.)
I am pretty sure that her back-ups were designed and implemented by somebody else though.

So, in summary, I feel that the background information you have shared is great, but would probably be lacking without a couple of concrete, if simply representative, workflow examples.

Tony Jay
Logged

davidgp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 758
    • davidgp fotografia
Re: Storage systems and methods for photographers
« Reply #13 on: February 02, 2017, 02:21:55 am »

Hi Joe,

I see that Jim already gave you a lot of suggestions... Not much experience with RAID systems, so I can not comment there, but the topic of trying to restore files from your backup I fully support it.

Now, since you are mentioning NAS systems like Synology... I will suggest for those users, if possible, to use BTRFS filesystem over EXT4 since it is resilient to bit-roting https://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/

For Apple users, looks like they are going to do a transition to a new filesystem this year or next one (long overdue), I hope they also add bit-rooting support, I think the beta versions available right now does not have it. Not sure for Windows if NTFS supports it, but I will suspect they don't


Enviado desde mi iPad utilizando Tapatalk

Jim Kasson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2370
    • The Last Word
Re: Storage systems and methods for photographers
« Reply #14 on: February 02, 2017, 04:06:26 pm »


As for enterprise SATA drives, I've seen some interesting info come out of Backblaze: https://www.backblaze.com/blog/hard-drive-benchmark-stats-2016/

Thanks. I switched to HGST drives for workstations the year before last, and HGST Helium drives for new hotswap arrays last year. I think that enterprise drives do make sense if not easily swappable, and may make sense if they are. Looks like HGST is their most reliable vendor, although they don't have much experience with HGST 8TB drives.

Jim

davidgp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 758
    • davidgp fotografia
Re: Storage systems and methods for photographers
« Reply #15 on: February 03, 2017, 03:11:03 am »

Thanks. I switched to HGST drives for workstations the year before last, and HGST Helium drives for new hotswap arrays last year. I think that enterprise drives do make sense if not easily swappable, and may make sense if they are. Looks like HGST is their most reliable vendor, although they don't have much experience with HGST 8TB drives.

Jim

Hi Jim,

Being a Backblaze user I have being following their studies for a long time. Even if initial stats for them show that model X is better than model Y. The problem for them it is that maybe it is more cheaper and easy to get drive Y in large quantities than X... This is the reason you see they buy more Seagates than other brand, even after their problems with the 1.5 or 3 tb drives that they have years ago...

Also, it was last year that they started to buy 8TB drives in large quantities, after the ratio dollar per gigabyte was interesting for them (you also have to consider that they put 45 drives into each rack unit, so for them, using less rack units is part of the equation).

But, they always have some rack units with newer and expensive drives to test the waters before buying large quantities of an specific drive.

Regards,

David

Dan Wells

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1044
Re: Storage systems and methods for photographers
« Reply #16 on: February 17, 2017, 12:10:03 am »

QNAP has an interesting new product out there that combines some of the advantages of NAS and DAS - it's a NAS (actually a series of them - they come in 4, 6 and 8 bay versions that are confusingly named the 682T (4-bay), 882T (6-bay) and 1282T (8-bay)) that connect via gigabit Ethernet, 10 Gig Ethernet (fast, but you don't have anything else that uses it) AND Thunderbolt. The odd naming scheme is because they're counting dedicated SSD caching bays (2 on the 682T and 882T, 4 on the 1282T) as drive bays - if you include the 2.5" SSD bays, a 682T DOES have 6 bays - only 4 of which accept regular hard drives... To further confuse things, it accepts 2 M.2 SSDs in addition to the 2 2.5"SSDs, and it doesn't count those slots as bays!

The advantage of the Thunderbolt connection is that it's extremely fast. I just got a 682T, and I've seen it break half a gigabyte per second consistently as I've transferred data to it (from the very quick PCIe SSD on my MacBook Pro). I've never seen another external drive go that fast in the real world, and the closest I've seen anything come is very expensive Fibre Channel RAIDs (I suspect a modern Thunderbolt DAS with SSD caching would be in the same range, and a Thunderbolt SSD might be close). Yes, USB 3.0 is theoretically rated that fast if you had enough fast drives connected to it, but I've never seen a USB 3.0 device come CLOSE to that - I've rarely seen one break 200 MB/s(even with a SSD in a USB 3.0 housing). USB of any variety always seems to have overhead that keeps it well below its rated speed, which only matters if your drives are very quick (fast RAID or SSD) to begin with, because USB 3.0 is faster than any single spinning drive. Standard gigabit NAS units are about as fast as most USB 3.0 drives - I've seen them get very close to their wire speed of 128 megabytes (1 gigabit) per second. The QNAP is more than 4 times faster than a conventional NAS, three times as fast as a fast USB 3.0 drive, and will hold its own against even the fastest Thunderbolt RAIDs (I've used an 8-bay GSPEED Shuttle, among the fastest Thunderbolt RAIDs around, and it isn't any faster).

Unlike a standard Thunderbolt enclosure, the QNAP system simultaneously functions as a NAS. It's designed to be hooked up over Thunderbolt and Ethernet at the same time, using the obscure networking function of Thunderbolt - it actually adds an extra, very fast network to your computer, running over Thunderbolt cabling (which means you can't daisy-chain anything else to the same Thunderbolt port - a problem if you only have one port).  . You can access the NAS from any other computer on the same network, and even use cloud-based functions in the same way as any other QNAP or Synology unit. Of course, those functions won't be blindingly fast - they'll depend on the network infrastructure like any other NAS - quite fast if it shares a gigabit network segment, very slow if you're running cloud-based functions over a cell connection. If you have a 10 gigabit network, it should run extremely fast, using its built-in 10 gigabit support (I haven't been able to test this, because I have no other device with 10 gigabit support).

Because it is a NAS that also happens to have a fast direct connection, it has most of the advantages and drawbacks of a NAS. It can perform a substantial number of functions on its own, including backing itself up to an external drive, offloading memory cards directly, and serving as a media server without tying up a computer. It can be set up as a Time Machine target, backing up all the Macs on the network (and making a multiple backup if it is further set up to back itself up to another drive). It will even share a printer over the network, although I wouldn't try this with a photo printer. On the other hand, it is comparable to any other NAS to set up, not to a Thunderbolt RAID (which is plug and go). The setup is wizard-based and takes less than half an hour, but it DOES require a modicum of knowledge and could intimidate less technical users.

Yes, it's an expensive unit, especially for smaller capacities where the very expensive enclosure will overwhelm the drive costs. The enclosure alone is $1900 for the 4-bay 682T, which is substantially more expensive than most NASs and many Thunderbolt RAIDs. Expensive Thunderbolt RAIDs come close to that price without drives, especially when you add features like sophisticated SSD caching, which the 682T and its big brothers have.

Once you add large drives (and there is no reason to use a machine like this with any but big, fast drives), it is comparable in price to higher-end Thunderbolt RAID units, which are its primary competitors (a 32 GB (raw capacity - 24 TB once RAID 5 steals a drive) unit with HGST enterprise drives is about $800 more than the OtherWorld 4-bay RAID, which uses your Mac to do the RAID calculations and uses cheaper drives , but is actually about $100 cheaper than the 4 bay, 32 TB GSPEED Studio, which does the work in the enclosure like the QNAP and uses the same drives). I wish QNAP made a version without the 10 gigabit Ethernet, a multi-hundred dollar feature that most photographers and videographers aren't set up to use, although perhaps a useful future-proofing.

As well as being a worthwhile consideration at the high end of the storage market, the QNAP Thunderbolt NAS series serve as a technology demonstration - QNAP or somebody else could put Thunderbolt networking in a less expensive NAS box that didn't feature 10 gigabit Ethernet, a very fast processor or SSD caching. A 4 bay unit in the $800 (empty) range should be possible by eliminating the most expensive non-Thunderbolt features from the present line? That would be entirely competitive with higher-end conventional NAS units and midrange Thunderbolt RAIDs, and the flexibility to be both at once would appeal to many users. A $1200 unit could keep the sophisticated SSD caching, but drop the 10 gigabit Ethernet and downgrade the processor somewhat. Something in that range would compete with the better Thunderbolt RAID units, and actually undercut many of them on price.
Logged

Pictus

  • Full Member
  • ***
  • Offline Offline
  • Posts: 216
    • Retouching
Re: Storage systems and methods for photographers
« Reply #17 on: February 17, 2017, 09:50:01 am »

Intel Atom C2000 chips are bricking products
http://www.guru3d.com/news-story/intel-atom-c2000-chips-are-bricking-products.html

"Folks with Synology DS1815+ NAS boxes have been reporting complete hardware failures; the DS1815+
is powered by an Intel Atom C2538. Other vendors using Atom C2000 chips include Asrock, Aaeon, HP,
Infortrend, Lanner, NEC, Newisys, Netgate, Netgear, Quanta, Supermicro, and ZNYX Networks.
The chipset is aimed at networking devices, storage systems, and microserver workloads."
Logged

Dan Wells

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1044
Re: Storage systems and methods for photographers
« Reply #18 on: February 17, 2017, 05:45:00 pm »

This should get out broadly to the photographic community! Many low-end to midrange NAS boxes use the affected Atom chips (some similar unitsuse ARM chips, many midrange boxes use Pentium series chips one step above the Atoms, and faster NAS units often use a Core i3 or even higher). I was a bit surprised to find out that the popular Synology 1515+ and 1815+ units are Atom-based (I thought they were Pentiums). How many photographers have one of these (or any number of other NAS boxes) and don't even realize that their "disk drive" has a processor? They're easy enough to set up that many people don't think of them as servers (which is, of course, what they are).  Several of the WD My Cloud series use similar Atoms, and those are perhaps even more dangerous because they are always sold with disks in them, and they take even less user involvement to set up - my guess is that 80% of their owners have never paused to think about their processor...
Five years ago, servers were tricky enough to set up that nobody had a file server on their network without knowing what it was (unless an IT consultant had installed it without explanation)...  Now, you can buy a file server at Best Buy for under $200 for the least expensive WD and Seagate single-drive units, and just plug it into your router. They're a fantastic convenience, but something like this shows us that they're also complex devices that need to be understood.
Network hardware in general is getting cheap enough that misunderstood devices are cropping up everywhere. Routers plugged into routers (although, ironically, that configuration makes life difficult for hackers because of double address translation, so there's some reason to use two routers in succession - or the first router could be a cheapie provided by the cable company, while your whole network runs off the second), lots of "accidental servers" (not just file servers - the worst of it is all the TVs, thermostats and what have you that broadcast web interfaces - is the convenience of having your fridge online really worth the unsecured web server?), and even WiFi drives meant to provide extra storage to cell phones, but that connect to random networks.

Dan
Logged

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1365
Re: Storage systems and methods for photographers
« Reply #19 on: February 22, 2017, 02:52:26 pm »

I haven't seen the issues with the Atom processors, but it's a great reason to check your backups more frequently AND make sure you can work around an issue that crops up.

The QNAP boxes are interesting.  For starters, they aren't Thunderbolt enclosures, but 10Gig Ethernet NAS with a Thunderbolt port that'll let you do peer to peer networking at 10gbps.  Take a peak at these: https://www.asus.com/us/Motherboard-Accessory/ThunderboltEX_IIDUAL/   So basically they are removing the end user from setting up a very small 10gbps network, which is a good thing.  A true Thunderbolt enclosure takes the PCIe and works from there, doing a PCIe to PCIe bridge, then adapters & controller cards to devices like hard drives.  The easy test is if it works without a computer attached, it's got one inside.

10Gbps networking is expensive, with the switches costing ~$100 per port, and adapters ranging anywhere from $30 to $200 each.  I am actually looking forward to more of the 802.11bz aka 2.5gbps and 5gbps over cat5e & 6 respectively. 
Logged
t: @PNWMF
Pages: [1] 2 3   Go Up