Pages: [1] 2   Go Down

Author Topic: Next gen Thun 3 8 disk array advice?  (Read 2115 times)

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Next gen Thun 3 8 disk array advice?
« on: March 12, 2018, 12:48:50 AM »

Team,

I am about to run out of capacity on my trusted Thunderbolt 2 Pegasus2 array.

I am considering the following options:
- Promise Pegasus3 R8
- G-Technology Shuttle XL 8 bays
- Areca ARC arc-8050t3 8-bays

Anybody with first hand experiences in terms of (in decreasing importance for me):
- Reliability (connection issues, failures,...)
- Speed and speed variability
- cost performance
- Ease of management

Thanks.

Cheers,
Bernard

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 881
Re: Next gen Thun 3 8 disk array advice?
« Reply #1 on: March 12, 2018, 08:43:28 AM »

How big is your current array, and what size are you thinking of moving to?

Do you need to have everything DAS, or can you migrate off some old stuff to a NAS setup?

What other storage tier-ing (fastest, slower, backups, etc) do you have going on in your workflow?

-Joe
Logged
t: @PNWMF

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Next gen Thun 3 8 disk array advice?
« Reply #2 on: March 12, 2018, 09:57:20 AM »

How big is your current array, and what size are you thinking of moving to?

Do you need to have everything DAS, or can you migrate off some old stuff to a NAS setup?

What other storage tier-ing (fastest, slower, backups, etc) do you have going on in your workflow?

Thanks Joe.

I am currently using a 32TB Pegasus2 in Raid 6 as main drive, backed up to a 32TB qnap Nas that just died on me and is being replaced by a Synology 8x10=80TB unit (also Raid 6).

I am looking at having all my data available on live fast storage with the exact same configuration as my backup, so I am looking for a 8x10TB Thunderbolt 3 array that I will configure in Raid 6 as well.

Cheers,
Bernard

mcbroomf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 635
    • Mike Broomfield
Re: Next gen Thun 3 8 disk array advice?
« Reply #3 on: March 12, 2018, 01:33:39 PM »

Hi Bernard,
I can't help with any info or opinions to your original questions, but it would be great if you could post an update when you make a decision and see some performance/rel results.  although too late for me (and you have larger storage requirements I think) I just ordered a new PC build from Puget with TB3 and will be ordering an Akitio Thunder 3 Quad as my primary backup storage (primary storage will be on-board and 2nd backup is a NAS).

Also interested in which drives you choose.

Regards

Mike
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Next gen Thun 3 8 disk array advice?
« Reply #4 on: March 13, 2018, 05:10:43 AM »

Hi Bernard,
I can't help with any info or opinions to your original questions, but it would be great if you could post an update when you make a decision and see some performance/rel results.  although too late for me (and you have larger storage requirements I think) I just ordered a new PC build from Puget with TB3 and will be ordering an Akitio Thunder 3 Quad as my primary backup storage (primary storage will be on-board and 2nd backup is a NAS).

Also interested in which drives you choose.

Sure, I will.

For now I have chosen Seagate Ironwolf 10TB disks for my new Synology NAS.

Cheers,
Bernard

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 881
Re: Next gen Thun 3 8 disk array advice?
« Reply #5 on: March 13, 2018, 11:45:51 AM »

Checking to see if you're going to go 10gbps with the NAS.  I'd go with the SSD accelerator on the Synology then trunk the 4x1gbps ports into a 10gbps switch. TB3 - 802.3bz(NBase-T) is pretty amazing, especially when 1gbps will be the bottleneck between storage units.
Logged
t: @PNWMF

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Next gen Thun 3 8 disk array advice?
« Reply #6 on: March 13, 2018, 08:18:50 PM »

Checking to see if you're going to go 10gbps with the NAS.  I'd go with the SSD accelerator on the Synology then trunk the 4x1gbps ports into a 10gbps switch. TB3 - 802.3bz(NBase-T) is pretty amazing, especially when 1gbps will be the bottleneck between storage units.

No, I just went for a unit with 1Gbps ports. The nas is just a back up and although the initial copy of the 23TB of data is going to take 4 days, the subsequent delta back ups will be quite fast.

I have therefore decided to spend more money on live storage than on the back up nas.

The previous qnap nas was higher end... but it didn't prevent it from dying on me and I never really leveraged its potential.

Cheers,
Bernard

mkihne

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 81
Re: Next gen Thun 3 8 disk array advice?
« Reply #7 on: March 13, 2018, 11:05:43 PM »

I have had the Areca 8050 tb 2 for about about two years. I used WD 4tb SE blue (enterprise) drives for 32TB in raid 10 for 16 TB useable. Instructions left something to be desired but not too difficult to set up. Worked well. Very fast. No disk troubles. Had a hot spare as well. Takes a while to build. Fan noise on startup, but quiet enough for me next to my computer. At one year, I had the upper fan go out and gave up on phone tech and replaced it myself. Actually ran it that way for 4 months with no temp issues. No temp issues on one fan, so good to know.

Regardless of the fan issue, I purchased another 8050 tb 3, again with 4tb WD enterprise (yellow) drives. Better instructions this time. You can use it in limited capacity while it builds but I left it over nite. Same on the noise(almost nonexistent). Again very fast. Raid 10. If I had significant video, would do Raid 5 (recommended). My previous Areca is still working well in a backup situation.

I have a Pegasus 4 bay on tb 2 I’ve had since my original Areca 16tb mirrored with no issues. I bought the bare case from Apple but I have not seen it offered again, only populated with less satisfactory drives, so did not repurchase a Promise.
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Next gen Thun 3 8 disk array advice?
« Reply #8 on: March 13, 2018, 11:24:14 PM »

Thanks a lot!

Areca is indeed an interesting option too.

Cheers,
Bernard

mkihne

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 81
Re: Next gen Thun 3 8 disk array advice?
« Reply #9 on: March 14, 2018, 12:34:49 AM »

Just curious Bernard, why raid 6 over raid 1+0? I can understand raid 6 over raid 5, but why not raid 1+0? Other than some economies on available disc space, the write performance hit vs 1+0 and performance degradation during rebuilds on raid 6 were concerns for me in going with 1+0. Also several articles on the likelihood of raid 6 going out of favor similar to what has happened to raid 5, especially with larger drives and/or 3tb(especially Seagate), apparently a carryover from 1.5tb drives.

Mike
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Next gen Thun 3 8 disk array advice?
« Reply #10 on: March 14, 2018, 04:06:26 AM »

If I am not mistaken, with Raid 10, if you loose one drive on both sides at the same time, you are dead, right?

With Raid 6, you can loose 2 drives and still be able to re-build the array.

But I am far from being a raid expert, this is just my understanding.

Cheers,
Bernard

mkihne

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 81
Re: Next gen Thun 3 8 disk array advice?
« Reply #11 on: March 14, 2018, 10:49:22 AM »

As the Areca is my working drive and I have various other means of backup, I don’t select on the basis of failures. Raid 10 will fail completely only if two drives containing identical information (identical one from each mirror) fail at the same time. I could lose 4 drives if all from the same mirror and still be working without much of a performance hit until rebuilt.

I do not take a parity write hit while using my working drive (ie: x2 for raid 10 vs x6 for raid 6=significantly slower writes for raid 6, even raid 5 with same number of drives is faster(x3) than raid 6 writes. Raid 10 has no parity writes.

In case of a drive failure, my recovery is a background activity with virtually no discernible performance hit unlike raid 6 which may severely decrease performance, important to me on my working drive. As you can see above, I use enterprise drives because those whose livelihood depends on it use them(yes I know about Back Blaze). I consider their data interesting but not very pertinent to my situation, other than brands, disc size failures, etc. Seagate drives of certain sizes seem to be favored now, but what next?

As discs get larger in capacity, it is fairly well accepted that the likelihood of an URE increases during rebuild and with the slower rebuild of raid 6 leaves a fairly long time window vulnerability for raid 6 during rebuild. In the enterprise arena, no one is recommending raid 5 because of these and other issues and some interesting articles are appearing suggesting raid 6 will not be recommended beyond 2019. A Google search will give you some interesting results relating to this.

Everyone works a little differently, so I’m not critical of your decision but only curious as I obviously went through the same process when selecting my setup, reemphasizing that this is my working drive. I have mirrored drives and bare drive off site redundancies and my previous Areca 8050 tb2 as a mirror of my new 8050.

Mike
« Last Edit: March 14, 2018, 12:38:38 PM by mkihne »
Logged

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 881
Re: Next gen Thun 3 8 disk array advice?
« Reply #12 on: March 14, 2018, 12:07:15 PM »

If I am not mistaken, with Raid 10, if you loose one drive on both sides at the same time, you are dead, right?

With Raid 6, you can loose 2 drives and still be able to re-build the array.

In a RAID 1+0 you have a bunch of RAID1 pairs of drives, then a RAID0 across those RAID1 pairs.  In this implementation, you can lose half the drive, just as long as it's 1 drive from each pair.  In RAID 0+1, there are 2 RAID0 arrays, that are then setup in a RAID1 mirror - lose one drive and half your disks have to rebuild, lose a second drive from the other array and your data is gone.

Parity calculations aren't what they use to be - especially with the caching ability we currently have.  Parity rebuilds suck, but with these products and modern hardware, and they're faster than the hard drives can write the data out.  SATA disks are still only 7200 rpm, and can only do some 75 read/write operations per second each.  Thus the spinning piece of rust in the computer is the slowest part.

One area NAS'es are better than DAS is the caching or tiering options available - the Synology can take a card with 2x M2 SSD drives and it'll greatly improve the write speed (where there is that performance hit in RAID5/6).

Like everything, there is tradeoffs.  We trade off write performance for a larger amount of usable space.  If we want faster write, we trade off usable space, or use a more expensive drive (SAS, SSD).
Logged
t: @PNWMF

mkihne

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 81
Re: Next gen Thun 3 8 disk array advice?
« Reply #13 on: March 14, 2018, 02:20:48 PM »

Hi Joe,

I'm curious(again). Would you consider a SAS or NAS for your primary working drive for photography work, assuming Bernard's(or my) situation? I've laid out my situation in above posts. I have an iMac and MacPro which I use less now, so my working solution is to use external drives and mine are primarily thunderbolt connected. Others may have internal bays and use those in jbod or raid as I did with my old Mac Pros.

All of us could likely use top notch single drives, either spinners or SSD, but when you start moving files around or even during import, preview builds in LR, etc. you can spend a lot of time waiting for drives. Importing cards is obviously a bottleneck, but difficult to streamline. I use Lexar's dock module with a 500Gig drive module, to which I import, 2 CF modules and an SD module. I then move them over thru thunderbolt to my raid 10 drive and backup to other resources from there. To me, that would be best accomplished thru fast connections and fast drive setup(for me, raid).

I tend to do multiple backups before I begin any processing and then confirm my data by checking my data chain. I want speed with reasonable protection(redundancy) for this and do not reformat my cards for a period of time just in case. I like the relative efficiency of my work flow with protection of data first, but speed and efficiency next.

Therefore my case for DAS raid 10, resulting in reasonable redundancy and preserving best performance possible. I have also considered SAS solutions to be likely beyond the needs for a photography centered system like mine, especially since I do not work from a network. If I did, I would likely consider NAS, but only as a backup solution, not primary working drive. Also, NAS brings in possible network related issues which, by Murphy's law, always occur at the worst possible times. ;D

I will admit that I have not used NAS on a personal level, but professionally am aware of significant network related downtimes from a user but not an IT perspective. Just my opinion(and you know what they say about opinions). I'm an old dog but always willing to learn new tricks, however.

Mike
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11142
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Next gen Thun 3 8 disk array advice?
« Reply #14 on: March 15, 2018, 08:32:01 AM »

Those are good points, I need to study this more.

I could go with a 12 bays unit and still hit my 50~60 usable TB target even with Raid 10.

Thanks for the suggestion.

Cheers,
Bernard

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 881
Re: Next gen Thun 3 8 disk array advice?
« Reply #15 on: March 15, 2018, 08:34:26 AM »

Hey Mike,

Yep, opinions are like a lot of things, and the thing to keep in mind is that there are lots of ways to do things, none of which are wrong.  It comes down to, personal preference, environment, even bad experiences form how we decide things.  I even started writing a Medium article about storage bits, but need to keep working on it, as it's still a draft even though it's live, IMHO. 

For context, I've got around 25tb of mostly photos, and my primary work machine is a Macbook Pro that I use for my day job as well.  I have a 40tb Synology as my spinning primary storage, setup in RAID6 and have added the SSD caching feature.  My process is to import from cards to either the internal drive, or a USB-C enclosed SSD drive as my working space.  If I need the cards back in action asap, I'll immediately make a second copy from card to the NAS, otherwise I'll copy the processed files from the SSD's to the NAS before the cards go out.  My backups happen directly from the NAS to external USB3 drives, be it I'm home or not.

Comparing workflows, we both work off DAS first, but I'm quick to move off to NAS for long term storage, while you rely on the DAS RAID 10 for you long term primary storage.  If I need to work on something that has already been moved to the NAS, I'll copy it back to the SSD and do something else while the copy happens.  I tend to only use single spinning drives for backups.  I trade off having a large 'working' drive for a much faster, but smaller & portable SSD.

You've meet Murphy I see ;) Yea, he shows up at the worst of times.  I wouldn't consider a SAS based setup unless you're dealing with Terabytes of rewritten info constantly, and it's part of a larger NAS/SAN setup - the price point of SSDs, even doing mirrored drives, is much cheaper & faster than anything that spins.  The thing about SAS 10/15k drives is that they don't wear out after so many write cycles like SSD's do.  But at what point is the trade off of replacing the SSD every 6-18 months worth it?  Time is money, and I'd rather not sit around waiting.

NAS's have actually been pretty good to me over the years.  There are some major advantages on the technical level, things like ZFS that you wouldn't get on a Mac or PC, but there can be some challenges for those who aren't technically inclined.  I do IT for a living, so some of my selections aren't of the 'best for your average user', but I moved from a NAS4free ZFS setup to a Synology just because I didn't want to fuss with it and need to grow the amount of storage I had available. 

-Joe
Logged
t: @PNWMF

mkihne

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 81
Re: Next gen Thun 3 8 disk array advice?
« Reply #16 on: March 15, 2018, 09:02:22 AM »

Thanks for responding Joe.....food for thought. One of my reasons for asking is that I am updating my ancient home network/internet(a LAN, by definition, I guess) and may want to be sure it is capable and optimal for NAS if I so desire.

Mike
Logged

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 881
Re: Next gen Thun 3 8 disk array advice?
« Reply #17 on: March 16, 2018, 01:14:26 PM »

So for network speeds, it's an amazing time.  NBASE-T or 802.3bz allows for 2.5gbps and 5gbps over existing Cat5e/Cat6 respectively.  I'm looking at a TB3-10gbps adapter that's around $300, and Netgear makes a trunk-able switch that's 8x1gbps and 2x2.5/5/10gbps for $300.  The reason I want the trunk-able is so I can take the 4x1gbps ports from the Synology, since I can have either the 10gbps port, or the 2x M.2 SSD accelerators.

The one thing I didn't really touch on is how someones workspace ends up applying into the NAS or DAS setup.  If you have a nice desk setup you always work at, having a few extra cables to plug in, or are constantly attached since it's a desktop, can provide an extreme speed advantage.  If your workspace is mobile, as in where ever you set your computer down, then there are different considerations.  I can get a 50' CAT7 network cord for 10gbps of wired access, I can't find a 50' optical TB3 cord - just TB2 so far.

There will be more NBase-T gear coming out over the next few months, but we've got a great start.  Here's what I'm looking at as of 3/15/18
Netgear GS110EMX
AKiTiO Thunder3 or Promise SANLink3-t1 TB3 to 10gbps network adapter
ASUS XG-C100C 10gbps NIC

-Joe
Logged
t: @PNWMF

Chris Kern

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 508
    • Chris Kern's Eponymous Website
Re: Next gen Thun 3 8 disk array advice?
« Reply #18 on: March 16, 2018, 06:03:39 PM »

NAS's have actually been pretty good to me over the years.  There are some major advantages on the technical level, things like ZFS that you wouldn't get on a Mac or PC, but there can be some challenges for those who aren't technically inclined.

I've never understood why ZFS hasn't been more widely adopted.  If memory serves, at one point (just before Sun Microsystems was gobbled up by Oracle), Apple was reported to be planning to replace its native HFS filesystem with ZFS, but it never happened.  (Because Sun Microsystems was gobbled up by Oracle?)

Microsoft's NTFS would seem to be an even more logical candidate.  Is it still necessary to defragment NTFS on a regular basis?  Yugg.

For anyone with a large datastore to maintain—including most serious photographers, professional or amateur—filesystem integrity is a real issue.  I don't see the mainstream platform suppliers taking it very seriously.  We should.

Joe Towner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 881
Re: Next gen Thun 3 8 disk array advice?
« Reply #19 on: March 17, 2018, 11:33:35 AM »

Actually APFS takes some of the good things about ZFS and slowly is rolling them out.  BTFRS (used by Synology) also takes some of the upsides to ZFS and puts them to use.  WinFS tries, but I really don't need the overhead.

The funny thing is that I've seen lots of NAS4free & FreeNAS setups that are of the 45 disk size because ZFS is so much more resilient than hardware RAID can be.
Logged
t: @PNWMF
Pages: [1] 2   Go Up