Pages: [1] 2   Go Down

Author Topic: Thunderbolt  (Read 22180 times)

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Thunderbolt
« on: April 08, 2013, 08:54:23 pm »

Team,

Would those in the know have visibility on upcoming Thunderbolt raid storage options?

We hear rumors of updated Mac pros coming, I would personally be pairing it it with a fast 6 bays thunderbolt attached storage that would have to:
- Be silent,
- Be reliable,
- Be reasonnably afforable,
- Have an embedded Raid 5/6 engine.

I don't find many options out there meeting these basic criteria.

Would any of you have visibility on what else is coming our way from reputable vendors like Qnap, Synology, G-Technology, Wiebetech,...?

Cheers,
Bernard

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Thunderbolt (and 4K display support)
« Reply #1 on: April 08, 2013, 10:14:42 pm »

Would those in the know have visibility on upcoming Thunderbolt raid storage options?
Intel is happy to gather such news at https://thunderbolttechnology.net/news and NAB seems to be generating some news on this front, like the second generation (double speed) version and the new Blackmagic 4K Super 35mm camera with Thunderbolt port.


P. S. Intel's NAB announcement of third generation "Redwood Ridge" (DSL4510 and DSL4410) Thunderbolt controllers that support 4K displays (through version 1.2 of DisplayPort) seems like the last key ingredient for a worthwhile MacPro upgrade and "Retina Cinema Displays". They seem likely to pair with Intel's coming Haswell processors, which I think means June at the earliest.

Do not confuse this with the fourth generation double speed "Falcon Ridge" Thunderbolt also announced, but not coming till late this year or early 2014.
« Last Edit: April 09, 2013, 12:34:01 pm by BJL »
Logged

Ellis Vener

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2151
    • http://www.ellisvener.com
Re: Thunderbolt
« Reply #2 on: April 09, 2013, 12:35:40 pm »

There is the Drobo 5D. Much, much quieter than older Drobo units  and you can configure for the Drobo version of RAID 5 or 6 (but with more flexibility). The hardware seems to be more robust as well. Alas only 5 bays but unlike small RAID systems it is scalable in capacity. I've been running one fro about 5 months with WD red 3TB HDDs.  I also hav a 64GB M-SSD  installed to speed up accessing previews in Lightroom.
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Thunderbolt
« Reply #3 on: April 10, 2013, 09:34:09 pm »

There is the Drobo 5D. Much, much quieter than older Drobo units  and you can configure for the Drobo version of RAID 5 or 6 (but with more flexibility). The hardware seems to be more robust as well. Alas only 5 bays but unlike small RAID systems it is scalable in capacity. I've been running one fro about 5 months with WD red 3TB HDDs.  I also hav a 64GB M-SSD  installed to speed up accessing previews in Lightroom.

Thanks, Drobo seems to be an interesting option.

Cheers,
Bernard

John.Murray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 886
    • Images by Murray
Re: Thunderbolt
« Reply #4 on: April 10, 2013, 11:55:08 pm »

Promise Pegasus R6 - now accepts 4tb drives 16TB at RAID 6 .....
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Thunderbolt
« Reply #5 on: April 11, 2013, 07:02:54 am »

Promise Pegasus R6 - now accepts 4tb drives 16TB at RAID 6 .....

A couple of questions:

- Do they sell empty bays?
- How is the noise level?

Thanks.

tived

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 714
    • http://
Re: Thunderbolt
« Reply #6 on: April 11, 2013, 07:42:18 pm »

How fast is Thunderbolt? like moving data from external to internal devices... e.g. moving data

Henrik
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Re: Thunderbolt
« Reply #7 on: April 11, 2013, 08:04:02 pm »

How fast is Thunderbolt? like moving data from external to internal devices... e.g. moving data

No first hand experience, but in theory it is faster than the fastest available SSD based raid external devices.

All the more with HDD based raid, it should clearly remove any interface related bottleneck, so external storage becomes de facto as fast as internal one.

Cheers,
Bernard

John.Murray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 886
    • Images by Murray
Re: Thunderbolt
« Reply #8 on: April 11, 2013, 10:26:42 pm »

Bernard:  As far as I know, only available w/drives.  The enclosure itself is very quiet - dual variable speed fans at the rear, any noise will be from the drives themselves.

I'm a huge fan of Promise, very well made, well thought out - we use the larger rackmount sas units for enterprise storage, I once got an email from one unit warning me that one of the redundant controller batteries was overheating......

Tived:  Thunderbolt is basically a PCIe external bus - supporting 20GBs (10Gbs each direction - full duplex).  Intel just announced a new TB chipset that will double to 40GBs - compatible with existing connectors and cabling
« Last Edit: April 11, 2013, 10:30:24 pm by John.Murray »
Logged

kevk

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 94
    • Kevin Kucks Photography
Re: Thunderbolt
« Reply #9 on: April 11, 2013, 10:46:28 pm »

...Thunderbolt is basically a PCIe external bus...
...and I believe Thunderbolt is still pretty much Mac only - there are only a few PC laptops and motherboards that support it, and as far as I know there are no widely compatible Thunderpolt PCIe add-on cards for existing PCs with PCIe slots.

Kevin
Logged

tived

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 714
    • http://
Re: Thunderbolt
« Reply #10 on: April 11, 2013, 11:44:24 pm »

Thanks guys,

the speed sounds good 10-20GB/s each way, that will take a lot of drives to push this much data through though - BUT not impossible  ;D

as for the early adoption, yes Apple is  probably out first for the novelty, while there are alternatives in the Windows world, though not nearly as fast. We will just wait and see, this sort of investments isn't just something you do because it sounds good, letting it mature is always a good route to take.

Never the less its very interesting technology - got to keep an eye on it.

Henrik
Logged

John.Murray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 886
    • Images by Murray
Re: Thunderbolt
« Reply #11 on: April 12, 2013, 01:21:42 am »

...and I believe Thunderbolt is still pretty much Mac only - there are only a few PC laptops and motherboards that support it, and as far as I know there are no widely compatible Thunderpolt PCIe add-on cards for existing PCs with PCIe slots.

Kevin

Yes and no.  Intel worked closely with Apple on the initial rollout of the technology (was internally called lightpeak at Intel, intended to be an optical interface).  

Asus currently has TB offerings - you will not find a PCIe to TB "card", it needs to be implemented at the Mainboard's Chipset southbridge, the same way the internal PCIe bus is implemented.  Sorry if I confused anyone, what I really meant that TB is *equivalent* to a PCIe bus....

I'm dissapointed at the state of TB in the windows side; expecting 2nd generation socket 2011 (X79 - C6XX) mainboards to have TB - such is not the case (but then neither has Apple)  I'm also dissapointed at the lack of some innovation around this new technology; why hasn't anyone developed a TB display with internal GPU - what a perfect opportunity to intimately match GPU LUT's, at the vendors chosen bit depth, directly to the panel....  

« Last Edit: April 12, 2013, 02:28:00 am by John.Murray »
Logged

Craig Lamson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3264
    • Craig Lamson Photo Homepage
Re: Thunderbolt
« Reply #12 on: April 13, 2013, 01:35:19 pm »

Yes and no.  Intel worked closely with Apple on the initial rollout of the technology (was internally called lightpeak at Intel, intended to be an optical interface).  

Asus currently has TB offerings - you will not find a PCIe to TB "card", it needs to be implemented at the Mainboard's Chipset southbridge, the same way the internal PCIe bus is implemented.  Sorry if I confused anyone, what I really meant that TB is *equivalent* to a PCIe bus....

I'm dissapointed at the state of TB in the windows side; expecting 2nd generation socket 2011 (X79 - C6XX) mainboards to have TB - such is not the case (but then neither has Apple)  I'm also dissapointed at the lack of some innovation around this new technology; why hasn't anyone developed a TB display with internal GPU - what a perfect opportunity to intimately match GPU LUT's, at the vendors chosen bit depth, directly to the panel....  



I have  Gigabyte z77 board with Thunderbolt.  I have yet to use the ports.


http://www.gigabyte.us/products/product-page.aspx?pid=4279#ov
Logged
Craig Lamson Photo

James Clark

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2347
Re: Thunderbolt
« Reply #13 on: April 14, 2013, 02:15:31 pm »

I run my LR4 catalog and all images off of an external Thunderbolt drive - On my Mac Mini server (16GB RAM), the difference between that setup and running with all files on the internal SSD is negligible, and mostly nonexistent.  The only difference I see is that the TB external drive is a hair slower at loading preview images.
Logged

Ellis Vener

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2151
    • http://www.ellisvener.com
Re: Thunderbolt
« Reply #14 on: April 14, 2013, 02:24:05 pm »

Logged

andyptak

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 469
Re: Thunderbolt
« Reply #15 on: April 14, 2013, 07:17:50 pm »

I have an Asus laptop with Thunderbolt. Now that everyone seems to have orphaned Express Cards it's the only way to shoot a Phase One back tethered to a PC.

Love the Asus. I've always had good luck with their products and this is no exception
Logged

TStanding

  • Newbie
  • *
  • Offline Offline
  • Posts: 6
Re: RAID 5 reliability - was Thunderbolt
« Reply #16 on: April 22, 2013, 01:34:37 pm »

Worth a careful read: http://www.zdnet.com/blog/storage/why-raid-6-stops-working-in-2019/805

Can someone help me understand the math here?  The spec for the Seagate 4 TB disks is one unrecoverable error every 10^14 bits.  If I assume 10 bits per byte (8 for encoding and 2 for error correction) as an estimate, then I should have one uncorrectable error every 10^13 bytes or on error every 2 * 10^10 sectors.  But the error rate used in Robin Harris's ZDNet article, quoted above has an error rate of one error every 2 * 10^8 sectors (one every 2 * 10^11 bytes).

Why is my number two orders of magnitude less than his?

Our empirical tests don't mirror his prediction either.  We have been running RAID 5 volumes flat out with a write/read/verify test for days on end.  I know the test is working because it has flushed out many obscure bugs in our driver, some of which take 2 - 6 days to reproduce.  Our typical test run is 48 hours on three separate Macs with a total of over 20 TB read and verified (1.2 billion i/os).  I think we've run this test 5 times so far and I've yet to see a single read error from the disk.  We have code which logs the read errors to the system.log file which I know works as we've been shipping the error logging code for 3 years.

I'ld love to hear other people's comments on the ZDNet article and their experience with the reliability of RAID 5 volumes

Thanks,

Tim Standing
SoftRAID LLC
Logged

John.Murray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 886
    • Images by Murray
Re: Thunderbolt
« Reply #17 on: April 24, 2013, 12:19:52 am »

My biggest problem with any form of RAID that involves parity, is the rebuild time when a spindle fails.  I've had numerous RAID 5 implementations run without a hitch for several years - the vast majority in fact.  However, we also experienced a degraded array in a high volume medical imaging application; seriously affecting processing for the several days it took to rebuild; all the while crossing our fingers hoping another another spindle would not fail (another actually did shortly after we removed the array from live service).  After determining the overall effect, we came to the conclusion we would have been better of restoring from a backup and losing a full day!

Never again; we have since stuck with only RAID10 (RAID 1 + 0) using dual controllers, rebuild times are much faster, offsetting the loss of overall storage capacity. 

Here's some quick examples of various RAID using 6 3TB drives

RAID Type: RAID5
Size (GB) = 15000
Number of disks = 6
Space efficiency = 0.833333333333333 (83.3333333333333%)
Fault tolerance = 1 disk(s)
RAID IO penalty (read) = 1/1 (one RAID IO per each host IO)
RAID IO penalty (write) = 4/1 (4 RAID IOs per each host IO)

RAID Type: RAID6
Size (GB) = 12000
Number of disks = 6
Space efficiency = 0.666666666666667 (66.6666666666667%)
Fault tolerance = 2 disk(s)
RAID IO penalty (read) = 1/1 (one RAID IO per each host IO)
RAID IO penalty (write) = 6/1 (6 RAID IOs per each host IO)

RAID Type: RAID10
Size (GB) = 9000
Number of disks = 6
Space efficiency = 0.5 (50%)
Fault tolerance = 1 disk (min) to 3 disks (max)
RAID IO penalty (read) = 1/1 (one RAID IO per each host IO)
RAID IO penalty (write) = 2/1 (2 RAID IOs per each host IO)

Although space efficiency for RAID10 is the lowest, take a look at Fault Tolerance and IO penalty.
Logged

paulrrulon

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Re: Thunderbolt
« Reply #18 on: April 26, 2013, 02:20:37 pm »

I am currently using the Western Digital thunderbolt drive 6 tb and am happy with it. I have about 2 tb of data on it & it works well. I am not sure if this would handle your needs as I am an amateur photographer and only shoot on weekends so 6 tb is adequate for my needs but might not be for yours. I do believe they make them in 8tb sizes also so that may be an option for you. After purchasing the d800, I had to upgrade my system to handle all those extra pixels, so I purchased the Thunderbolt external drive & bumped the ram up to 16 Meg. This should hold me for awhile. My next upgrade will be the ssd. However the price is still a little high so I shall wait a few more months & maybe it will come down. I am however using a desktop & not a laptop.  Here is a link.
 http://www.wdc.com/en/products/products.aspx?id=630
Hope this helps.
Logged

tived

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 714
    • http://
Re: Thunderbolt
« Reply #19 on: April 26, 2013, 07:11:52 pm »

Paul,

The WD, I am sure is very nice but two spindle disk will not challenge the Thunderbolt interface in terms of throughput.

John,

It's been a while since I was involved in building and setting up system for commercial clients, back then they were always crying over the cost of doing it right, right is perhaps not the right word, but building systems where they are more efficient when rebuilding from failure as you described above. Getting an extra controller and more disks will save valuable time. So it's nice to see that you put this to practice.

All the best

Henrik
Logged
Pages: [1] 2   Go Up