Pages: 1 2 [3]   Go Down

Author Topic: Looking for large capacity storage solution  (Read 27930 times)

Christopher

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1499
    • http://www.hauser-photoart.com
Looking for large capacity storage solution
« Reply #40 on: November 04, 2009, 03:30:46 pm »

Quote from: knweiss
Let me quote from Google's paper Failure Trends in a Large Disk Drive Population (pdf): "The figure shows that failures do not increase when the average temperature increases. In fact, there is a clear trend showing that lower temperatures are associated with higher failure rates. Only at very high temperatures is there a slight reversal of this trend."


Well another fact is that there are very effective, completely noise free fans. They just cost more, which makes everything expensive.
Logged
Christopher Hauser
[email=chris@hauser-p

knweiss

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 71
Looking for large capacity storage solution
« Reply #41 on: November 04, 2009, 03:46:08 pm »

Quote from: BernardLanguillier
Thks for the detailed feedback.
Let me add one more thing: When you think about double digit TB storage: Don't forget file system you are going to use. If I would buy/build a storage solution with this capacity for myself I would make sure it runs ZFS (paper) which does end-to-end checksumming (plus many other advantages). I would not trust so much data to a last-gen file system like HFS+, NTFS, ext3/4, etc because 24 TB is an awful lot of data to lose. Too bad Apple recently decided to drop the ZFS project...
Logged

mmurph

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 506
    • http://
Looking for large capacity storage solution
« Reply #42 on: November 04, 2009, 03:54:15 pm »

Sorry, but no need to bother with apple on a server:

In computing, ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software, licensed under the Common Development and Distribution License (CDDL). The ZFS name is a trademark of Sun.[2]


Buy the Dell Xeon and slap it on!    

Or, on the high end, we ran many Sun servers for 1 million unique visitor login per month web sites (Fortune 5 company data center.)  "A" is just one certain niche, really workstations, not servers (who needs high end video, etc. on storage box).
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Looking for large capacity storage solution
« Reply #43 on: November 04, 2009, 04:22:56 pm »

Quote from: reburns
Christopher,

Another option is backup software like "NTI Shadow" that came with the ReadyNAS box I'm using.  It does real-time continuous backups.

Does it also delete the files on the backup unit when the files on the main drive are deleted? If it does then it is not a good solution.

Cheers,
Bernard

Christopher

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1499
    • http://www.hauser-photoart.com
Looking for large capacity storage solution
« Reply #44 on: November 04, 2009, 04:25:17 pm »

Quote from: BernardLanguillier
Does it also delete the files on the backup unit when the files on the main drive are deleted? If it does then it is not a good solution.

Cheers,
Bernard

I don't know that program, but I would expect that this feature is there, but can be customized or turned off.
Logged
Christopher Hauser
[email=chris@hauser-p

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
Looking for large capacity storage solution
« Reply #45 on: November 04, 2009, 04:26:47 pm »

Quote from: Christopher
The N7700PRO sounded quite nice, but some reviews of the N7700 said it is quite loud and only usable in a separate room, which would not work for me, does somebody know if they changed the fans used in the PRO version ?

Funny how these companies just don't seem to understand how their products will be used, isn't it?

MY current Wiebetech 6TB unit is great in all accounts but the noise level, although it is still pretty decent.

Apple manages to design totally silent Mac Pros, yet few storage companies are seemingly able to design a truly quiet enclosure.

Cheers,
Bernard

reburns

  • Full Member
  • ***
  • Offline Offline
  • Posts: 146
    • Ralph E Burns Photography
Looking for large capacity storage solution
« Reply #46 on: November 04, 2009, 04:39:00 pm »

Quote from: knweiss
Let me quote from Google's paper Failure Trends in a Large Disk Drive Population (pdf): "The figure shows that failures do not increase when the average temperature increases. In fact, there is a clear trend showing that lower temperatures are associated with higher failure rates. Only at very high temperatures is there a slight reversal of this trend."

Can you answer this?  Is there any known correlation between thermal cycling and failure?  I should think that heating & cooling will be more severe than maintaining a hot temperature because the thermal expansion will cause mechanical stresses on solder joints and connections.  It could be that if the average temperature is low but peak temperatures remain similar, the drive would have undergone more severe thermal cycling.  Presumably the Google folks looked for that correlation and didn't find anything worth reporting.  That goes into the persistant question if it's better to leave the system running 24/7 or power it down when not it use.  I once asked my buddy who worked doing disk drive testing and he didn't have a clear answer; "Sleep mode vs. thermal cycling is a never ending discussion.  Constantly “hot” is also a risk so I follow a middle ground.  If I know I wont be on the computer for a day or more I shut it down.  It’s the “green” thing to do.  Some drives only do a self diagnostic after a power cycle which means if a drive is starting to have issues and it runs constantly it may die w/o any notification.  Do all you can to keep the drives cool either way."
Logged

Christopher

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1499
    • http://www.hauser-photoart.com
Looking for large capacity storage solution
« Reply #47 on: November 04, 2009, 04:41:25 pm »

Quote from: BernardLanguillier
Funny how these companies just don't seem to understand how their products will be used, isn't it?

MY current Wiebetech 6TB unit is great in all accounts but the noise level, although it is still pretty decent.

Apple manages to design totally silent Mac Pros, yet few storage companies are seemingly able to design a truly quiet enclosure.

Cheers,
Bernard

Yes I read the review, which was quite nice until I cam to the point: This unit is clearly designed to be in a separate server room. We would not want to have this noise level in our office room. I just thought, great ..........

I know that for you it is not option, because you need a LOT more space than I do, but that is one main big point for the Lacie. It has one large 120mm fan, which is really silent.
Logged
Christopher Hauser
[email=chris@hauser-p

reburns

  • Full Member
  • ***
  • Offline Offline
  • Posts: 146
    • Ralph E Burns Photography
Looking for large capacity storage solution
« Reply #48 on: November 04, 2009, 04:42:56 pm »

Quote from: BernardLanguillier
Does it also delete the files on the backup unit when the files on the main drive are deleted? If it does then it is not a good solution.

Cheers,
Bernard

If you want me to research further, then it will benefit us both.  What I do know is that you select how many prior archive copies are left on the backup before deleting.  If it indeed deleted the file when you deleted the local copy then the Shadow would be completely worthless as user error is a common culprit.  I admit I've only been using the new system for about one month.
Logged

mmurph

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 506
    • http://
Looking for large capacity storage solution
« Reply #49 on: November 04, 2009, 05:08:33 pm »

Quote from: BernardLanguillier
yet few storage companies are seemingly able to design a truly quiet enclosure.

The Intel SS4200-E is rock solid and whisper quiet. Also fast and cheap!

http://www.newegg.com/Product/Product.aspx...&Tpk=ss4200


It is a core Linux 2.6 box made by Intel for OEM's. You can also buy it with Windows Home Server installed.

Celeron CPU, 512meg of RAM, dual fan, gigabit, temperature monitoring. You can upgrade to dual core and 2GB if you like.  Takes 4 drives.

I bought the base box for $135 on sale. Loaded it with 4x1.5 TB in RAID 10 - the best way to use the base as-is. (4x 1.5 TB WD Caviar Green ay $100 each shipped.   Total cost $535. This is 3rd line storage ...)

Boots in 1 minute, shuts down in 15 seconds.   The OS comes loaded on a 512meg flash drive. Swap out to whatever you want on that drive, or via IDE - FreeNAS, ZFS, etc. WHS install directions are on teh Intel site. Just requires a RS232 cable.


Rock solid, quiet, and **fast**:

This is from Small Net Builder as of 1 year ago:

Performance

It’s fair to say that the SS4200-E turned in screaming fast performance numbers.  For my comparisons, I looked at 1000 Mbps RAID 5 read and write performance for all BYOD NASes.  As the radio announcer says, “hold the calls – we have a new winner”.  


I **was** going to load Windows Home Server on the thing, but it is just too perfect as a Linux 2.6 box as-is. I login with SSH (via wireless n from my laptop) when the limited web interface doesn't tell me all I need to know.  Mostly it just sits there and does it's job all alone ....

I have seen them as low as $95 as open box. Last sale was a little too high, about $170.  I am going to buy 1-2 more just to play with.

I LOVE THIS BOX!!!!

(The top Newegg comment has a pretty good summary on hacking/using this box. $200 shipped w/o disks now, not on sale.  Quote, that I agree with:  "Fans are automatically spun down to be nearly inaudible after a brief blast during startup. This is the case with the standard CPU and also with an upgrade to dual core.")


Cheers!
Michael
« Last Edit: November 04, 2009, 07:02:17 pm by mmurph »
Logged

Dan Wells

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1044
Looking for large capacity storage solution
« Reply #50 on: December 17, 2009, 10:26:46 pm »

I've used a number of these types of storage solutions. First of all, I would never trust anything LaCie (to be fair, I've never used any of their four-drive boxes, but I've had a huge collection of failures on their dual-drive units (they sell a lot of very attractively priced mid-high capacity dual-drive systems, which I was using (and installing for others) a lot of for a while until I started to notice their elevated failure rate). A colleague has both an Apple XServe RAID (in for repair right now, but ran flawlessly for four years, and I'm pretty sure the repair won't result in data loss) and a newer Promise RAID (the unit Apple now sells), which is flawless after about a year and a half - never lost a drive on either one, and the Promise is RAID 6 (can tolerate double drive failure) PLUS hot spare (will replace a down drive automatically). The Promise is big, noisy and expensive, BUT it seems as close to foolproof as you can get - you'd have to be pretty blind not to notice a TRIPLE drive failure, and the drive you change in any failure always comes in as the hot spare, so there's no risk of detonating your data by pulling the wrong drive during a rebuild (you let the rebuild happen onto the hot spare BEFORE replacing the drive). The 16 bay Promise costs close to 15 grand fully populated, AND wants to live in the basement (noise), but it's as safe as you can get short of an offsite backup (and have you priced a T3 line lately?). One big advantage of 16 bays is that the percentage loss to parity is very low - RAID 5 always takes one drive for parity, and RAID 6 takes two, no matter how many drives there are in total, so the Promise running RAID 6 plus a hot spare actually loses less capacity than a 4-bay system running a barely adequate RAID 5!
     I'm using an 8-bay EZQuest myself right now, and it works well ($3500 for 8 TB - $5000 for 16 TB). Unlike the REALLY expensive systems, the intelligence on this sucker is in the PCI-E controller board, not the RAID case. I bet a homemade version of this could be put together somewhat cheaper - the RocketRAID card they use is about $400, then any 8-bay case should work - unless the 4 eSATA to single InfiniBand connector they use is very expensive. The case connects to the card using 2 InfiniBand connectors, each of which is carrying 4 eSATA channels (it's just combining the signals, not translating). My next one will be a similar concept, but probably homemade. I'll also use RAID 6 next time - this one is RAID 5...

                                                                                         -Dan
Logged

Plekto

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 551
Looking for large capacity storage solution
« Reply #51 on: December 20, 2009, 10:16:45 pm »

Quote from: reburns
Can you answer this?  Is there any known correlation between thermal cycling and failure?  I should think that heating & cooling will be more severe than maintaining a hot temperature because the thermal expansion will cause mechanical stresses on solder joints and connections.

What happens 90%+ of the time is that the controller board on the drive eats itself due to heat and thermal cycling.  I can't tell you how many drives I've recovered by swapping in an identical controller board from another drive.  Of course, the downside is that the controller board also keeps track of bad sectors and other necessary data, so this is only recommended for yank-the-data recovery operations and not as a method to get the drive working properly again.  The drive itself usually has to run for 5+ years before the bearings or controller motor itself actually dies.

Also, some designs run too hot.  Maxtors, for instance, ran too hot and without air blowing over the controller board, they ate themselves almost always in a year or so.  Quite a few external 3.5" drives also have this issue - like the MyBook series.  No fan and too much heat equals a dead chip sooner or later.(often the SATA interface card in this example)

The advantage of 2.5 inch drives is that they suffer none of this as they don't get hot enough in the first place for it to matter.  And they are quite a bit less power hungry and take up less space.  This can often mean a full size smaller case as well.  Oh, and you can stack a ton of them in a giant raid array if you want without exceeding the limits of your power supply or overheating the rest of your machine.
Logged
Pages: 1 2 [3]   Go Up