Pages: 1 ... 3 4 [5]   Go Down

Author Topic: Now we're talkin STORAGE  (Read 26795 times)

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
Now we're talkin STORAGE
« Reply #80 on: January 13, 2007, 10:09:57 pm »

Quote
Here's image no. 14 on the same disc. ...
[{POST_SNAPBACK}][/a]
Another excellent image, and the scan might respond to some sharpening.  

Dang, if your eye was that good 42 years ago, I don't even want to see any of your current work.

Nill
~~
[a href=\"http://www.toulme.net]www.toulme.net[/url]
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Now we're talkin STORAGE
« Reply #81 on: January 13, 2007, 10:54:02 pm »

Quote
Yes; yes; possibly, right at the moment, but at some point...

Nill
~~
www.toulme.net
[a href=\"index.php?act=findpost&pid=95619\"][{POST_SNAPBACK}][/a]


It'll have to be at some point. There's an insidious trap to the digital game. One can easily end up taking huge numbers of digital images, simply because one can with no additional expense.

But there is an expense, and that's the time taken to sort through them all and categorise them.

I'm currently at an impasse. I simply have too many images.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Now we're talkin STORAGE
« Reply #82 on: January 14, 2007, 12:20:42 am »

Hey! Look at this image, no. 13 from the same disc. I have to thank you guys for provoking me into a controversial stance, otherwise I wouldn't have bothered revisiting these images   .

[attachment=1537:attachment]
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
Now we're talkin STORAGE
« Reply #83 on: January 14, 2007, 12:37:28 am »

Quote
Dang, if your eye was that good 42 years ago, I don't even want to see any of your current work.

Nill
~~
www.toulme.net
[a href=\"index.php?act=findpost&pid=95620\"][{POST_SNAPBACK}][/a]


Thank you for your kind comments but I fear I may have gone backwards  since those days.
Logged

Craig Arnold

  • Full Member
  • ***
  • Offline Offline
  • Posts: 219
    • Craig Arnold's Photography
Now we're talkin STORAGE
« Reply #84 on: January 14, 2007, 05:57:41 am »

5 pages of thread about backups and no-one has even mentioned a tape drive?

If the data is important then backing up to tape is still the staple method. Particularly with this sort of data which is really just an archive with very little daily change a tape is ideal.

The LTO drives can manage hundreds of Gb per hour and hold up to 400Gb on a single tape. Backing up the size archive you are talking about could easily be done in a week, and maintenance could be done on a weekly schedule. Not onerous.

It's not cheap, but not much more expensive than maintaining everything on multiple hard disks.
Logged

djgarcia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 341
    • http://improbablystructuredlayers.net
Now we're talkin STORAGE
« Reply #85 on: January 14, 2007, 11:51:57 am »

Quote
And I still don't understand the appeal of a mirroring RAID system (e.g., RAID10) over, say, RAID5 plus redundant on- and off-site HDD backups.  Yes it provides more protection against some failure in the RAID itself, but not against all the other stuff that can bite you... OS burps, viruses, surges, operator headspace errors, etc.[a href=\"index.php?act=findpost&pid=95485\"][{POST_SNAPBACK}][/a]
The main advantages of RAID 10 are the better redundancy (under certain circumstances), faster write times, and the fact that you could recover the whole array from a single disk from each mirror pair. I use it in my data array but my backup array which has the last 3-5 images of each of my logical system & data disks is RAID 5. My system disk is striped (RAID 0, which as Jani mentions has no redundancy).

When working your storage strategy, you need to keep in mind everyone's storage requirements have differences. My basic average requirement per single photo image adds up to around 220MB, 20MB for the raw 17MPxl file and 200MB for the working PSD. I could store maybe 18-20 in a DVD ... This is very different from Nil's, who has much smaller files but many, many more. How the storage configuration responds to his files will be very different from my files.

To sumarize:

- It cannot be over-emphasized, RAID (especially 0!) does not replace backups. It's complementary technology that can make your life more trouble-free and save you time when used properly.

- RAID 0 is strictly for performance, the more spindles (drives) the faster, but also more fragile. You lose one drive and you've lost the whole array content, not just the one drive's. 2 drives are almost not worth it but you get to 4 and you're flying. Get smaller drives because they add up. And DO NOT use more than one partition actively or you'll lose the performance advantage. IMO best for system and scratch disks.

- RAID 1 uses mirrored pairs, so you can lose one drive of each mirror and survive. You lose half the storage capacity of the combined disks. It has noticeably better write performance than RAID 5 because it doesn't have to calculate parity, but slower reads because it is not striped. Safe but slow.

- RAID 5 uses parity for redundancy and stripes the drives. You lose only 1 drive's worth of storage capacity so you can get relatively inexpensive huge capacities. Fast reads, writes similar to a single drive. You can lose 1 drive and function. IMO good for backup of several drive images.

- RAID 6 is like RAID 5 but uses a second drive for redundant parity. You can lose 2 drives and function, but you lose 2 drives from storage capacity. IMO best for backup if you can afford it.

- RAID 10 uses two or more RAID 1 mirrored pairs and stripes them (RAID 1, + RAID 0 on top) so it's much faster than RAID 1. Needs at least 4 drives. A 4-drive RAID 10 reads as fast as a 3-drive RAID 5 because it's in pairs, but writes faster than a 4-drive RAID 5. IMO best for data if you can afford it.

- Keep in mind that in striped arrays (0, 10, 5, 6) speed is gained from separate drives being accessed. A 4-drive RAID 5 will read faster than a 4-drive RAID 10 because the RAID 5 is accessing 3 spindles and the R10 only 2 (one in each mirror).

Hope I got it down clearly and correctly ...
« Last Edit: January 14, 2007, 12:23:33 pm by djgarcia »
Logged
Over-Equipped Snapshooter - EOS 1dsII &

djgarcia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 341
    • http://improbablystructuredlayers.net
Now we're talkin STORAGE
« Reply #86 on: January 14, 2007, 12:20:47 pm »

Sorry, I mistakenly placed all parity in RAID 5 in a drive but that is what RAID 3 does. RAID 5 actually distributes parity among all the drives. I edited the original message to that effect.
Logged
Over-Equipped Snapshooter - EOS 1dsII &

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
Now we're talkin STORAGE
« Reply #87 on: January 14, 2007, 12:46:39 pm »

Has tape really gotten price competitive with HD's?  I had a SCSI tape drive a few years ago, and it was a PITA.

Nill
~~
www.toulme.net
Logged

jani

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1624
    • Øyet
Now we're talkin STORAGE
« Reply #88 on: January 14, 2007, 02:40:52 pm »

Quote
Has tape really gotten price competitive with HD's?
Yes and no.

If you want an LTO-3 or S-DLT drive for personal use, you'll probably not find it worth your while.

Now consider the possibility that you instead are a company with larger storage needs.

If you want to build a 12-disk RAID 5 with 750 GB drives, that will cost quite a bit of money.  If you want to purchase a ready-made solution, it will cost a bit more.

Tape robots may win simply on convenience and storage density.
Logged
Jan

narikin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1371
Now we're talkin STORAGE
« Reply #89 on: January 21, 2007, 07:00:02 am »

Quote
If you want to build a 12-disk RAID 5 with 750 GB drives, that will cost quite a bit of money.  If you want to purchase a ready-made solution, it will cost a bit more.
[a href=\"index.php?act=findpost&pid=95707\"][{POST_SNAPBACK}][/a]
I put a 12 disk raid 6 array for $3500 into my desktop computer.
its working right now on the machine I'm writing this from..

no biggie to put together, took about an hour to install, and I have enough storage for 3 years, with double redundancy.

one disk went down after 3 months, and it rebuilt the array in 2 hours. trouble free for last 9 months, including a relocate.

you still need deep backup for fire/flood/theft emergencies, but otherwise its a near bulletproof solution.
Logged
Pages: 1 ... 3 4 [5]   Go Up