optimum hard drive setup for photoshop

[Philmar]

I am currently reviewing my computer set up for my digital imaging processing PC.

After upgrading my current hard drives I will have a surfeit of hard drives. I'm not sure how many drives to have in my PC to optimize CS3.

I plan to have Vista 64 and apps on a 150 GB Raptor and data on a WD 1 TB Black caviar. I will also have XP and apps on an older generation 32GB Raptor. 3 drives...thus far.
I am not sure what to do from that point onwards. I will have 2 older generation WD SATA 320 GB drives that I could keep around for scratch disk/Windoze file paging. But I don't know if these older technology drives will be of any benefit. Perhaps it could be a hindrance.
Is there still wisdom in having the scratch drive on a separate dedicated drive or has this changed now that we are running systems able to utilize more than 2+ GBs of RAM?
I imagine the answer will be specific to my PC needs. So, FWIW, I'll be running 4 GBs RAM on Vista 64. I rarely do batch operations. I am a photo enthusiast/hobbyist who shoots in RAW and I usually process each RAW file individually making adjustments in ACR and in CS3. After I've processed about 20 - 30 30D RAW files, I'll run these in a batch PS action that converts them to 8 bits, downsizes and sharpens. This is the only time I notice my system slowing down and anything going to the scratch drive. So I am not a pro that has to batch process 8 GBs of critical files for clients.
So considering my needs, is there any point of adding another drive or 2 to serve as a PS scratch drive or windoze file paging disk?
If there is still value in a dedicated drive, is one of the old WD SATA drives up to the task? Is the older generation 32GB Raptor going to be up to the task if it is already half full due to XP/apps I've loaded on to it. Will 64 bit Vista and 4 GBs of RAM be of such an improvement that I can do away with XP (and its associated apps) on this Raptor drive and use it for scratch drive/paging?
Or will I still get some benefit from a new smaller black caviar as scratch drive? The cost of such a drive is of no importance.
I don’t overclock or have a hot powerful video card. My current case, mobo, cooling system and PSU will support a 5 drive set up.
Thanks for your help.

[Jack Flesher]

My .02 with the disclaimer I am running Mac/Leopard, not Vista:

Yes, there is still benefit to having scratch on a separate dedicated drive.  Moreover, there is clear evidence that having that scratch drive be as fast as possible is beneficial, the faster the better.  In my case, I have the OS and programs on a dedicated array, then a separate 4-drive striped (RAID-0) array with a thin partition dedicated to CS scratch.  My own benchmarks were as follows running my own standardized action on a 50 MB file containing sizing, various filters and sharpening that forces CS to scratch regardless of amount of RAM installed. With scratch on a single drive it ran in 1:04; scratch on a 2-drive stripe, 41 seconds; scratch on the 4-drive stripe, 28 seconds.  

Cheers,

[Philmar]

Yes, there is still benefit to having scratch on a separate dedicated drive.

Thanks

Moreover, there is clear evidence that having that scratch drive be as fast as possible is beneficial, the faster the better.

Yes, I read this in the Adobe [a href=\'index.php?act=findpost&pid=0\']techNotes[/a]. It also mentions a RAID set up as being the fastest. I guess I need specific information about my set up as my level of computer sophistication precludes me from any kind of RAID set up. As a photo enthusiast/hobbyist I don't think researching/investigating RAID configurations is critical for me. I want to keep it as simple as I can.

Now, I've tried reading Tom's Hardware disk performance charts but there are far too many measures to make sense of what I should be looking at. So, in the opinion of those in the know:

Are one of my old generation WD 320 GB SATA drives up to the task as a dedicated PS scratch disk?

What about an old generation 32 GB Raptor with about 12 GBs of free space? Is this  better choice? Or is Vista and it's ability to use all of my 4 GBs of RAM going to be so much of an improvement that I'll never want to play in XP again thereby giving me a good reason to wave goodbye to XP, abandon plans for a dual boot system and dedicate the whole 32 GB raptor to scratch disk?

Or are recent developments in high density drive technology so profound that a newer WD black caviar would be a better choice? And if so, which is the smallest size of this drive that I should use?

Thanks for any help - it is truly appreciated.

[Jack Flesher]

Now, I've tried reading Tom's Hardware disk performance charts but there are far too many measures to make sense of what I should be looking at. So, in the opinion of those in the know:

Are one of my old generation WD 320 GB SATA drives up to the task as a dedicated PS scratch disk?

What about an old generation 32 GB Raptor with about 12 GBs of free space? Is this  better choice? Or is Vista and it's ability to use all of my 4 GBs of RAM going to be so much of an improvement that I'll never want to play in XP again thereby giving me a good reason to wave goodbye to XP, abandon plans for a dual boot system and dedicate the whole 32 GB raptor to scratch disk?

Or are recent developments in high density drive technology so profound that a newer WD black caviar would be a better choice? And if so, which is the smallest size of this drive that I should use?

1) Old Raptors are SATA1 connection, the newer drives are all SATA2 -- unless your 32 is even an older SCSI drive? -- and SATA2 is 2x as fast as SATA1 in theoretical throughput.

1a) If it is an older SCSI drive, my tests showed SATA1 was nearly as fast in most imaging applications and SATA2 faster -- not sure why other than dirve technology has improved a lot over the past few years (primarily due to platter density).

2) 12 GB of free space is not enough for scratch IMO -- you typically need at least 30 for it to be effective with large files, and I regularly see single scratch files over 30G residing on my scratch disk.  You can determine this by viewing "scratch size" in the info panel of CS and monitoring it for your uses.

3) By "old generation" WD 320, I am not sure how old that might be -- again if SATA1, drives have come a long way; OTOH, if it is a SATA2 drive then it's going to be faster.  

3a) HOWEVER:  Larger drives have denser platters and by default thus offer a lot faster I/O than smaller drives with lower density platters spinning at the same speeds; why the newest dual-platter 640's and three-platter 1TB's and 1.5TB's are so fast in the Tom's or BareFeats tests.  The soon to be released WD 2TB Blue, Black and ES 7200 drive (not the Green 5400-7200) should set a new record for drive I/O.

Best,

[Philmar]

WOW!!! Jack you are fantastic. Thank you soooo much.

Am trying to answer the rest of the question myself (though others are free to give their take or opinion).
Can anyone tell me which of Tom's hardware performance benchmarks is most relevant for a HD used solely as a PS scratch disk?

Average Read Transfer Performance

Average WriteTransfer Performance
 
Database I/O Benchmark Pattern

File Writing Performance

Fileserver I/O Benchmark Pattern

Interface Performance

Maximum Read Transfer Rate

Maximum Write Transfer Performance

Minimum Read Transfer Performance

Minimum WriteTransfer Performance

Random Access Time

Workstation I/O Benchmark Pattern

[Jack Flesher]

Don't mean to cobble up all the responses, but here is a link I think you should look at -- and I personally like to weight toward the random read and random write benchmarks since that is precisely what your OS and program drive is doing most of the time you tag it.  Your image drive by contrast is doing mostly sustained large file reads and writes, so I'd use that benchmark for a storage drive:

http://www.barefeats.com/hard112.html

Okay, I'll shut up now and let somebody else talk ,

[Philmar]

In my case, I have the OS and programs on a dedicated array, then a separate 4-drive striped (RAID-0) array with a thin partition dedicated to CS scratch.

I still go back to look at your set up and can get my head wrapped around what you've written. So how many drives is this in total? How many are in the OS/program dedicated array? Then there's a RAID 0 array of 4 drives in which a mall partition was created for CS Scratch. So where is your data? In the 4 drive array? Doesn't RAID 0 increase the risk of a hD failure losing your data by magnitude of 4? Do you then have some system that backs up your data daily to external drives?

[Jack Flesher]

I still go back to look at your set up and can get my head wrapped around what you've written. So how many drives is this in total? How many are in the OS/program dedicated array? Then there's a RAID 0 array of 4 drives in which a mall partition was created for CS Scratch. So where is your data? In the 4 drive array? Doesn't RAID 0 increase the risk of a hD failure losing your data by magnitude of 4? Do you then have some system that backs up your data daily to external drives?

Sorry, let me explain more thoroughly.

1) I have 6 total drives in my Mac Pro.  There are 4 main drive slide out bays and the MB has two additional free SATA2 ports on it.  So without any added SATA card you can attach another 2 SATA2 drives for 6 total.  This device replaces the stock optical bay and allows the two added drives to sit in the lower optical bay while maintaining a single optical drive up top.  Note you cannot use this device if you want/need two *internal* optical drives: http://www.maxupgrades.com/istore/index.cf...;Product_ID=158

2) So, now that is cleared up, I have the 2 drives in the lower optical bay in RAID-0 for a 2-drive stripe. These are WD 640G Caviar black drives, very fast dual-platter high density drives with 32MB buffers each, but very affordable for that performance at roughly $75 each.  In RAID-0 they act as a very fast 1.25TB OS drive with a huge space reserve for temporary storage of files on the desktop.  

3) Now that leaves the 4 drives in the regular drive bays.  Here I have 4 WD 640G Caviar Blue drives -- as fast as the black version, but only 16MB buffers and $10 per drive cheaper, so this gives me 2.5 TB of relatively cheap, but screaming fast drive storage.  Here I partition off a thin 30G stripe on the fastest outer rim of each drive in the array and use this as dedicated screaming fast 4x30G or 120G RAID-0 scratch drive.  

3a) That leaves 600G on each remaining drive. Here I partition off 450G on each drive, so 4x450G for 1.8TB RAID-0 of screaming fast data storage; where I store my working image files and how I get the super read/write performance when working on my images.

3b) The last 150G, or slowest part of each drive is left non-RAID as simple, single drive data storage. Here I use 3 of those for spare bootable copies of my OS. My OS gets cloned over and updated to these 3 partitions a few different times each week, automatically scheduled and run using Carbon Copy Cloner. The last partition is used for Leopard's Time Machine for a different type of back-up and 150G is arguably smallish for that purpose, but the fact is given my particular back-up routine, I don't really need TM at all. Anyway, having these redundant bootable OS partitions allows me to re-boot my machine and re-build my OS instantly in case of a failure on my main OS RAID-0 array per #1.

4) Important: Note that RAID-0 is done for performance only and has reduced reliability over any single drive installation.  As such, it is imperative you maintain bootable copies of your OS and redundant copies of all your images as a mandatory practice if you choose to go this route for performance!

5) Note that it takes about 30 minutes total to clone over a full image of my OS including all loaded software, programs and files from any of the slow single partitions, so I can totally rebuild my system in less than an hour AND can keep working the entire time it is rebuilding(!)  

6) All of my images are then backed up onsite to a FW800 DROBO, basically an automated 4-drive RAID-5 array for mass storage. This is not a particularly fast device, but it is reliable and has single drive failure protection with the RAID-5 configuration.  If the image array goes down, it would take several hours, or basically overnight, to rebuild it reading from a DROBO, but the images (and other data) do remain accessible from the DROBO while that is going on.

7) In case the DROBO fails at the same time as my main box (like in the event of a fire at my studio), I have all my image files further backed up to single bare drives and stored offsite for maximum security.

Makes better sense now I hope?

[emcphoto]

3) Now that leaves the 4 drives in the regular drive bays.  Here I have 4 WD 640G Caviar Blue drives -- as fast as the black version, but only 16MB buffers and $10 per drive cheaper, so this gives me 2.5 TB of relatively cheap, but screaming fast drive storage.  Here I partition off a thin 30G stripe on the fastest outer rim of each drive in the array and use this as dedicated screaming fast 4x30G or 120G RAID-0 scratch drive.

Hi Jack,

Forgive me if this is remedial but when partitioning Raid 0 how do you choose the outer stripe of each drive?  I see this mentioned everywhere but not how to do it.

Thank you.

[Jack Flesher]

Hi Jack,

Forgive me if this is remedial but when partitioning Raid 0 how do you choose the outer stripe of each drive?  I see this mentioned everywhere but not how to do it.

Thank you.

Yes, it is not well documented for Leopard.  The trick there is to partition your drives FIRST, then drag the partitions into the various RAID sets you want to create.  Other soft raids or hardware raids may allow you to partition after the raid array is formed.

[fike]

Someone above describes Jack as a fanatic, and that is very true.  Jack is building an amazingly powerful and high-end machine.  For working with 30D RAW files, this setup is way overkill.  RAID striping is great.  Faster IS better, but this recommendation is not for working with moderate-sized raw files that a 30D makes.  

One note on SATA1 versus SATA2.  The throughput difference is very incremental.  That 35 GB Raptor you have lying around is a fantastic scratch-disk for 8MP RAW files.  You could spend the many hundreds or a thousand $$ putting together RAID drive arrays to optimize photoshop, but my approach is typically a bit more value-focused than Jack's bleeding-edge.


Remember to:
- Separate PS scratch drive from pagefile drive and OS/Apps drive
- Separate OS Pagefile drive from Scratch drive and OS/Apps drive.


So, you probably need the following:
- OS/Apps drive (c:)
- Data Drive
- Scratch disk (that spare raptor you got)
- OS Pagefile drive.

Don't get me wrong, Jack's recommendations are awesome and fantastic, but they will cost lots of time and money to implement and maintain.  Consider something a bit more modest to go with your modest and effective 30D.

[fike]

WAIT!!  Jack was called "Fantastic," not a "Fanatic" like I said.  Oops!  Sorry.  That was a serious Freudian slip.  

Hahahaha..hehehe...sorry Jack.

[Jack Flesher]

Don't get me wrong, Jack's recommendations are awesome and fantastic, but they will cost lots of time and money to implement and maintain.

Well let's see...

To Cost: My WD drives cost $70 each and one of them will beat an older SATA1 Raptor in benchmark; two of them in RAID-0 for scratch will kick an older raptor's A$$.  Personally, I do not see $140 as "lots of money" when talking computer performance, but respect that others may.

To Time:  It took me all of about 5 minutes to set up the RAID-0 as described.

To Maintenance:  Here I actually expect my 4-drive RAID-0 array to go down at any point in time.  And when it does, it will take me all of 5 minutes to rebuild it.  (And if a drive or two failed, then obviously some money to purchase the replacements, but then I'd replace any single drive that failed too...)  Reloading the array with all of my working files may take some time, but that can go on by itself overnight when I am at home sipping my evening martini.  Moreover, I would suspect my newer WD 640's will outlast any long in the tooth Raptor as far as reliable remaining lifespan  

FWIW, I also cycle my drives through my systems about every 18 months.  Because of moving parts, they are an item that likely will fail over time -- especially if I try to save money and run them for as long as possible -- so I systematically swap them out and use the old ones for the redundant offsite storage media.  So I see using "old" drives as false economy.  Again, I respect other's views may vary.
 
Cheers,

[fike]

Well let's see...

To Cost: My WD drives cost $70 each and one of them will beat an older SATA1 Raptor in benchmark; two of them in RAID-0 for scratch will kick an older raptor's A$$.  Personally, I do not see $140 as "lots of money" when talking computer performance, but respect that others may.

To Time:  It took me all of about 5 minutes to set up the RAID-0 as described.

To Maintenance:  Here I actually expect my 4-drive RAID-0 array to go down at any point in time.  And when it does, it will take me all of 5 minutes to rebuild it.  (And if a drive or two failed, then obviously some money to purchase the replacements, but then I'd replace any single drive that failed too...)  Reloading the array with all of my working files may take some time, but that can go on by itself overnight when I am at home sipping my evening martini.  Moreover, I would suspect my newer WD 640's will outlast any long in the tooth Raptor as far as reliable remaining lifespan  

FWIW, I also cycle my drives through my systems about every 18 months.  Because of moving parts, they are an item that likely will fail over time -- especially if I try to save money and run them for as long as possible -- so I systematically swap them out and use the old ones for the redundant offsite storage media.  So I see using "old" drives as false economy.  Again, I respect other's views may vary.
 
Cheers,

okay, I was basing the cost on six hard drives and at least one controller-card.  As for complexity and problems, for the new and uninitiated, RAID can be problematic and have a learning curve.  Some folks would rather their equipment just work without much effort or management.  RAID is not without management time.

[Jack Flesher]

okay, I was basing the cost on six hard drives and at least one controller-card.  As for complexity and problems, for the new and uninitiated, RAID can be problematic and have a learning curve.  Some folks would rather their equipment just work without much effort or management.  RAID is not without management time.

Just in case you were unaware:

1)  A MacPro does not need a controller card to run 6 SATA drives as that functionality is built into the motherboard, along with a 2@ IDE harness.  Note in the newest MacPros that the DVD drive or drives (it can take 2) are SATA interface and do take up the slots that were otherwise available on earlier boxes that used IDE DVD drives.  

2) Leopard itself contains a disk utility that allows you to create RAID 1 or 0 sets quickly and easily, no controller or other software RAID required.  Though SoftRaid is a very nifty piece of utility software for folks that want a nicer GUI and the ability to also do RAID 5 or 10 (0+1) in addition to 0 and 1.

Cheers,

[fike]

Just in case you were unaware:

1)  A MacPro does not need a controller card to run 6 SATA drives as that functionality is built into the motherboard, along with a 2@ IDE harness.  Note in the newest MacPros that the DVD drive or drives (it can take 2) are SATA interface and do take up the slots that were otherwise available on earlier boxes that used IDE DVD drives.  

2) Leopard itself contains a disk utility that allows you to create RAID 1 or 0 sets quickly and easily, no controller or other software RAID required.  Though SoftRaid is a very nifty piece of utility software for folks that want a nicer GUI and the ability to also do RAID 5 or 10 (0+1) in addition to 0 and 1.

Cheers,

I didn't know that.  Is the RAID software RAID from OS or hardware RAID?  

Anyhow, I thought the OP was a windows user because of his mention of Vista.  Some Mboards support two-hard drive RAID, but not too many have much more.

[Jack Flesher]

I didn't know that.  Is the RAID software RAID from OS or hardware RAID?

It is software, built into the OS, but an optional card is available for hardware RAID too.

Anyhow, I thought the OP was a windows user because of his mention of Vista.  Some Mboards support two-hard drive RAID, but not too many have much more.

He is on Vista. Initially I only addressed his question about drives, then only got specific about my Mac when he asked for a more complete explanation of my personal set up.

Cheers,

[Dustbak]

I have just built a 3drive stripe set on my Mac with 2 partitions (basically as Jack mentioned) which is extremely easy to do. I have a 300Gb partition for PS scratch and a 2.6 Tb Data drive. I am now trying to figure out where the best location is for my programs (like PS), I have them on the 640Gb system drive at this moment. It is such a shame that PS is mainly only able to use 1 core, all that power and not being able to use it

I don't have the extra partitions to backup the OS. I don't care whether that blows up eventually or not. It is pretty easily rebuilt even if it takes me a day and some time to get it to where I exactly want it. It so rarely happens that I don't feel the need to do that (years ago was the last time, knock on wood. It might be almost time again...)

BTW Windows has the same kind of possibility in the drive manager to create RAID1 or a stripe set (RAID0). XP at least, never did like Vista but Vista probably has the same option

A lot of current windows MoBo's have the ability to do a hardware Raid (even 5 & 10). I had a Raid5 on my last Windows machine with that.

[fike]

I'm not sure that software RAID from windows offers a very big performance improvement while it uses processor cycles.  aftermarket mboards frequently have some sort of RAID, but PCs from Dell and the like rarely have RAID.

[Dustbak]

I have always built my own PC's (since mid '80's) and never bought something like a Dell. The last couple of years most MoBo's do have Raid onboard, at least the ones I have been considering. The likes of Asus are not particularly after market in my book. However the stuff they often put into Dells are IMO.

I have never compared windows software raid with the mac software raid. Windows stripe set does give quite a disk performance boost no idea how much more cpu cycles Windows takes over Mac. In most cases I used software raid on Windows200x servers. Mirrors (Raid 1) only gives better read performance but worse write performance.

I am mostly over to Mac since about 3 years.