Luminous Landscape Forum
Equipment & Techniques => Computers & Peripherals => Topic started by: mdijb on September 10, 2009, 07:21:37 pm
-
I am about to purchase a new MacPro and want to use a separate internal scratch Disk.
1- is there much difference in performance between a 7200 and 10,000 rpm drive--the price difference is significant?
2-I have seen people using between 10 and 300 GB for this purpose. It seems that the large dics are not needed. What is the best size for just dedicating the drive to Photoshop use?
MDIJB
-
If I am not mistaken, 10,000 RPM speeds are only available in huge capacity hard drives. I can not imagine that you would need such a large drive capacity for a scratch drive. The optimal size of a scratch drive depends on the size of the files you routinely edit in Photoshop. I have heard from many sources that the scratch drive should be at least four or five times the size of your largest files. So even a relatively small 7200 RPM hard drive will prolly suit your purpose.
If you are really concerned about speed, you should be putting your money into more RAM. More available memory minimizes scratch drive use.
The best source of information I've seen about optimizing Photoshop on Macs is provided by Lloyd Chambers. You should check out his website, www.macperformanceguide.com (http://macperformanceguide.com), as well as his blog.
Good luck.
John
-
Yes, RAM first. On the other hand, a 10,000 RPM drive is much faster than a 7,200 drive. The largest performance improvement I have seen over the past few years of upgrades was a 10,000 RPM Western Digital Raptor. They are far superior to the 7,200 drives. They can be had as small as 150GB I think. If you really want to go crazy get a decent SSD for your scratch disk.
-
You can buy 4 640G 7200's for the price of a single 300Gig 10K drive -- and if you put those 4 7200 drives in RAID-0 it will smoke the single 10K disk on I/O... I partition off the outer 160G edge of a 4-drive RAID-0 and dedicate that for scratch. I use the rest -- over 2TB -- for working image storage and it's also very fast on reads and writes of large image files. However, since it is RAID-0 it is failure prone and needs to be backed up...
-
You can buy 4 640G 7200's for the price of a single 300Gig 10K drive -- and if you put those 4 7200 drives in RAID-0 it will smoke the single 10K disk on I/O... I partition off the outer 160G edge of a 4-drive RAID-0 and dedicate that for scratch. I use the rest -- over 2TB -- for working image storage and it's also very fast on reads and writes of large image files. However, since it is RAID-0 it is failure prone and needs to be backed up...
That RAID-0 will absolutely smoke a single 10,000 RPM drive, but it isn't simple and it is 4x prone to failure. Also, your hardware (mboard and enclosure) need to support more than 5 drives (RAID plus your boot). That isn't always available.
KISS--Keep It Simple Stoopid.
-
That RAID-0 will absolutely smoke a single 10,000 RPM drive, but it isn't simple and it is 4x prone to failure. Also, your hardware (mboard and enclosure) need to support more than 5 drives (RAID plus your boot). That isn't always available.
KISS--Keep It Simple Stoopid.
FWIW, with Mac OSX, RAID-0 is VERY simple to do with the built-in OS drive management software -- takes maybe a minute. And FWIW2, the Mac MB supports up to 6 SATA2 drives WITHOUT adding a card! And who cares if a scratch array fails? Just rebuild it... Stoopid
-
FWIW, with Mac OSX, RAID-0 is VERY simple to do with the built-in OS drive management software -- takes maybe a minute. And FWIW2, the Mac MB supports up to 6 SATA2 drives WITHOUT adding a card! And who cares if a scratch array fails? Just rebuild it... Stoopid
Okay, just get an OCZ Vertex SSD. Simple and even faster than RAID 0.
-
Okay, just get an OCZ Vertex SSD. Simple and even faster than RAID 0.
RAID-0 4 of those and you're really talking I/O
-
Add lots of ram first as others have stated, however I diverge from there, since 1998 I only use SCSI HD for the heavy stuff and the reason is simple, they are simply a lot more reliable than everything else. RAID 0 is fine but so is RAID 10. I know what so many people will say, SCSI is so much more expensive, but guess what (I KNOW I SHOULDN'T SAY IT) since 1998 I have NOT had 1 SCSI HD failure, but have had 4 IDE failures, 2 in the past year and I use a lot more SCSI drives than IDE/Serial etc. Apple IIRC used to use a lot of SCSI?
-
Ram is 500-2000X faster than a hard drive. How you do it, be it 64 bit OS, a ram disk, or whatever is up to you...
-
FWIW, Lloyd and I tested RAM discs in Leopard for scratch about a year ago and they were not as fast as our 4-drive RAID-0 on our CS benchmark test. (I have 24G of RAM in my Mac Pro.) That may have changed with Snow Leopard, so definitely worth a look. The other side of the issue is how well -- or in this case how poorly -- CS manages scratch overhead, especially for newer machines.
-
It is a pity that there are no 10-16Gb SSD drives. Otherwise use 4 of them in raid 0 and you have a very nice scratch disk. Combine that with 24Gb of RAM and you have a very fast solution.
My main problem is still image storage. Right now I'm using 4 640GB drives in RAID 0 (gives me around 300-400 Mbs read and write speed), which is ok but not as fast as I would wish. My dream would be 6-8 300-500 SSD Drives (that would give me a write and read speed at around 1,5 Gbs), but that dream will have to wait a bit.
One site note, the 10,000 Raptors are not really that fast anymore. Current 7,200 Harddrives are as fast when it comes to writing and reading speed.
-
It is a pity that there are no 10-16Gb SSD drives. Otherwise use 4 of them in raid 0 and you have a very nice scratch disk. Combine that with 24Gb of RAM and you have a very fast solution.
There is a very good OCZ Vertx SSD drive that is only 30GB. MicroCenter in the US has them for $100 after rebate.
-
Personally, I'm waiting to see what Adobe does with CS-5 Mac and then decide what to do about scratch. If they code it to actually utilize installed, available RAM, then scratch becomes almost unnecessary except perhaps for very large, layered files. And with some wishful thinking here, utilizing a fast OS array may be adequate. Worst case would be the same solution I am currently using -- the thin outer rim partition of a 4-drive stripe for scratch and the remaining for working image storage. Regardless, I may move my OS from its current 2-drive RAID-0 array to a single really fast SSD, or if they get cheap enough, a pair of smaller fast ones -- 64G x 2 is plenty -- in RAID-0.
If money were no object, I'd have OS on a 2x64G Intel X25E SSD R-0, scratch on a dedicated 4x32G Intel X25E SSD R-0, and image storage on a separate 4x256G Crucial SSD R-0 array. Of course that is like $5,000 in SSD drives, which is not out of the question for some, but still seems a bit extravagant for me. Maybe if I get a good bonus this year LOLOL!
-
Personally, I'm waiting to see what Adobe does with CS-5 Mac and then decide what to do about scratch. If they code it to actually utilize installed, available RAM, then scratch becomes almost unnecessary except perhaps for very large, layered files. And with some wishful thinking here, utilizing a fast OS array may be adequate. Worst case would be the same solution I am currently using -- the thin outer rim partition of a 4-drive stripe for scratch and the remaining for working image storage. Regardless, I may move my OS from its current 2-drive RAID-0 array to a single really fast SSD, or if they get cheap enough, a pair of smaller fast ones -- 64G x 2 is plenty -- in RAID-0.
If money were no object, I'd have OS on a 2x64G Intel X25E SSD R-0, scratch on a dedicated 4x32G Intel X25E SSD R-0, and image storage on a separate 4x256G Crucial SSD R-0 array. Of course that is like $5,000 in SSD drives, which is not out of the question for some, but still seems a bit extravagant for me. Maybe if I get a good bonus this year LOLOL!
If you get that bonus, you can slip me a few X25s too.
-
FWIW, Lloyd and I tested RAM discs in Leopard for scratch about a year ago and they were not as fast as our 4-drive RAID-0 on our CS benchmark test.
I'd like to point out that a SSD isn't *exactly* the same as a DDR-based ram disk. But DDR ram disks are *really* expensive. OTOH, if you have 16GB or more, you can do nicely with a good 64 bit OS in a pinch. You'll also have to move the swap file if you're using Windows, as well as the temp and swap directories for Photoshop/etc to the thing or you'll not see most of the benefit.
Yes, they cleaned up the 64 bit code in Snow. Very slick now, as it should be.
-
If you get that bonus, you can slip me a few X25s too.
Heck, I am such a gear slut I may fold and do it regardless of bonus -- but either way, you are on your own for obtaining your SSD's LOLOLOLOLOLOL!!!
-
I have just built a new PC with 2 Velociraptor HD in raid 0 for the OS and 4 1TB WD Caviar Black which I was planning to set up in Raid 5 for the RAW files. I chose raid 5 and not raid 0 for the added advantage that you can recover data in case a single drive failure. I always keep a back up of my RAW files, but not always back up my xmp sidecar or psd/tiff files from photoshop.
I have partitioned the 4 drive array in 3 separate volumes. The first 1gb volume I will dedicate for Photoshop scratch, and the remaining divided in 2 equal volumes, one for Raw files and the other half for edited files, such as psd or tiff files.
I was wondering if I would gain any benefit if I make a Raid 0 volume of 1gb out of the 4 drive array for the scratch disk instead and then set up another raid 5 volume from the remaining space, which I later partition in 2 drives in windows 7.
Any ideas please.
Thanks
Etienne
-
There's no such thing as an SSD with TRIM with RAID at the moment, and haven't even heard rumors of something like that being added in the foreseeable future. OCZ Vertex is the only SSD I'm aware that has TRIM in their latest firmware, but TRIM doesn't work with RAID. The drive itself would work in RAID, though.
In English: while SSDs should work perfectly in RAID, putting SSDs in a RAID will result in degrading I/O performance over time until someone comes up with firmware supporting the TRIM feature or at least garbage collection with RAID.
The question is whether TRIM makes a noticieable degradation in I/O, and how long it takes for it to take place. From my experience comparing my old Raptor with the recently purchased OCZ Vertex I'm quite sure that even with massively degraded performance an SSD will run laps around any HDD setup, RAID or not.
FWIW, SSDs are the best and cheapest way to make a massive improvement in disk-use intensive applications.
-
I am about to purchase a new MacPro and want to use a separate internal scratch Disk.
1- is there much difference in performance between a 7200 and 10,000 rpm drive--the price difference is significant?
2-I have seen people using between 10 and 300 GB for this purpose. It seems that the large dics are not needed. What is the best size for just dedicating the drive to Photoshop use?
MDIJB
This is one use where SSD drives really shine. For example, you can get an Intel 80GB SSD drive for about $240, and that is plenty of space. But, I think it's better to use your $$ for more RAM so Photoshop doesn't need to go to the scratch disk.
-
I have just built a new PC with 2 Velociraptor HD in raid 0 for the OS and 4 1TB WD Caviar Black which I was planning to set up in Raid 5 for the RAW files. I chose raid 5 and not raid 0 for the added advantage that you can recover data in case a single drive failure. I always keep a back up of my RAW files, but not always back up my xmp sidecar or psd/tiff files from photoshop.
I have partitioned the 4 drive array in 3 separate volumes. The first 1gb volume I will dedicate for Photoshop scratch, and the remaining divided in 2 equal volumes, one for Raw files and the other half for edited files, such as psd or tiff files.
I was wondering if I would gain any benefit if I make a Raid 0 volume of 1gb out of the 4 drive array for the scratch disk instead and then set up another raid 5 volume from the remaining space, which I later partition in 2 drives in windows 7.
Any ideas please.
Thanks
Etienne
1 GB is not enough for scratch. Think more in terms of 64 or 128 Gigs for scratch -- with 4x1TB drives, you have more than enough space to partition off 16G or 32G off the top of each drive for scratch. And yes, I'd RAID-0 that first partition since there is no concern if the R-0 array fails for scratch...
-
I'm finishing up a new build for Lightroom, Photoshop and Premier. After doing much research it seems the best (cost/performance) solution for a scratch disk is 2x7,200 rpm HDD's in Raid 0. I just purchsed 2 WD Black Caviar 160 GB drives at $43 each. Partioned off 30GB for scratch and will use the rest for photos being processed and then move them off to a single hard disk for archive. If you lose the Raid, no loss. If you insist on going with SSD's, an expert said the SLC's would be better suited to the task rather than MLC's. He explained why but it was beyond my pay grade.
My final drive configuration will be an Intel X-25 v2 80GB for OS & apps, 2-WD 160GB in Raid 0 for scratch and working files and a WD Veloceraptor 300GB for current projects. Total cost was $580.
-
Put your working files and scratch partitions on the same spindle(s) is a mistake.
-
Put your working files and scratch partitions on the same spindle(s) is a mistake.
True, but really only an issue if you're reading/writing your image file array and tagging scratch array all at the same time .
-
True, but really only an issue if you're reading/writing your image file array and tagging scratch array all at the same time .
Like when you save an image you've been working on or load in other images to do a composite or perhaps a stitch or maybe an HDR, or when your OS or AV decides to hit one while you use the other?
It's always a bad idea :-)
-
Like when you save an image you've been working on or load in other images to do a composite or perhaps a stitch or maybe an HDR, or when your OS or AV decides to hit one while you use the other?
It's always a bad idea :-)
Uh, once again, you have to be saving or reading one file from your image array WHILE you're tagging scratch because you're in the middle of some operation on some other (probably big) file in CS --- most folks don't do that with regularity, so it isn't ALWAYS a bad idea as you suggest... Moreover, a 4-drive RAID-0 array is pretty dang fast to begin with, so even if you do occasionally run both operations concurrently, it is still a lot fast than any single drive scratch solution.
Cheers,
-
my desktop setup:
10,000 rpm boot drive 150gb - noticeable improvement in program loading and execution
8gm of RAM - probably not enough (!!) but many of my multi-layered files exceed 1GB.
scratch disk is two 7200 drives joined in a striped raid set - they are used for nothing else, got inexpensive ones
most external drives are handicapped for scratch use at 5200 rpm, or their interface.
one exception is ESATA, which is about as fast (I think) as an internal drive running at equiv RPM
David
-
Uh, once again, you have to be saving or reading one file from your image array WHILE you're tagging scratch because you're in the middle of some operation on some other (probably big) file in CS --- most folks don't do that with regularity, so it isn't ALWAYS a bad idea as you suggest... Moreover, a 4-drive RAID-0 array is pretty dang fast to begin with, so even if you do occasionally run both operations concurrently, it is still a lot fast than any single drive scratch solution.
Cheers,
If you're opening numerous large files to stitch or merge or HDR etc then you can easily be reading from the image portion of the array whilst accessing the scratch portion. You only need some background operation (OS or AV for example) to hit the image array whilst you're making use of the scratch and the problem exhibits. If you have a long operation in the background and choose to move to some other app, you can end up causing conflict, too.
A 4 drive array is fast, but it loses buckets of speed when it has to move the head to another part of the disk to read or write data, it causes your channel bandwidth to be tied up doing more than one operation and if you're using mainboard RAID as most people are (and not a dedicated controller card) you're using more resources to handle two RAID operations at once (certainly less of an issue with current processors and memory access, but still a factor).
Hence I would say it's always a bad idea because there are too many occassions on which you can have a conflict and reduce performance, let alone the increase risk of disk failure due to increased usage (which might be small, but still real). For the price of disks these days and with most mainboards providing capacity for multiple arrays (or spend a reasonable amount and get a real controller) it's a much better idea to keep the images and the scratch apart.
-
If you're opening numerous large files to stitch or merge or HDR etc then you can easily be reading from the image portion of the array whilst accessing the scratch portion. You only need some background operation (OS or AV for example) to hit the image array whilst you're making use of the scratch and the problem exhibits. If you have a long operation in the background and choose to move to some other app, you can end up causing conflict, too.
A 4 drive array is fast, but it loses buckets of speed when it has to move the head to another part of the disk to read or write data, it causes your channel bandwidth to be tied up doing more than one operation and if you're using mainboard RAID as most people are (and not a dedicated controller card) you're using more resources to handle two RAID operations at once (certainly less of an issue with current processors and memory access, but still a factor).
Hence I would say it's always a bad idea because there are too many occassions on which you can have a conflict and reduce performance, let alone the increase risk of disk failure due to increased usage (which might be small, but still real). For the price of disks these days and with most mainboards providing capacity for multiple arrays (or spend a reasonable amount and get a real controller) it's a much better idea to keep the images and the scratch apart.
Have you done testing on the impact? Throughput and seek times of any decent 4 disk RAID 0 array are so good that I can't imagine how such conflicts would cause noticeable degaradation of performance, even when using HDDs.
And risk of disk failure due to increased usage is academic and infinitesimal.
-
Have you done testing on the impact? Throughput and seek times of any decent 4 disk RAID 0 array are so good that I can't imagine how such conflicts would cause noticeable degaradation of performance, even when using HDDs.
And risk of disk failure due to increased usage is academic and infinitesimal.
No, why would I deliberately slow my system down? As soon as I need to access two streams of data down the same pipe I'll lose performance. Seek times are good because the load is distributed over 4 devices - if you make it do twice as many seeks, you'll clearly reduce performance. There's just no need to setup in the manner.
Regarding disk failure, with a 4 disk RAID 0 you have quite a reasonable risk of losing a drive (particularly if you don't have a UPS to protect you) and if you have images on it, then you're heading to your backups. Increased usage will lead to an increased failure rate.
The bottom line is, there's really no need to do it this way. Use two arrays.
-
I think there is a lot of misreading of existing information out there.
RAID became a common place practice on Bus Mastered controller technology. This includes SCSI, SAS and FC. Having multiple disks and multiple files systems on a given bus was not an issue because the controller itself would dictate traffic on the channel.
SATA on the other hand supports (and is optimized) for one disk per channel only. We're seeing amazing performance because the Operating System's file subsystem, device driver, and ondisk cache have become tightly integrated. Splitting a disk into partitions, means we now have 2 filesystems to contend with. Any switch between the 2 will involve cache invalidation and rebuild - a very expensive operation. Whether this is important to you is of course your own decision, but is definately not considered a best practice.
I'm all for RAID 0 on a pair of short stroked high RPM drives for swap only. I wouldn't even consider using the slack space for any reason.
-
I'm all for RAID 0 on a pair of short stroked high RPM drives for swap only. I wouldn't even consider using the slack space for any reason.
While I agree in general with regards to most systems, there is a school of thought that actually thinks that it's the actual disk access and the speed of the read/write and access that changes based on the function being used...
See: Moving Your Users Directory - OSX Leopard (http://macgurus.com/productpages/guides/MoveUsers_part1.php)
This is taking advantage of the fact that most OS calls tend to be small transactions rather than large transactions often associated with scratch files for Photoshop (or the massive disk I/O required for opening an image).
We'll find out in a few weeks...I'm putting in a new MacPro with the Apple raid card and 4 SAS 15K drives–a pair partitioned according to the above partitioning scheme and the other 2 drives stripped for Photoshop scratch space. This puts the base OS, the Applications folder and the PS scratch disk all on separate physical volumes...
There's another aspect of PS scratch that you should keep in mind...the base level scratch in Photoshop is a function of the physical installed ram allocated to Photoshop AND the size of your images...Photoshop will, by default, create a default scratch partition based on the physical ram allocated to Photoshop–and that's gonna change with 64 bit computing...in the past we've generally gotten by with the 5-10X scratch to ram...but if you have a 64 bit OS (such as Win 7, 64 bit and Snow Leopard and hopefully the next version os Photoshop), your scratch file sizes may grow...possibly a lot if you have a ton on ram on the motherboard. My MacPro will have 32 gigs so the amount of scratch I'm allocating is a pair of 600GB SAS drives-so, over one TB of scratch...
And normally I would say hard drives are cheap–and the are, but not SAS drives...so the fact I'm throwing a TB of super fast scratch disk at Photoshop should tell ya something...
I'll report back when my system is in...
-
Well I think it all is a question of money. What I have now is quite nice.
- 120GB SSD for OS and programs
- 220Gb SSD for LR files
- 4x500GB RAID 0 as scratch for PS and PTGui
- 6x2000GB RAID 5 for image storage
The two SSDs are on the mainboard controller, the Raid 0 on a additional controller and the Raid 5 on another RAID controller.
Well I can say, I never have worked so quickly. It is amazing how fast everything gets with such a setup.
Transfare rates around 400-600Mb/s between the RAID arrays, instant program loading times, I really love it. I have never opened 20 x P65+ in PTGui so fast or opened a 20GB file in PS.
Is it the best ? Certainly not if I had the money, I would get 2x 120GB System drive as SSD with SLC mem and a SATA3/SAS6 interface and the same for the LR HD.
You could go even further and use 4 SLC SSDs in RAID 0 for a scratch disk, which would be fatser than any HD regardless (SATA/SAS)
If one has the money, one could do that with the whole image storage, but hell I don't even wanns start because for thet kind of money, one can buy a new computer every 6 months, or a car.
In the End there are two main mistakes made today. People who think they can use MLC based SSDs in a RAID setup and all other people who actually put all there RAID systems on one controller.
-
Well I think it all is a question of money. What I have now is quite nice.
- 120GB SSD for OS and programs
- 220Gb SSD for LR files
- 4x500GB RAID 0 as scratch for PS and PTGui
- 6x2000GB RAID 5 for image storage
The two SSDs are on the mainboard controller, the Raid 0 on a additional controller and the Raid 5 on another RAID controller.
Well I can say, I never have worked so quickly. It is amazing how fast everything gets with such a setup.
Transfare rates around 400-600Mb/s between the RAID arrays, instant program loading times, I really love it. I have never opened 20 x P65+ in PTGui so fast or opened a 20GB file in PS.
People on the OCZ forum post benchmark results (http://www.ocztechnologyforum.com/forum/showthread.php?66483-AS-SSD-Benchmark-Screenshots-%28post-yours%29) which give an idea of the monstrously fast speeds you get with SSDs. Unraided you get 200+ MB/sec sequential read, 2-disk RAID gives 400+. Somebody has 8 in RAID going past 1.3 GB per sec (http://www.ocztechnologyforum.com/forum/showthread.php?66483-AS-SSD-Benchmark-Screenshots-(post-yours)&p=470215&viewfull=1#post470215). Not shabby for consumer setups fraction of the price of enterprise solutions.
As an amateur setup I have a reasonably priced computer which I recently upgraded with a single OCZ Vertex for OS and scratch (the horror) and 8 gigs of RAM with 64 bit Win7. I just tested it by opening a 2.4 gig stitched bracketed multi-layered panorama, took 105 seconds. With my older system with 4 gigs and HDDs it took 10+ minutes - no kidding. I think it was mainly because the OS ran out of memory and swapped everything to HDD.
SSD is the most cost-effective ugprade one can make to a computer lacking it, with more memory coming a close second.
-
While SSD's are great, they're not the best choice for operations that do a lot of random writes and reads as in scratch. This is due to the SSD's performance degradation issues. Spinners in RAID-0 do not suffer this limitation.
Re the issue of scratch and data on the same spindles, I do agree with Farmer in theory -- however, in my experience with my machine I essentially never run into the issue of both operations tagging the spindle set concurrently. First, my machine set-up is an 8-core MacPro 3.2 with 24G RAM. I have an SSD for OS and apps, and then 4 SATA2 spinners in RAID-0 with a thin outer rim partition dedicated to scratch and the rest to image storage. On my machine, even when I'm doing a big pano, I typically process all the raws in C1, then read those tiffs into AutoPano. I have 24 gigs of ram in my machine, so those files all load, never tagging scratch. I wait for the pano to render, then bring the single large file into CS to perform my regular localized edits. About the only time I run into a double tag issue is when I'm batch processing a bunch of raws from a shoot in C1, then start working on a massive file in CS -- which is rare -- but even then I do not 'notice' the performance hit on my machine as the C1 batch is usually finished before I get very far along with my CS edits. CS5 will change this too, at least hopefully, if it utilizes onboard RAM more effectively. But as always, I respect that YMMV.
PS for Jeff: I would reconsider the SAS drive array. SATA3 is already here, and in a very short while you'll be able to load your box up with 4 @ 2TB SATA3 drives for probably less than the 4 smaller SSDs are going to cost, and it will likely be a lot faster. Just sayin...
Cheers,
-
While SSD's are great, they're not the best choice for operations that do a lot of random writes and reads as in scratch. This is due to the SSD's performance degradation issues. Spinners in RAID-0 do not suffer this limitation.
This is only true for SSDs without TRIM to a certain extent. Granted very few SSDs in operation have TRIM or even garbage collection, but all new OCZ Vertexes ship with TRIM, and their latest firmware includes TRIM support for all Vertex drives. Don't know about other manufacturers.
Even without TRIM they will whip HDDs in random read/writes since they have no moving parts, and doubt they would perform worse than the fastest HDD even with prolonged use.
If you're refererring to the limited read/write cycles of SSDs, that's FUD in normal use - you'll be upgrading your computer way before it becomes an issue.
The biggest caveat with SSDs is that there's no TRIM for RAID setups in the foreseeable future.
-
Did not realize the OCZ was shipping with TRIM already -- awesome!
-
Did not realize the OCZ was shipping with TRIM already -- awesome!
Yeah, it's pretty recent, and that's why I finally took the plunge and bought one and am quite happy with it. Will certainly add another drive in the future for scratch. I think I'll avoid RAID setups since TRIM doesn't work with them - but again I'm confident it would still be faster than HDD RAID.
SSDs still have some minor shortcomings. OCZ's official setup guide (http://www.ocztechnologyforum.com/wiki/index.php?title=How_to_set_up_Windows_on_a_VERTEX) claims that I should let the drive idle on Windows log-in screen for hours on end each week for garbage collection to clean up the pieces missed by TRIM. I'm highly skeptical about the necessity and utility of this and I'm not going to do any of it. I've taken benchmarks and will check occasionally to see if there's performance degradation.
-
Yeah, it's pretty recent, and that's why I finally took the plunge and bought one and am quite happy with it. Will certainly add another drive in the future for scratch. I think I'll avoid RAID setups since TRIM doesn't work with them - but again I'm confident it would still be faster than HDD RAID.
.
Ok just to clarify once again, only consumer graded SSDs (MLC) need the TRIM command and these are the drives which loose a lot of performance in a RAID. ( RAID = NO TRIM )
Compared to that SLC based SSDs, don't need it and can be used with RAID, the only drawback is that they are even more expensive.
It is also still true, that MLC based RAID system WILL loose a lot of performance over time. At a point it will be slower than any normal HD Raid, so it makes only sense for a temp. volume (Scratch), which can be refomratted and delted from time to time, or one should use professional SSDs.
In the end I think it doesn't make sense yet to use SSDs as storage. I mean if I take myelf I would need around 4TB live storage, which would mean with RAID 5 or 10 one would use 3-4 x 2TB drives. In SSDs one would need around 20 SSDs to get it.
So a SSD storage makes only sense for someone who only needs his current images on a fast RAID array.
-
Ok just to clarify once again, only consumer graded SSDs (MLC) need the TRIM command and these are the drives which loose a lot of performance in a RAID. ( RAID = NO TRIM )
Compared to that SLC based SSDs, don't need it and can be used with RAID, the only drawback is that they are even more expensive.
It is also still true, that MLC based RAID system WILL loose a lot of performance over time. At a point it will be slower than any normal HD Raid, so it makes only sense for a temp. volume (Scratch), which can be refomratted and delted from time to time, or one should use professional SSDs.
In the end I think it doesn't make sense yet to use SSDs as storage. I mean if I take myelf I would need around 4TB live storage, which would mean with RAID 5 or 10 one would use 3-4 x 2TB drives. In SSDs one would need around 20 SSDs to get it.
So a SSD storage makes only sense for someone who only needs his current images on a fast RAID array.
I wasn't referring to SSDs in storage sense, but OS and scratch disk sense. But I guess some people can afford to use SSDs for storage...
-
I wasn't referring to SSDs in storage sense, but OS and scratch disk sense. But I guess some people can afford to use SSDs for storage...
A few perhaps ;-), I would prefer to upgrade my whole computer every year for the same money ;-)
-
PS for Jeff: I would reconsider the SAS drive array. SATA3 is already here, and in a very short while you'll be able to load your box up with 4 @ 2TB SATA3 drives for probably less than the 4 smaller SSDs are going to cost, and it will likely be a lot faster. Just sayin...
Yeah, well the whole point of my system upgrade was to do it once and forget about it for 3 years (which is what I did last time). I know some SATA 3's are out there but I don't think they will be as fast as 1%K SAS drives...so I'll deal with it. All my images and LR cataloges will actually be on an external 6 drive stripped array (backed up to a duplicate 6 drive stripped array) that will be giving me sustained 350-400 MB/second...
and considering the 6 drives will be 2TB, that means I'll have just under 12TBs of backed up on-line storage that should be pretty fast...
So, of course, I order my MacPro then the rumors start about the dual 6-core boxes...ya know what? don't care...I just know my dual quad-core from 2007 is pretty darn slow and I need an upgrade...
-
Well, different strokes... .
Personally, I think 4 of these in R-0 would be pretty freaking kick-butt even in my old 8-core box: http://www.newegg.com/Product/Product.aspx...N82E16820148349 (http://www.newegg.com/Product/Product.aspx?Item=N82E16820148349)
-
Well, different strokes... .
Personally, I think 4 of these in R-0 would be pretty freaking kick-butt even in my old 8-core box: http://www.newegg.com/Product/Product.aspx...N82E16820148349 (http://www.newegg.com/Product/Product.aspx?Item=N82E16820148349)
As long as you completly delet them every month it should be nice, otherwise as scratch these SSDs in RAId would be a quick desaster. ( RAID = NO TRIM )
-
As long as you completly delet them every month it should be nice, otherwise as scratch these SSDs in RAId would be a quick desaster. ( RAID = NO TRIM )
Not for scratch. For folks that want an uber-fast working image I/O storage tank. As I indicated earlier in this thread, I do NOT recommend SSD's for scratch because of the wear issue, and scratch is exactly the type of operation that clogs SSD's. As a working image storage pot though, they would be the performance kings, easily outperforming an SAS array at not too much difference in cost once you factor in the SAS controller. (Actually, IIRC AnandTech did a comparison about a year ago and a pair of SATA2 SSD's in R-0 are about the same speed as a 4x R-0 SAS array in a sustained read or write, and smoked the SAS on random I/O -- SATA3 SSD will be better still...)
The downside for SSD's remains though --- depending on how much I/O you actually do, you'd want to scrub (recondition) an SSD array on some regular schedule, probably at least quarterly, to keep performance up... And one simple wipe and reformat won't do it, you need to repeat that about 4 times to reset the drive's write controller chip. So if you factor in the time required for this maintenance across your overall usage, things get muddier. Personally, if I was after performance, I'd still go with the SSDs over an SAS array, but that's me...
Cheers,
-
Well, different strokes... .
Thought I would follow up and let people know that I do have my new Mac Pro installed and "broken in"...
I have my main storage external in a pair of Burly Systems 6 bay in a RAID 0 (stripped) array...I have two of these and RAID 1 is backed up to RAID 2 nitely...I also have a Drobo Pro that get an occasional backup from RAID 2.
For the internal Mac Pro drives I have 4 600 gig SAS drives. I have my boot on a single drive and my Users folder on a second drive. I also have a RAID 0 stripped array as my Photoshop CS5 scratch drive. These SAS drives are running off of an Apple RAID card...the dual quad core, 2.93 GHz main CPU plus 32 gigs of ram are very, very fast for Photoshop CS5 (I'm running Ps CS5 with 70% allocated which works out to 21818MB for Photoshop).
The Burley Arrays are giving me about 350MB/sec through put and the SAS Boot and User drive is at up to 426 MB/sec for reads and 142 MB/sec writes...the scratch array is up to 434 MB/sec reads and 289 MB/sec writes (which really helps Photoshop for the rare times I hit scratch).
So far my only issue is the heat...I've had to add a fan under the deck to help get rid of the heat that is generated...it isn't so bad when the weather is cold but Chicago has recently had days in the 80's and the heat generated is really noticable (I have to work with my shoes off).
Rumor has it Apple will be coming out with new towers in July and SATA 3 and SSDs might be worth looking into in the near future but as of now, I'm happy with the new system and Photoshop CS5.
-
It sounds like you've built an awesome hot-rod Jeff, congrats on the new machine!
-
Off subject question for J. Schewe: I noticed that you've selected a relatively unknown (to me) DAS by Burly Arrays. I visited their web site as I'm in process of upgrading my system. I like the dual PM system http://www.burlystorage.com/ccp0-catshow/burlydualpm.html (http://www.burlystorage.com/ccp0-catshow/burlydualpm.html) but am experiencing sticker shock as this is more than the competitors. What drove you to this solution? I'd purchase a single system and raid 0 three drives and use the other three as a backup.
On another side note, I had an interesting call with the Burly support guy who is also a 20 year practicing photographer with Lightroom experience. He suggested that I stay away from the 600GB SATA 6.0Gbps 10000RPM - 3.5" - Western Digital VelociRaptor in a Raid 0 pair. He said that it's a 2.5 inch disk and that I'd get better scratch performance using a single 1TB SATA 7200RPM - 3.5" - Seagate Barracuda® 7200.12. The reason is that storage works from the outside of the disk and that a larger sized disk with more space would out perform the smaller platters. I don't know if this is correct or not but I did find this statement to have a true ring to it.
I keep tweaking my system specs as I learn more about what works well with Lightroom and Photoshop. By the time I decide I suspect that 6-core systems will be the norm. Oy
-
Off subject question for J. Schewe: I noticed that you've selected a relatively unknown (to me) DAS by Burly Arrays. I visited their web site as I'm in process of upgrading my system. I like the dual PM system http://www.burlystorage.com/ccp0-catshow/burlydualpm.html (http://www.burlystorage.com/ccp0-catshow/burlydualpm.html) but am experiencing sticker shock as this is more than the competitors. What drove you to this solution? I'd purchase a single system and raid 0 three drives and use the other three as a backup.
On another side note, I had an interesting call with the Burly support guy who is also a 20 year practicing photographer with Lightroom experience. He suggested that I stay away from the 600GB SATA 6.0Gbps 10000RPM - 3.5" - Western Digital VelociRaptor in a Raid 0 pair. He said that it's a 2.5 inch disk and that I'd get better scratch performance using a single 1TB SATA 7200RPM - 3.5" - Seagate Barracuda® 7200.12. The reason is that storage works from the outside of the disk and that a larger sized disk with more space would out perform the smaller platters. I don't know if this is correct or not but I did find this statement to have a true ring to it.
I keep tweaking my system specs as I learn more about what works well with Lightroom and Photoshop. By the time I decide I suspect that 6-core systems will be the norm. Oy
I've got the 600GB SAS 15k drives-they are indeed 3.5" not 2.5" and they cost a lot more than the 10K. And yes, my System and User folders are on two separate drives on the outside of the platters (first partition) as suggested by Rick. The internal 2 drive array is only for Photoshop scratch. All 4 drives are running off of the Apple raid card since it supports SAS drives and can be made to boot from either one of the drives connected to it or an arrayed pair connected to it.
The jury is still somewhat out regarding the use of two drives and the first partitions for faster small block random read and write access...It's not at all easy to compare and test this. Things seem real fast now compared to my aging old MacPro. But the only slightly daunting aspect is the fact it's "complicated" in that you have two separate places where stuff has to be and it's up to you to keep track and back it up.
As to the Burly Dual PM, at about $700 before drives, I don't know where else you'll find a 6 drive box...I've seen 4, 5 and 8 drive boxes but this is the only 6 bay I'm aware of. I compared this box to Cal Digit, G-Tech, Buffalo Station and a few others I think. This was the best overall deal when it came to the box, the drives, the card and the final price and capacity.
I've had a Burly raid 5, 4 bay box running 24/7 for 5 or 6 years on a server...once you get the stuff set up, it's durable. They ain't sexy looking...that's why I hide mine under the desk where all you can see is the blinking lights!
:~)
-
Hey all,
I just read trough the thread and one thing kept bugging me the whole way. This is a thing that I think only Jeff Schewe has taken into account (that I noticed).
There was a lot of talk about 7200 and 10000rpm drives and then the better option, which is SSD - drives, debate about Trim and different Raid - levels.
But no-one except Jeff mentioned a raid controller. I don't know Macs too well, but since they are Intel based these days, I'm pretty sure they are basically the same when it comes to on board sata controllers (which suck big time).
If you really want performance out of your raid arrays, the only way to fly is an "third party" controller card. Preferably from a good manufacturer (Areca comes to mind) and which support Raid 6.
Why Raid 6 when my mobo does Raid 5?
Raid 5 is quite prone to hard-drive failures. Lets say that you buy 4 disks for your array. You'll probably go with same size disks (right), then you'll want same kind of drives for your array (right or wrong, depending), but the last thing will be the thing that'll probably bite you in the backside of things in the future. You buy them from the same place at the same time (definitely wrong).
This is fine when everything is fine. But these days when hard drives are cheap, you'll probably go with 1-2Tb drives. The problem starts to build up.
The next part is when they start failing (not if). They are bought at the same time, so they are probably from the same batch. They have been used about the same amount of time, so when one of them fails and you replace it, the controller starts to build the array again. During this time the activity is quite high on the disks and with large arrays the rebuild time is so long, that you are lucky if another drive doesn't go out during the rebuild. This is where Raid 5 fails, in the middle of the rebuild, another drive fails and the whole array is gone.
Raid 6 patches this a bit, since two drives can go, so it's reducing the possibility of a total failure, but then again, Raid 6 support is quite minimal (you'll have to go with the controller card), you'll have to get more disks for this to work and finally, Raid 6 is slower than Raid 5.
On that note, fake raid systems (mobo/software raid) are not really good, since Raid 0 and Raid 1 are the only ones that work even close to what they are supposed to. Raid 0 does make the I/O faster, but not by too much. Raid 1 is slower than one disk by itself and Raid 5 performance is a bad joke.
I use fake raid (Raid 1 arrays only) on my backup server, because it doesn't have to be that fast. Only thing I want is some redundancy. I do have two SSD - drives on my main rig in Raid 0, but that's because I wanted them to be a bit faster and also I wanted the combined space of 2 x 64Gb for my operating system.
What comes to Trim support and the lack of it, my disks are quite early and cheap versions, which are without Trim and I cant "feel" any slowing down yet (been using this installation of Win 7 for about 5 months).
And if you don't store anything important to the disks (Photoshop scratch, operating system (which is non critical to me) etc.) you can do a re-format with a linux live disk, so that your disks are returned to the state when they were brand new. That should fix the lack of Trim if you do it often enough.
These were just my 0.02$, so feel free to comment.
Janne
-
Well, RAID controllers are probably the most important thing. In my new workstation there are two, one very powerful one from LSI for my 5 x 2TB drives which make a RAID 5. Why RAID 5 ? Because it is just faster than RAID 6 and it is secure enough. I have another two external RAID 6 backup devices.
The second RAID controller is a Adaptec and uses RAID 0 with 4 x 500GB drives, which make up my scratch disk.
There are two more SSDs hanging on the mainboard, which are not used in any RAID, but as System and Lightroom drive.
I can say that this system is just amazing. I made my decision against SAS drives because I don't see any huge benefit. I personally think that a combination of SSDs and normal SATA drives is the better choice.
-
Hey Cristopher,
Sounds like a sweet system and it seems that you've made solid choices when building it.
I can understand your Raid 5 choice here well, since Raid 6 is somewhat slower system. But as a only means of backup or disk space without backup, I'd stay far away from 5.
Janne
-
Hey Cristopher,
Sounds like a sweet system and it seems that you've made solid choices when building it.
I can understand your Raid 5 choice here well, since Raid 6 is somewhat slower system. But as a only means of backup or disk space without backup, I'd stay far away from 5.
Janne
Yes you are absolutely correct that as a backup it should be a RAID system which can manage two failures. As a Working RAID drive I prefer the speed of RAID 5.
-
I've noticed these last 6 months while managing a 6-disk zfs (3x1TB, 3x1.5TB) system that raid5/6 is not reliable enough for storing data. Bit flips happen and raid5/6 (or whichever level) doesn't have a mechanism to check the data when it's read.
I scrub my disks on zfs once per week and almost every time there's at least a few bits changed here and there causing a 32 byte block to be reconstructed. I believe that currently zfs is the only production capable filesystem that has end-to-end data integrity (btrfs is years away from production quality).
So for actually storing data on big and cheap disks, I'd stay away from any raid system if you want to keep your files bit perfect.
I've actually noticed that my lightroom library doesn't have any performance problems residing on the server that would actually impact my workflow, performance is almost as good as on ssd (gigabit network of course).
-
What comes to Trim support and the lack of it, my disks are quite early and cheap versions, which are without Trim and I cant "feel" any slowing down yet (been using this installation of Win 7 for about 5 months).
On my Intel SSDs, trim is definitely useful. I have several hundreds of thousands of e-mails in my e-mail database, basically everything I sent and received since 1993: my e-mail program takes a few seconds to start on a fast hard drive, and around 1 second on the SSD. After a month or so, the SSD is down to the speed of the hard drive. Running the Intel utility restores the near instant response of the SSD. Of course, this is an ideal scenario for the SSD as all those mails are organized in hundreds of folders which have their own databases and indexes. In order to display the folder structure and the mail headers, there are lots of small reads all over the place, and lots of small changes that can fragment stuff as I use the mail program.
-
On my Intel SSDs, trim is definitely useful.
Even nicer than trim IMO is the sandforce solution... Currently available on enterprise-level OCZ and OWC SSD's...
-
To think that none of this interchange would have been possible (or certainly would have been thought lsd induced) when I bought my first 500C. Happy, innocent days!
Rob C