Pages: 1 [2] 3 4   Go Down

Author Topic: PC specs for processing large photos  (Read 24456 times)

dmerger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 680
PC specs for processing large photos
« Reply #20 on: December 13, 2005, 10:39:22 am »

Tim, here is what I consider to be the most relevant quote from the first article.

"The enthusiasm of the power user community combined with the marketing apparatus of firms catering to such crowds has led to an extraordinarily erroneous belief that striping data across two or more drives yields significant performance benefits for the majority of non-server uses. This could not be farther from the truth! Non-server use, even in heavy multitasking situations, generates lower-depth, highly-localized access patterns where read-ahead and write-back strategies dominate. Theory has told those willing to listen that striping does not yield significant performance benefits. Some time ago, a controlled, empirical test backed what theory suggested. Doubts still lingered- irrationally, many believed that results would somehow be different if the array was based off of an SATA or SCSI interface. As shown above, the results are the same. Save your time, money and data- leave RAID for the servers!"

Here is what I consider to be the most relevant quote  from the second article.

"If you haven't gotten the hint by now, we'll spell it out for you: there is no place, and no need for a RAID-0 array on a desktop computer.  ...  Bottom line: RAID-0 arrays will win you just about any benchmark, but they'll deliver virtually nothing more than that for real world desktop performance. That's just the cold hard truth."

Tim, you seem to have your heart set on RAID 0, and I hope it does deliver a significant performance increase for you.  I'm not ready to jump onto the RAID 0 bandwagon, yet.  I'll wait until I see a competent, unbiased test that shows a significant performance increase when used for Photoshop scratch disk.
Logged
Dean Erger

Ben Rubinstein

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1822
PC specs for processing large photos
« Reply #21 on: December 13, 2005, 03:48:24 pm »

Correct me if I'm wrong, I thought that CS2 could only use 2 gig of RAM?

I have a p4 1.6 with 2 gig ram and a fast couple of HD's with the program on one and the caches/pics on another.

Running batch actions on 240 RAW files (open, PTLens, auto contrast, shadow/highlights, smart sharpen, save) took over 4 hours with nothing else running. The annoying thing is that short of a major system upgrade, i.e. dual processors, etc, it ain't going to get that much better. A wedding can easily be 300 files that need batching.
Logged

dandill

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 84
    • http://quantum.bu.edu
PC specs for processing large photos
« Reply #22 on: December 13, 2005, 04:43:10 pm »

Quote
...
Running batch actions on 240 RAW files (open, PTLens, auto contrast, shadow/highlights, smart sharpen, save) took over 4 hours with nothing else running.
[a href=\"index.php?act=findpost&pid=53482\"][{POST_SNAPBACK}][/a]
I'd be curious to know whether the limiting step was disk IO or RAM (page file usage). Is that something you could deteremine, say by using the Task Manager to monitor an example batch of, say, 10 files? Also, do you have a dedicated partition for the page file? If not, I think that would surely help, though I don't know how much.
Logged
Dan Dill

DarkPenguin

  • Guest
PC specs for processing large photos
« Reply #23 on: December 13, 2005, 04:56:10 pm »

Even if PS doesn't use the extra memory windows will use it for disk cache (or just as memory for other apps) which should help.

But to do that many RAW files you'll want more more memory, more CPU and probably a faster disk.

All of that should be relatively cheap to do at least until you add in your 19,000% VAT or what ever it is you pay.
« Last Edit: December 13, 2005, 04:57:54 pm by DarkPenguin »
Logged

Tim Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2002
    • http://www.timgrayphotography.com
PC specs for processing large photos
« Reply #24 on: December 13, 2005, 06:55:20 pm »

Quote
Correct me if I'm wrong, I thought that CS2 could only use 2 gig of RAM?

[a href=\"index.php?act=findpost&pid=53482\"][{POST_SNAPBACK}][/a]

I believe that's true (actually 1.7 gig is closer, if I recall correctly)  I would suspect that the next version would support closer to 4 - given the fact that Vista should be widespread by then...  and the mpx wars are continuing.
Logged

Ben Rubinstein

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1822
PC specs for processing large photos
« Reply #25 on: December 13, 2005, 07:19:19 pm »

I've been wanting to go for a laptop since I'm opening a studio and would prefer that to having two desktops networked, finding a laptop that takes even 2 gig ram is hard enough, haven't even seen any that take over, at least not in a normal price range, sigh...

Dark Penguin, it just keeps adding up doesn't it, all my ram slots are filled so it would mean trashing one stick to upgrade, the motherboard doesn't support 64bit (I assume, it's two years old) and so a new processor, new motherboard and say 4 gig ram would set me back a new lens.

Who ever said that digital was cheap?  
Logged

kaelaria

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2223
    • http://www.bgpictures.com
PC specs for processing large photos
« Reply #26 on: December 14, 2005, 11:39:09 am »

RAID0 is VERY useful in power gaming desktops.  It lowers my winxp boot time, cut the install time in 1/2, cuts the install time of large games and more importantly, cuts the load time dramatically.  Photoshop...doesn't do a thing, it's too small, so are images.
Logged

kaelaria

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2223
    • http://www.bgpictures.com
PC specs for processing large photos
« Reply #27 on: December 14, 2005, 11:48:36 am »

Quote
Tim, you should do some research on actual RAID 0 performance for a desktop PC.  Every test report that I've read found no significant performance increase.  This topic has been discussed on this forum in the past.
[{POST_SNAPBACK}][/a]


You are OBVIOUSLY not a gamer with top level hardware, or you wouldn't make such incorrect statements.  See this article for the straight poop: [a href=\"http://www.viaarena.com/default.aspx?PageID=5&ArticleID=392&P=5]http://www.viaarena.com/default.aspx?PageI...ticleID=392&P=5[/url]
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
PC specs for processing large photos
« Reply #28 on: December 14, 2005, 03:26:35 pm »

Quote
RAID0 is VERY useful in power gaming desktops.  It lowers my winxp boot time, cut the install time in 1/2, cuts the install time of large games and more importantly, cuts the load time dramatically.  Photoshop...doesn't do a thing, it's too small, so are images.
[a href=\"index.php?act=findpost&pid=53525\"][{POST_SNAPBACK}][/a]
You just proved the very point you seem to be attempting to refute--that RAID 0 offers limited performance benefit to Photoshop users & digital photographers.
Logged

kaelaria

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2223
    • http://www.bgpictures.com
PC specs for processing large photos
« Reply #29 on: December 14, 2005, 03:32:39 pm »

Read more carefully.  He tried to say it was only useful in servers, not ANY desktops, which is 100% false.
Logged

Tim Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2002
    • http://www.timgrayphotography.com
PC specs for processing large photos
« Reply #30 on: December 14, 2005, 03:58:32 pm »

If I'm maxed out on as much ram as PS can effectively make use of; and I'm regularly hitting efficiency levels of between 10 and 90% (eg large 16 bit stitched panos) would I be unreasonable in expecting a non-trivial improvement from a raid 0 configuration?
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
PC specs for processing large photos
« Reply #31 on: December 14, 2005, 04:32:24 pm »

Quote
Read more carefully.  He tried to say it was only useful in servers, not ANY desktops, which is 100% false.
The context of discussion here is Photoshop and other applications of interest to digital photographers. By your own admission, and the results obtained from others' tests, RAID 0 offers little benefit to the digital photographer. The issue of whether it is useful to servers or gamers is entirely separate, and irrelevant here. Or do you have any real-world testing that demonstrated RAID 0 having benefit to Photoshop users?
Logged

kaelaria

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2223
    • http://www.bgpictures.com
PC specs for processing large photos
« Reply #32 on: December 14, 2005, 04:37:08 pm »

Nope, as I said - RAID0 is not for PS.  The context here that I am addressing is clearly stated in the other posts...the other poster claimed there was no benefit to a 'desktop' pc vs. a server.  Many people use thier desktops for much more than JUST PS, and raid0 is a great benefit to other areas of that use.  He didn't say Photoshop, he said desktop.
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
PC specs for processing large photos
« Reply #33 on: December 14, 2005, 05:04:26 pm »

Quote
...the other poster claimed there was no benefit to a 'desktop' pc vs. a server.
And for the great majority of "desktop" users, that statement is correct. About the only thing other than high-end gaming where RAID 0 would offer significant benefit is video editing, and neither of those are of much importance to most "desktop" users. You're beating a strawman and a dead horse.
Logged

dmerger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 680
PC specs for processing large photos
« Reply #34 on: December 14, 2005, 06:15:24 pm »

First, let me apologize if the following quotes do not display correctly.  This is my first attempt to use the quote feature.

Kaelaria, please read what I wrote more carefully.  You have repeatedly attributed statements to me that I did not make.  One slip would not be so bad, but repeated incorrect statements about what I wrote is not appreciated.  

Quote
QUOTE(dmerger @ Dec 12 2005, 04:30 PM)
Tim, you should do some research on actual RAID 0 performance for a desktop PC.  Every test report that I've read found no significant performance increase.  This topic has been discussed on this forum in the past.
*




You are OBVIOUSLY not a gamer with top level hardware, or you wouldn't make such incorrect statements.


Just what about my statement that you quoted is incorrect, Kaelaria?  

[/QUOTE]He tried to say it was only useful in servers, not ANY desktops, which is 100% false.
Quote

I did no such thing.  If you have a beef, it's with the people at AanandTech and Storage Review.  

..the other poster claimed there was no benefit to a 'desktop' pc vs. a server. Many people use thier desktops for much more than JUST PS, and raid0 is a great benefit to other areas of that use. He didn't say Photoshop, he said desktop.
Quote

Another blatantly false statement.

Kaelaria, before you accuse me of making incorrect or false statements, you should read what I wrote, and not put words into my mouth.  I don't appreciate it.

I never said RAID 0 would not be beneficial.  I was merely trying to help Tim by pointing him to some independent, in-depth test reports.  He seems to have opted to ignore the conclusions and whole point of the reports by lifting out a couple of lines out of context.  That's entirely fine with me.  He's free to ignore the test reports.  Maybe the reports are wrong.  Maybe Tim knows more than the testers at Storage Review and AanandTech.  Maybe you do too, Kaelaria, and I have no problem with you stating your disagreement with the test reports.  But, please, don't attribute statements to me that I did not make.
Logged
Dean Erger

DiaAzul

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 777
    • http://photo.tanzo.org/
PC specs for processing large photos
« Reply #35 on: December 14, 2005, 06:21:34 pm »

Quote
If I'm maxed out on as much ram as PS can effectively make use of; and I'm regularly hitting efficiency levels of between 10 and 90% (eg large 16 bit stitched panos) would I be unreasonable in expecting a non-trivial improvement from a raid 0 configuration?
[{POST_SNAPBACK}][/a]

I would disagree very much with the other posters and say that (for the scratch disk) you would benefit from Raid 0 configured drives.

All the information that has been posted so far has been for applications launched from Raid 0 drives and in the test scenarios quoted average file sizes around 60-100kbytes (for the first article) and not stated for the second. There was no information given for a photoshop configuration where the Raid 0 drive was used specifically as a scratch disk.

Now, reading back into the two articles that refuted the benefits of Raid 0 I do have some problems with the way that the tests were conducted. The most important aspect to bare in mind is that windows buffers IO transfers in memory and, depending on the configuration of the application, can read ahead data in chuncks of 16Kbytes up to Mbytes. Also, modern day disks also include cache memory of 8-16Mbytes. Now, given that the files sizes used in the tests could be quite small it is entirely possible that files were written to disk, but subsequently read back from the cache. One would expect in this situation for a non Raid and Raid system to be close in terms  of performance. Even if there was a cache miss, there is possibility to give acceptable speeds for non raid drives provided the file can be read into the buffer in perhaps one or two chunks.

So, to summarise the issue here, the tests quoted against Raid-0 focus on tests which may or may not be using relatively small file sizes, may or may not be benefitting from large read/write buffers and caching on disk, and don't use a separate Raid-0 purely for photoshop scratch disk. I would conclude, therefore, that the articles are irrelevent to the discussion in hand and that for file sizes of 250Mbytes+ then Raid-0 will quite probably give a performance advantage over non raid configurations (I believe that the game demonstration with large amounts of cached data is more credible than the sterile business mark tests).

I too am looking for a new PC and the following is broadly in line with the spec I am planning to go with (though an 8-way due core opteron = 16-cores, would also be quite nice ;-))
[a href=\"http://www.armari.co.uk/system.asp?SysID=362]Armari Workstations[/url] - if you want serious grunt then check out the servers ;-)


ciao ciao  
« Last Edit: December 14, 2005, 06:28:49 pm by DiaAzul »
Logged
David Plummer    http://photo.tanzo.org/

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
PC specs for processing large photos
« Reply #36 on: December 14, 2005, 06:28:17 pm »

Has anyone conducted RAID 0 vs single drive tests using Photoshop doing a batch RAW conversion and save-to-disk? We seem to be arguing over a few anecdotal impressions and tests that may not have any relevance to a digital photographer with say 400-2000 RAWs to batch process for a web gallery or whatever.
Logged

kaelaria

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2223
    • http://www.bgpictures.com
PC specs for processing large photos
« Reply #37 on: December 14, 2005, 07:19:09 pm »

Quote
Here is what I consider to be the most relevant quote  from the second article.

"If you haven't gotten the hint by now, we'll spell it out for you: there is no place, and no need for a RAID-0 array on a desktop computer.  ...  Bottom line: RAID-0 arrays will win you just about any benchmark, but they'll deliver virtually nothing more than that for real world desktop performance. That's just the cold hard truth."

[a href=\"index.php?act=findpost&pid=53449\"][{POST_SNAPBACK}][/a]


Don't play the blame game.  You posted this.  Don't say ' oh, I didn't write it, so I didn't say it'.  Yeah, ya did.  You CLEARLY said you agree with it.  And you are dead wrong.  I'm not putting any words in your mouth.

You can read and quote test reports till the cows come home.  You've admitted you DON'T EVEN HAVE OR HAVE TRIED A RAID0 ARRAY, so obviously you are talking out your corn hole.  I on the other hand, have been using RAID0 arrays in my personal and professional systems for over 4 years, and have one as I type, beside me.  I also have single drives in this system, and have seen first hand, OBVIOUS results between the two drive systems.  I'm not simply pasting text from the internet without fully understanding it.  Don't pretend to be a computer guy just because you read articles.  It helps to actually have the equipment being discussed.
Logged

kaelaria

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2223
    • http://www.bgpictures.com
PC specs for processing large photos
« Reply #38 on: December 14, 2005, 07:25:42 pm »

Quote
I would disagree very much with the other posters and say that (for the scratch disk) you would benefit from Raid 0 configured drives.[a href=\"index.php?act=findpost&pid=53553\"][{POST_SNAPBACK}][/a]

I thought so too until I actually tried it.  My last system (last month) used a raid0 array for the main boot drive, applications and windows cache file.  My ps scratch was on another single drive.

My currect system uses a single drive for boot and windows cahce file, with the PS scratch on a raid0 array.  

Using a benchmark script (I can't remember where it was right now, it involved a large pic of a horse), I tested a few configurations on the old system several months ago.  I found a good jump in performance when I moved the PS scratch off the same volume as the windows cache file.  

When I was setting up this new system, I wanted to be sure to maximize performance, so I tested a few other configurations as well.  I found only a couple seconds difference between having the ps scratch on the single or raid0 array, but still a good jump in performance as long as the two were separated.

The only reason I switched from the raid0 boot to the single drive, is I also run flavors of linux, and now can partition the single drive for them.  LILO still will not boot from a raid0 array.
Logged

BernardLanguillier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13983
    • http://www.flickr.com/photos/bernardlanguillier/sets/
PC specs for processing large photos
« Reply #39 on: December 14, 2005, 08:35:01 pm »

Quote
I thought so too until I actually tried it. 
...
When I was setting up this new system, I wanted to be sure to maximize performance, so I tested a few other configurations as well.  I found only a couple seconds difference between having the ps scratch on the single or raid0 array, but still a good jump in performance as long as the two were separated.
[a href=\"index.php?act=findpost&pid=53562\"][{POST_SNAPBACK}][/a]

Thanks for the info. A few questions if you don't mind:

- what types of drives did you configure in RAID0 for the scratch?
- how large were the files you used for this testing?
- what operations were done on these files?
- how big was the scratch at that time?

Thanks.

Regards,
Bernard
Pages: 1 [2] 3 4   Go Up