Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: aebolzan on February 08, 2013, 10:31:53 am
-
I am upgrading the PC that I use for editing in Lightroom. It uses an AMD Phenom X3 2.6 Ghz processor and I was looking at the AMD FX-8150 8-Core and AMD FX 6100 6-Core processors.....I saw in the videos of Michael and Jeff that they say that 4 cores makes a difference for LR....so the 6 cores seems more than OK and 8 cores should be even better...but..is that so? Can LR really use 8 cores or it would be just a waste of money?.
thanks for your advise,
Agustin
-
Can LR really use 8 cores ?.
Yes. You'll mainly see the improvements when importing and building previews, exporting and also with more responsiveness in the develop module too.
This assumes there aren't other major bottlenecks in the system.
-
Augustin, your question intrigued me. I normally run LR on a six core, single chip, 3.33 GHz machine. I find that it routinely maxes out all six cores. Like you, I wondered how many it could use. I had a machine with LR 4.3 installed on it, but the computer is mainly used to do image processing using Matlab. Matlab is very good at parallelizing, and it's most convenient to use 64 bit floating point per color plane, so the machine has two six core (twelve thread) processors.
Here's the top-level machine configuration:
(http://www.kasson.com/ll/T7600.JPG)
Here's what Intel has to say about the processors:
(http://www.kasson.com/ll/IntelE5.JPG)
The image disk system is a fast Hardware RAID-0 (striped) with three 3TB disks in the array. The OS/App disk is a SSD.
I asked LR to import 3500 Nikon D3 raw images. The CPU load looked like:
(http://www.kasson.com/ll/LRImport.JPG)
That was a big surprise. I waited for LR to render the images in the background, and the CPU load stayed about the same. At all times the performance was very crisp, with images coming up instantaneously whether or not they'd been rendered in the background or not.
I decided on a tougher test. I imported 17 6000x64000 pixel images from a Betterlight back. The CPU load never got any higher, but when I asked it to show me an image it hadn't yet rendered in the background, there was a noticeable delay. When I played with these big images in the Develop module, performance was snappy, and I once saw the CPU load get to 20%. Exports created loads of around 10%.
So what's going on here? Why can't LR use the CPU cores more effectively? Two things occur to me. The first is that the processing is disk-throughput bound. The disk subsystem is quite capable, but maybe too weak to keep up with the CPUs. The second is that LR is using the graphics processing system in preference to the CPUs. The display adapter is an AMD FirePro V4900, which, as graphics processors go, is not a particularly fast device. I have no convenient way of monitoring the GPU load on this machine. The machine has no dedicated graphics coprocessor installed.
One side point: LE is often criticized for being memory intensive, but on this machine it uses proportionally very little of the available memory. It has plenty of room to cache many complete images in RAM, but I can tell from the disk load that it is not doing so.
I have no complaints about the way LR performs on this machine, but it certainly doesn't use all the cycles that are available to it. I'm guessing that, if a 12-core computer can't get over a 20% load running LR, that it would be fairly easy to configure an 8-core machine where the CPU wouldn't be used to full capacity.
-
I did some web research that indicated that LR 4.x doesn't support GPU acceleration,. Can someone who knows verify that?
That leaves only the disk subsystem as the likely bottleneck.
I have read comments from Adobe people that some raw processing is difficult to parallelize. That might explain the Nikon import results, but with so many images to choose from it should be easy to vectorize by image. The Betterlight imports were from TIFFs.
Jim
-
AFAIK, lightroom cannot use any GPU acceleration.
Vectorization and threaded parallellisation is two different things. Very many problems can be either vectorized or threaded (or both) but the performance benefits may be less than intuitively assumed. There are also significant costs (code bloat, test bloat, low-level hard-to-maintain code, using developers for optimization instead of implementing new features...)
I have the impression that exploiting "latest & greatest" hardware to its fullest is not on the top of the Lightroom teams agenda. I might be wrong.
-h
-
You're right about vectorization. I was imprecise.
Jim
-
I asked LR to import 3500 Nikon D3 raw images. The CPU load looked like:
Are you building 100% previews on import ? I assume not from that screenshot. If all you're doing is moving files from one place to another it doesn't take much CPU power.
Building 3k 100% previews should keep the processors at 100% utilisation for an hour or more in my experience.
Try exporting all those 3k files as 16 bit TIFs and see what gets used.
-
Are you building 100% previews on import?
Here's what happens when LR is building the 1:1 previews:
(http://www.kasson.com/ll/LR1to1previewsa.JPG)
Same as before, except that LR is using more memory. This about ten minutes into the process. The memory use grew for the first few minutes, then stabilized at about 17GB -- that's total, not just LR's usage. BTW, the reason there are 24 core usage graphlets rather than 12 is because Windows apparently considers each thread as a "CPU" for the purpose of this display. When I looked at the CPU usage in detail, I saw peaks to 25%:
(http://www.kasson.com/ll/LRdngrendering.JPG)
I had LR build previews for some Hassy DNG files, and saw peaks to 34% total CPU load, however, the system saw no need to increase the CPU clock rate to "turbo", or even to nominal:
(http://www.kasson.com/ll/LRmaxFreq.JPG)
I looked at the disk usage during the preview construction process:
(http://www.kasson.com/ll/LRdisc1to1a.JPG)
It appears the disk is not being used to its capacity.
-
Try exporting all those 3k files as 16 bit TIFs and see what gets used.
Here you go:
(http://www.kasson.com/ll/LRexport.JPG)
The load is more constant than it is when creating the previews:
(http://www.kasson.com/ll/LRexport%20history.JPG)
All this got me wondering if there was something wrong with my disk subsystem. I used Passmark's benchmarking program to analyze the striped disk, which is where the LR catalog, cache, and the image files are. FIrst the overall results:
(http://www.kasson.com/ll/edrivemark.JPG)
Then the sequential read results, which should dominate the preview construction traffic:
(http://www.kasson.com/ll/edrivesr.JPG)
Then the sequential write results, which should dominate the export traffic:
(http://www.kasson.com/ll/edrivesw.JPG)
And finally, the random seek results, which are probably pertinent to LR accessing its database, assuming it's not stored in RAM:
(http://www.kasson.com/ll/edreivers.JPG)
In all cases, it appears that the disk subsystem offers a lot more performance than LR is using.
Is there something that's keeping all the cores from being used by any app? Matlab runs the way I'd expect, but maybe there's something that's keeping LR from using them all. It doesn't look like it from the CPUMark:
(http://www.kasson.com/ll/cpumark.JPG)
Even though the scalar performance (there I go again -- fifty years of habit die hard; mea culpa) of the CPU is not off the charts:
(http://www.kasson.com/ll/cpusinglet.JPG)
Something wonky in the RAM? Nope:
(http://www.kasson.com/ll/memmark.JPG)
-
I think the simple answer to the question is simply that from a software developer's perspective, it can be difficult to write code that utilizes all cores.
-
To answer the original question of "should I buy a computer with 6 or 8 cores?", we should remember that LR4 which we're testing today is well underway to be replaced by LR5, and before the OP's computer gets retired, there will probably also be an LR6 and maybe LR7.
So... buy for the future. Get as much bang for your buck as you can, but don't pay excessively for "bleeding edge" technology. Even if a slightly smaller machine might be good enough today, you might end up replacing it sooner, and that will remove any cost saving.
-
Those that say you'll max out your CPU doing normal operations either are 1 - full of BS or 2 - don't know how to measure. NO, you will not see a difference. The only time you will even max out a dual or quad core is by doing MULTIPLE operations concurrently such as 4 different exports at the same time. Each operation in LR can only use one core. So the most efficient use is one operation per core you have. 6 core means 5 exports going and normal use, or 6 exports and let her rip. Simple as that. For most normal operations, the disk system is by far the bottleneck anyway.
-
don't know how to measure.
So kindly enlighten me why my normal 4% usage on random cores changes to 100% on all 8 cores of my 3770K when I do an import into Lightroom ?
-
Post a screenshot of your cores maxed out during that import.
-
Not maxed out on this one, but hardly using a single core is it ?
-
Import is also using multiple operations. I have no idea what those spikes are, but that's not normal LR behavior at all. All the rest of that chart is though - the 20% or so flat parts. Still waiting for the screenshot you claim of 100% while importing...Should be extremely easy - copy a big folder of files to a card and import them somewhere, let it go for a few minutes to get a solid chart :) I'll hold my breath...or do you just want to concede you are option 1?
-
also - use the built in resource monitor like the others.
-
that's not normal LR behavior at all.
It's what I see here whenever I use LR and it's definitely using more than a single core.
-
Here you go: http://www.youtube.com/watch?v=l0hG3lp9KhE&feature=youtu.be
-
Not maxed out on this one, but hardly using a single core is it ?
Not wanting to step on any toes here but this image is for rendering isn't it.. not import? I don't have a card handy to do a test but I think my CPU runs pretty high while rendering...
-
Lol it wouldn't surprise me in the least if some of these guys don't know the difference :)
-
Not wanting to step on any toes here but this image is for rendering isn't it.. not import?
Indeed, for me rendering 100% previews is part of the whole importing process.
The important aspect here is not some pedantic inspection of the process I used for my screen shot, but whether LR uses multiple cores. I think my screen shots are pretty emphatic examples that it does.
I always have the CPU usage widget running on my desktop and it's pretty clear that LR uses multiple cores to speed things up on virtually every task. Even throwing the sliders about in develop can cause the CPUs to max out briefly (see attached screen shot throwing the CT about).
-
Thanks for proving my points! You are now options one and two at the same time :)
-
Thanks for proving my points! You are now options one and two at the same time :)
You're very quick to criticise, but so far you've offered no explanation of why LR would be behaving as it does or offering any evidence that it doesn't use multiple cores.
Adobe in their own document on optimising Lightroom's performance at http://helpx.adobe.com/lightroom/kb/optimize-performance-lightroom.html say:-
"Options that can help increase performance include:
64-bit, multiple-core processor (for best performance, up to six cores. The extra power is especially important if you use multiple or high-resolution monitors which require more power)."
Are they wrong too ?
-
You obviously didn't watch, or are incapable of comprehending, the video I posted. Good luck - those with more common sense will benefit from it :)
-
You obviously didn't watch, or are incapable of comprehending, the video I posted.
Yours ? oh dear :(
You're using a system that you admit is bottle necked by the disk system (your quote @ 2:45), hardly the basis for any credible demonstration is it ?
Then you think a quad core system running at over 70% utilisation across all cores isn't using multiple cores ?
And you have no answer to why Adobe recommend using multi cored systems or why systems suddenly jump to 92% utilisation when throwing the CT slider around in the develop module ?
-
I have many systems running LR. One is a dual core with hyper-threading (appears as 4); LR imports use 4 threads for importing on this system and all are maxed. I have a quad-core system hyper-threaded as 8. On this system imports don't max all cores, but most of them are churning away at very high utilization. I also have an 8 core system hyper-threaded to 16. On this LR doesn't use more than 8 threads. It basically schedules 1 thread/physical core and the hyper-threading capability doesn't figure into it.
-
Here's what happens when LR is building the 1:1 previews:
Jim, I'd be interested to see what your CPU usage charts look like with HyperThreading disabled, or with LR set to use only the 12 physical cores in your machine. (You can set the latter by right-clicking lightroom.exe in the Processes tab of Task Manager, selecting "Set Affinity", and then unchecking all of the odd-numbered CPUs, leaving the even-numbered ones checked. The even-numbered CPUs represent the physical cores.)
I've been running similar perf tests on my 4-core machine (8 logical cores with HyperThreading), both for rendering 1:1 previews and for exporting a set of images. In each case, LR4 (and LR5 Beta) uses at least 4 threads to render or export each image. (I suspect it uses each thread to render a different part of the image, with another thread to combine the results into a finished preview or exported image.) When I disable HyperThreading on my machine, all 4 cores have steady peaks at 98-100% utilization for each image, with dips to 25-50%.
-
Jim, I'd be interested to see what your CPU usage charts look like with ... LR set to use only the 12 physical cores in your machine.
Joe,
I followed your instructions and got the same results. I don't think the resource monitor changes the denominator when a program is restricted to the physical cores, so what that means is the task is running at about the same speed with the change you suggested as before. I tried building previews and exporting.
Jim
-
I followed your instructions and got the same results. I don't think the resource monitor changes the denominator when a program is restricted to the physical cores, so what that means is the task is running at about the same speed with the change you suggested as before. I tried building previews and exporting.
Thanks, Jim. You're right, the denominator doesn't change unless you disable HyperThreading at the BIOS level. So your total CPU utilization was around 21% when using just the 12 physical cores?
-
Thanks, Jim. You're right, the denominator doesn't change unless you disable HyperThreading at the BIOS level. So your total CPU utilization was around 21% when using just the 12 physical cores?
At best...