Luminous Landscape Forum

Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Ghibby on August 01, 2016, 05:20:01 am

Title: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on August 01, 2016, 05:20:01 am
Hi All,

Just wondering how the EOS 5Ds and 5Dsr shooters are getting on with Lightroom CC.  Over the last year or so lightroom's glacial speed on these files has been driving me nuts.  I have it all set up to optimum levels for performance, catalogue and RAW cache are on fast SSD's, only the images themselves are on HDD's and even these are fast WD Black drives.  I have tried with GPU support on and off and in all honesty there is no major difference in performance. I have an Nvidia Quadro 4000 card at work and a GTX950 at home. My machine is fairly powerful (32gb Ram, hex core i7 at 3.2Ghz), I have a similar spec at my office and the problems are the same. 

That is slow previews in all modules, especially the 1:1 previes which can take 30sec and longer to generate per image, slow adjustments in develop module and the entire package just feels like its running through treacle.  When I process lower res files up to say 24mp all is fine but by the time you hit 40-50mp files the performance takes a massive step down.  Too much in my opinion.  To me it feels like the whole architecture of the software just can't handle these high res files. 

Be very interested to hear if any of you are experiencing similar issues with high res RAW files.

At the moment I am really hoping that one 1 software will deliver on their promise of super fast raw processing with photo raw when it lands in the autumn! Perhaps it will make Adobe address the woeful performance of lightroom at long last. 

Ben
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Rory on August 01, 2016, 09:52:40 am
What is the resolution of your monitor?  Have you tried running Lr in a smaller resolution window?  I'm not suggesting this is an acceptable solution, just trying to pin down where the performance issues are happening.  I have a similar system and it normally takes 5-6 seconds to render a D810 file on a 2560x1600 monitor.  When I reduce the window size so that the image pane is under 1000 pixels on a side I get much faster performance in both the develop and library modules.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on August 01, 2016, 10:40:32 am
Hi Rory,

I use a pair of screens, one is 2560x1440, the other 1920x1200. Same performance issue on both, I only use the big one for LR. I'll try a smaller window and see what happens ....

Interesting, performance is way better, image preview pane is apx 1400 x 950 px. I can do this by maximising the size of the browser strip at the bottom of the window. Thanks Rory!!

Just timed 1:1 preview generation to 11 seconds with EOS 5Ds raw files, while this is still poor at least now the slider adjustment is much less laggy. Much better.

The question now is what can be done to allow working with a larger preview pane.  I'd hate to see what happens with a 4k screen!
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Rory on August 01, 2016, 10:50:28 am
Glad that helps a bit.  Optimizing performance in an app like Lr is a tough proposition but I think there is still lots of room for improvement.  In the import module the rendering of the embedded JPEGs is poor.  Almost every other viewer on the market is faster.  The caching logic in the library module could use some TLC.  Adobe has been  putting some effort into improving the develop module and I'm reasonably happy with it.  I've written my own raw converter and iterating through 50,000,000 pixels with sophisticated algorithms takes time.  Showing an approximation in a 1,000,000 pixel window is much faster.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on August 01, 2016, 11:12:24 am
Hi again Rory,

It will be very interesting to see what ON1 come up with in Photo RAW.  The video previews that have been shown so far look very promising.  What amazes me is that a giant like Adobe has not ensured their program (which probably has more users than all other RAW converters combined) is not on top of performance with high res imaging.  At the end of the day with the R&D they have at their disposal and the sheer power of modern computers these kind of performance issues should not be issues we have to deal with.  I mean even the camera itself can zoom in on the embedded preview to its RAW file an order of magnitude faster than LR can.

Agree with you on the import rendering of embedded JPEG's too, again it is one of those things that really takes the shine of using LR.

I remember having similar issues with the 21mp images from my 5D2 back in 2008-2009 and it took a long time for LR 3 to be able to handle this with ease.  Hopefully the next update to LR will have some performance boost up its sleeve. Somehow I doubt it though!  Perhaps if ON1 hit a home run it will give them some impetus to do something.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Rory on August 01, 2016, 11:18:38 am
I have not looked at ON1 for a while.  I'm not defending Adobe, but fast previews of JPEGs is one thing and fast previews of RAW files with multiple rendering instructions is another.  There is a reason the camera manufacturers embed JPEGs. 
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Paul2660 on August 01, 2016, 12:38:18 pm
Welcome to the club, I have moaned about this type of performance for years.  The higher resolution screens, 30" or even 27 inch cause a lot of problems with LR on 100% views,  now add either around 10 adjustment brushes, or 20 dust removal and go have a cup of coffee.  Or use the auto mask, and then attempt to refine some of it's mistakes, which creates a ton of speed issues.  It can become so bad that even once zoomed to 100%, then attempting to move the image will either crash LR or takes 20 to 25 seconds to refresh, just a bit frustrating.  And minimizing the LR window to a smaller size on a 30" screen used to help, but no longer seems to make a bit of difference, so possibly things have gotten worse.  I have the same resolution on my 30" screen as the OP.

The use or non use of modern, fast, well designed video cards and open CL is basically a waste also.  LR/Adobe states only the develop module has been written to take advantage of open CL, but I have never seen any difference on or off.  Unlike Adobe's own software Photoshop, which has an excellent design around open CL, and you can clearly see a difference visually and in a speed bench. 

I believe some find the Problem is the same on the Mac, I have read enough similar complaints from large heavy lifting Mac users. 

I have also given up on LR fixing this issue and work accordingly, trying to keep adjustment brushes to a lower number and no spot removal, instead go to CC for that.  As all digital files get larger and larger, this problem will just get worse.  But I notice on really anything 24MP and up with the conditions I previously mentioned, which is tragic as LR has an excellent tool set, one of the best.

LR is much more processor based, and system ram, so best thing I have found to do is possibly overclock a bit, turn off open CL support in LR as it eventually causes more issues over time.  And reboot after about 1 hour 2 hours max as LR seems to build up a lag over time, not sure if it's memory leak or just coding. 

Both machines I have are win10, 32GB of ram i7 4 and 3.6GHz, both with nvidia GTX970 4GB cards. You can benchmark LR on any of several open CL tools and there is literally no increase is the cuda processing needs, memory use, or need to increase fan speed to to work, the card just does basically nothing.  Same testing Adobe CC and you can see quite a bit of activity on the card.

Paul C
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 01, 2016, 05:13:37 pm
I'm using Lightroom CC with a MacBook Pro 15" 2.8Ghz 16GB RAM and 1TB SSD. Some observations about performance. In my office I have a 4K display.

For folders where I actively edit pictures I have 1:1 previews generated and I have the setting of never discard the previews automatically in the catalog settings. When I no longer actively edit a folder og images I will discard the 1:1 previews to conserve space. The 1:1 previews are only used in the Library module so therefore I will always browse pictures and check focus, composition etc. in the Library module. In the library module the navigation is instant and zooming in is also instant. This is using the external 4K monitor. Lightroom is for my use always fully maximized on the screen and in full screen mode.

So the performance is really good in the Library module as long as 1:1 previews are generated and with 4K screens.

In the Develop module it is another story. Turning off the graphics card is a disaster. If I do that the performance comes to a complete crawl. With the graphics card turned on and zoomed in to 1:1 and the details have been rendered, I can pan across the image very smooth. Zooming in to 1:1 goes quick but the details are not fully rendered and it sometimes take 5-10 seconds before they are. I always have lens correction turned on and for edited pictures typically a lot of corrections as well as local edits. This clearly slows down performance. I have noticed a serious performance degradation after having edited for a while and this only goes away if I restart Lightroom. This is quick and not a big deal, but like the bugs to fixed around this. Moving sliders often is slow to render the changes.

It is absolutely key to switch between the Library module and the Develop module and only stay in the Develop module to edit a specific picture and move back Library no navigate across images and zoom to 1:1 to check images.

I have previously posted a thread on the issues with 4K monitors and scaling on Macs. For my normal editing I use the scaling "looks like" 2560x1440 which is not ideal. Without having solid measurements it feels slightly more responsive if I use "looks like" 1920x1080 as the intermediate scaling before mapped to the 4K screen is much less in this case (Full HD scaling maps exactly to 4K the 2x Full HD resolution). Using the retina screen (2880x1800) on the MBP only Lightroom is considerably more responsive in the Develop module.

The observations above is editing files from the Canon 5DSR. Nikon D810 files are faster to edit. Since I'm not using windows I cannot speak about this, but I suspect that my findings should map to Windows as well depending on HW used.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: john beardsworth on August 02, 2016, 03:48:06 am
It is absolutely key to switch between the Library module and the Develop module and only stay in the Develop module to edit a specific picture and move back Library no navigate across images and zoom to 1:1 to check images.

While I agree with your general points about previews, Hans, this seems too hard-line - and in any case you may want to review it after the change introduced in 6.6 (and corrected in 6.6.1). Develop now caches 2 extra images in either direction, making it much more viable to navigate in Develop. The benefits were dramatic with 6.6 on my underpowered Mac Air, and the 6.6.1 update extended them to my Windows desktop (6.6.1 was because the change screwed high RAM machines).

John
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 02, 2016, 08:25:29 am
While I agree with your general points about previews, Hans, this seems too hard-line - and in any case you may want to review it after the change introduced in 6.6 (and corrected in 6.6.1). Develop now caches 2 extra images in either direction, making it much more viable to navigate in Develop. The benefits were dramatic with 6.6 on my underpowered Mac Air, and the 6.6.1 update extended them to my Windows desktop (6.6.1 was because the change screwed high RAM machines).

John

Although I see improvements in the Develop module in moving from one image to the next, I still find it slow to do that and zoom in and out when checking images. I think it is nice to have the improvement when you go to the next image in order to edit it and just do that by the arrow keys left or right while in the Develop module. But reviewing images and checking for sharpness etc. I find it better to stay in the Library module and there select the pictures to work on in the Develop module. This is basically how I work and I'm not sure this came across in what I wrote. I just suspect there are Lightroom users who are not aware of the 1:1 preview role and therefore do not go back and forth between the two modules when it is advantageous. The lag in the Develop module to render the details is annoying and I don't expect this to go away. In fact I'm not that happy about Lightroom working on the next and previous image and cache it. I least for how I work this just slows down my machine and makes the fan go more on my MBP. At least I would wish there was a setting in preferences where I could turn this off.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on August 02, 2016, 12:08:42 pm
For me one of the biggest issues is with sharpening the image, you need to be at 100% and panning around an image checking details even when the 1:1 preview is built in the develop module is complete pain in the arse, slow to redraw, fuzzy as the preview catches up the viewing area etc.  Don't even get me started on the issues if you want to do A/B comparison checks on a pair of image to evaluate effects of sharpening, noise reduction and even halos caused in skies by the clarity slider (even at low settings occasionally).  These are all serious flaws that should have been ironed out of what is a mature product now.

While I strongly feel something needs to be done, I cant imagine anything will for a long time yet. We certainly should not be making excuses for Adobe or letting them off the hook for these kind of issues. I mean moving modules just to select images, its a joke! This is basic stuff!  Clearly the comparative lack of competition in this type of software is hampering development of its performance, Adobe have got lazy and happy to just to rake in the profits of those monthly subscriptions.  Innovation and thus these performance issues will continue to plague lightroom until another few companies products outperform it and start to erode sales.

I have said it already in some earlier posts but I really can't wait to have a play with On1 PhotoRAW on its release.  If they deliver what they promise they will then I will certainly give it a long term trial to see if there is a viable alternative to LR for my needs.


Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: dwswager on August 02, 2016, 01:31:47 pm
Hi All,

Just wondering how the EOS 5Ds and 5Dsr shooters are getting on with Lightroom CC.  Over the last year or so lightroom's glacial speed on these files has been driving me nuts.  I have it all set up to optimum levels for performance, catalogue and RAW cache are on fast SSD's, only the images themselves are on HDD's and even these are fast WD Black drives.  I have tried with GPU support on and off and in all honesty there is no major difference in performance. I have an Nvidia Quadro 4000 card at work and a GTX950 at home. My machine is fairly powerful (32gb Ram, hex core i7 at 3.2Ghz), I have a similar spec at my office and the problems are the same. 

That is slow previews in all modules, especially the 1:1 previes which can take 30sec and longer to generate per image, slow adjustments in develop module and the entire package just feels like its running through treacle.  When I process lower res files up to say 24mp all is fine but by the time you hit 40-50mp files the performance takes a massive step down.  Too much in my opinion.  To me it feels like the whole architecture of the software just can't handle these high res files. 

Be very interested to hear if any of you are experiencing similar issues with high res RAW files.

At the moment I am really hoping that one 1 software will deliver on their promise of super fast raw processing with photo raw when it lands in the autumn! Perhaps it will make Adobe address the woeful performance of lightroom at long last. 

Ben

This is why I haven't given up Br/ACR.  It's not that the previews and things are that much faster, but the interface does not lag as bad while it is generating previews.  There is functionality of Lr I would (and sometimes do) take advantage of, but if I had to do everything in Lr, I would blow my brains out over the productivity hit.

Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 03, 2016, 10:37:18 am
For me one of the biggest issues is with sharpening the image, you need to be at 100% and panning around an image checking details even when the 1:1 preview is built in the develop module is complete pain in the arse, slow to redraw, fuzzy as the preview catches up the viewing area etc.  Don't even get me started on the issues if you want to do A/B comparison checks on a pair of image to evaluate effects of sharpening, noise reduction and even halos caused in skies by the clarity slider (even at low settings occasionally).  These are all serious flaws that should have been ironed out of what is a mature product now.

While I strongly feel something needs to be done, I cant imagine anything will for a long time yet. We certainly should not be making excuses for Adobe or letting them off the hook for these kind of issues. I mean moving modules just to select images, its a joke! This is basic stuff!  Clearly the comparative lack of competition in this type of software is hampering development of its performance, Adobe have got lazy and happy to just to rake in the profits of those monthly subscriptions.  Innovation and thus these performance issues will continue to plague lightroom until another few companies products outperform it and start to erode sales.

I have said it already in some earlier posts but I really can't wait to have a play with On1 PhotoRAW on its release.  If they deliver what they promise they will then I will certainly give it a long term trial to see if there is a viable alternative to LR for my needs.

I agree that Adobe should continue to improve Lightroom and some more competition would be good. I have previously posted about the lack of innovation and improvements in Lightroom except for the Develop module and what else comes from the Camera Raw side. But I have to say that in the Library modul with 1:1 previews generated, the performance is smooth on my MBP and zooming in to 1:1 is instant and panning is also smooth and instant in 1:1 there is no redraw lag.  Remember when an image has been edited the 1:1 goes away. I'm sure you know this but I also suspect many others are not aware of the role of the 1:1 previews. In the Develop module there is a lag when reviewing at 1:1 for sharpness and panning across the image. Back when I used windows back in 2009 and changed to the Mac I noticed the Mac was much faster in the Library module despite the Windows hardware was faster than the MacBook Pro dual core at the time. The comparison was on a 30" 2560x1600 display. I have not used Windows since the so can't comment on wether this difference is still there but it sounds like is.

Despite the things I'm not quite happy with and what I think could be improved, there is for me no alternative to Lightroom.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 03, 2016, 10:40:54 am
This is why I haven't given up Br/ACR.  It's not that the previews and things are that much faster, but the interface does not lag as bad while it is generating previews.  There is functionality of Lr I would (and sometimes do) take advantage of, but if I had to do everything in Lr, I would blow my brains out over the productivity hit.

I avoid using the machine while doing 1:1 preview generation mostly. I generate 1:1 previews during import and do something else until it is completed. That fits my workflow as most of my shooting is on my trips and I import pictures for each shoot which is typically a few hundred files. As mentioned in my previous post I find Lightroom in the Library module fast and not lagging at all on my MacBook Pro.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on August 10, 2016, 05:22:01 am
I have made significant improvements to the performance issues by optimising further and adding a further SSD. 

What I have done is to add a second SSD that only contains LR catalogues & previews as well as the camera raw cache which have set to 32.0GB.  I have also set this disc up as my PS scratch disc as well as the Windows page file location, (with Win 7 and 32.0gb Ram or more the page file is not exactly heavily used assuming you are not running lots of background processes).  The key here is that the operating system & programs are isolated on one disc (also SSD) from the LR Catalogues and camera RAW cache. 

In setting up the additional SSD I also purged the camera raw cache but left all my LR previews alone, I simply moved them to their new disc locations. Also updated my graphic card drivers to the very latest ones.

The performance has been considerably improved with fairly minimal expenditure.  What I am finding on my main system is that in the library module even with a second monitor set to loupe that browsing is much faster than it was previously, in fact I am seeing practically no negative impact on browsing speed on my main monitor (2560x1440) while the second monitor (1920x1200) is displayed, great for checking critical focus.

In develop module the speed switching between images is much faster and I can make all of the major slider adjustments in the basic, details, lens corrections & transform panels with comparatively minimal lag to the display.  There are still issues with the local adjustments and spot heal / clone which as far as I know are not supported by the GPU acceleration in LR as yet, much like the library module

Other performance issues are that I do see a general slow down after about 60mins of use, especially if I don’t switch modules fairly regularly.  Adobe still has a long way to go to improve the overall performance of LR but for now I can live with it a lot more comfortably than just a week ago.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: JRSmit on August 10, 2016, 11:34:28 am
Putting catalog with previews and acr cache on a separate ssd is what I also have done.
Make sure your disks, also the ssd's , keep at least 30% free or thereabout.  Perf drops rapidly if filled more.
Slow down after 60minutes can well be the ssd doing its garbage clean up.  That is why i use the samsumg sm863 (an datacenter drive). Does not have this cliff in the performance.
I also experience sometimes slow response because my HDD have fallen "asleep". Takes several seconds to wake up.
I use windows 10.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: dwswager on August 10, 2016, 01:53:44 pm
Since I use both ACR with Br and LR, I can tell you no amount of hardware can fix the problem.  They can help make Lr less slow than it would be otherwise.  There are 2 issues with Lr performance.  The first is just poor programming.  The second, however, is fundamental to Lr.  Because it's very nature, Lr must do reads and writes on a continuous basis to a structured database.  In Develop, for example, while ACR reads and writes once (after you click done or open), Lr is continuously updating.  This is where lightning fast SSDs help with Lr beyond what they would with ACR.

One performance tip is not to have it write to the sidecar xmp files. The faster way is to turn the catalog writing off and have it only write out once to the XMP file.  Of course, that negates the whole design.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: john beardsworth on August 10, 2016, 02:17:09 pm
One performance tip is not to have it write to the sidecar xmp files. The faster way is to turn the catalog writing off and have it only write out once to the XMP file.  Of course, that negates the whole design.

On the contrary, saving to xmp is almost irrelevant to the design. It's there for data exchange with other apps. Once all xmp is written, performance reverts to normal - if it is at all impacted by writing the xmp (yes, it was once).

Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 10, 2016, 03:46:10 pm
Since I use both ACR with Br and LR, I can tell you no amount of hardware can fix the problem.  They can help make Lr less slow than it would be otherwise.  There are 2 issues with Lr performance.  The first is just poor programming.  The second, however, is fundamental to Lr.  Because it's very nature, Lr must do reads and writes on a continuous basis to a structured database.  In Develop, for example, while ACR reads and writes once (after you click done or open), Lr is continuously updating.  This is where lightning fast SSDs help with Lr beyond what they would with ACR.

One performance tip is not to have it write to the sidecar xmp files. The faster way is to turn the catalog writing off and have it only write out once to the XMP file.  Of course, that negates the whole design.

I don't see much I/O going on while in the Develop module. This module is CPU bound.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Denis de Gannes on August 10, 2016, 06:42:34 pm
On the contrary, saving to xmp is almost irrelevant to the design. It's there for data exchange with other apps. Once all xmp is written, performance reverts to normal - if it is at all impacted by writing the xmp (yes, it was once).

Agreed however if you are using Lightroom for file management and processing of raw data and Photoshop / Photoshop Elements for further editing there is really no need to write to xmp.

After your editing and other file management options in Lightroom you use the edit in function in Lightroom to send the file to Photoshop (or another photo editing application) for further editing if needed. This function does not require you to write to xmp. Also no other non Adobe application can utilise the Develop Module edits from the xmp file.

To wit there is life without the use of xmp files and this is the main reason that I adopted Lightroom as the application for the processing of my raw files. Since I have been using Lightroom (2007) I have not used xmp files (other than to share with other users or the transfer data after I have returned from an external trip.). My workflow is "simple sweet" the metadata info is in the Lightroom Catalog and I do not have the need to manage xmp files.,     
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Denis de Gannes on August 10, 2016, 07:12:43 pm
Since I use both ACR with Br and LR, I can tell you no amount of hardware can fix the problem.  They can help make Lr less slow than it would be otherwise.  There are 2 issues with Lr performance.  The first is just poor programming.  The second, however, is fundamental to Lr.  Because it's very nature, Lr must do reads and writes on a continuous basis to a structured database.  In Develop, for example, while ACR reads and writes once (after you click done or open), Lr is continuously updating.  This is where lightning fast SSDs help with Lr beyond what they would with ACR.

One performance tip is not to have it write to the sidecar xmp files. The faster way is to turn the catalog writing off and have it only write out once to the XMP file.  Of course, that negates the whole design.
There is no option in Lightroom to "turn the Catalog writing off", Lightroom by design reads and writes to the catalog file. It only automatically reads from an xmp file at Import, if info is added to the xmp file after import then the user will have to manually instruct Lightroom to update the info in the xmp file to the Catalog File.
By design Lightroom operates on the premise that the info is in the Catalog, while Adobe Camera Raw / Bridge only works from the info in the file (xmp sidecar or in the file header).
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: dwswager on August 10, 2016, 08:25:02 pm
There is no option in Lightroom to "turn the Catalog writing off", Lightroom by design reads and writes to the catalog file. It only automatically reads from an xmp file at Import, if info is added to the xmp file after import then the user will have to manually instruct Lightroom to update the info in the xmp file to the Catalog File.
By design Lightroom operates on the premise that the info is in the Catalog, while Adobe Camera Raw / Bridge only works from the info in the file (xmp sidecar or in the file header).

Yes, that was my point.  It would negate the whole design of the program.  But constantly reading and writing to a database is much slower than a one time write to an xmp file.  However, if there appears significant opportunity to tighten up the code to improve performance.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Denis de Gannes on August 10, 2016, 08:52:18 pm
Yes, that was my point.  It would negate the whole design of the program.  But constantly reading and writing to a database is much slower than a one time write to an xmp file.  However, if there appears significant opportunity to tighten up the code to improve performance.

Is this really true ACR is retaining the edits in memory as you work while Lightroom is writing to the Catalog as you perform a function. The point in an earlier post is that Automatically write to xmp is creating an additional function for Lightroom and this must be creating a lag, whether this is significant or not is the issue. How fast these processes occure will be dictated by the weakest link, internal HDD vs external HDD via USB connection and or Internal SSD vs external SSD via USB connection.
Availability of free space on the respective drives also affect the performance.     
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 11, 2016, 10:45:50 am
Yes, that was my point.  It would negate the whole design of the program.  But constantly reading and writing to a database is much slower than a one time write to an xmp file.  However, if there appears significant opportunity to tighten up the code to improve performance.

The writes to the catalog in the Develop module is very little as would be expected with a SQL database. This is not the bottleneck in Lightroom as I mentioned before.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 11, 2016, 11:33:11 am
Agreed however if you are using Lightroom for file management and processing of raw data and Photoshop / Photoshop Elements for further editing there is really no need to write to xmp.

After your editing and other file management options in Lightroom you use the edit in function in Lightroom to send the file to Photoshop (or another photo editing application) for further editing if needed. This function does not require you to write to xmp. Also no other non Adobe application can utilise the Develop Module edits from the xmp file.

To wit there is life without the use of xmp files and this is the main reason that I adopted Lightroom as the application for the processing of my raw files. Since I have been using Lightroom (2007) I have not used xmp files (other than to share with other users or the transfer data after I have returned from an external trip.). My workflow is "simple sweet" the metadata info is in the Lightroom Catalog and I do not have the need to manage xmp files.,   

Correct and the only reason for having xmp write on is for recovery after a catalog corruption. Doing regular catalog backups is a good thing to do if you check the option for Lightroom to check the integrity of the catalog. There is only a need to keep the most recent catalog backup. If a catalog gets corrupted after a catalog backup the recovery process could look like this:

1) Copy the Lightroom catalog backup into the location where the catalog is normally stored for active use. In the same folder as where the previews are stored. This is now the active catalog.
2) For any new folders created in imported into Lightroom after the catalog backup need to be imported again into the active catalog (which is the one created in step 1 above). Lightroom will read all xmp files in the folder and copy the information into the active catalog.
3) For any folders where new files has been added, do a synchronize folder for each of them and import the files that were not in the catalog.
4) For any files that has been edited since the old catalog backup will have an ! mark in the thumbnail and you can click it and choose read from file.

Fortunately I never had to do this, but I do have xmp write turned on for that reason. Of course any virtual copies and collection updates are lost since the catalog backup was done. The performance impact on having xmp write on is very little.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: dwswager on August 12, 2016, 09:20:13 am
Correct and the only reason for having xmp write on is for recovery after a catalog corruption. Doing regular catalog backups is a good thing to do if you check the option for Lightroom to check the integrity of the catalog. There is only a need to keep the most recent catalog backup. If a catalog gets corrupted after a catalog backup the recovery process could look like this:


The other reason is so that Lr and Br/ACR can sync with each other.   Br/ACR do not have access to the Lr catalog data about images and as such the XMP data is the common link.  And as Br/ACR do not suffer from near the performance issues that Lr does, there are just some tasks much more efficient in Br/ACR than Lr.  This also applies to any other program that can read the XMP file, now or in the future.

BTW, it is not only the SQL writes that cause issues.  It appears that Lr, reads slider position movements and hesitations much more literally than ACR.  As such, I find that it is much quicker to trigger screen updates to match the new slider positions than ACR.  The good is that on smaller images with a quick system, it is much more "interactive" than ACR.  On the downside with large images, on a 4K screen this triggers screen updating even when you just hesitate the slide and this really slows down.  ACR seems to take the stance "make up your mind and then I'll update"  while Lr seems to try to keep up with you.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: john beardsworth on August 12, 2016, 11:25:45 am
Exchange of data with programs like Bridge is not the other reason for having xmp writing enabled - it is the reason. Backup isn't, since xmp fails to include a swathe of the information and work that you do in Lightroom. Corrupt catalogues are very rare, and the safeguard is to routinely backup your catalogue.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: JRSmit on August 12, 2016, 12:12:22 pm
Still have to experience e a corrupt catalog in LR and i use it every day.
Ayear or so I did an inquiry on who had experience ed a corrupt catalog. Turned out that either no one dared to mention it or it simply does not occur.
Also theLR catalog ba kup is just that only the database.  Not your previews or image files.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: donbga on August 12, 2016, 09:48:11 pm

At the moment I am really hoping that one 1 software will deliver on their promise of super fast raw processing with photo raw when it lands in the autumn! Perhaps it will make Adobe address the woeful performance of lightroom at long last. 

Ben

Have you considered Capture One as an alternative. It should be a an excellent alternative.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: dwswager on August 13, 2016, 11:53:39 am
Have you considered Capture One as an alternative. It should be a an excellent alternative.

Capture One is a great alternative to the Develop Module in Lr, but not for Lr.  In fact, a great alternative to the develop module is ACR!!!  All the tools, much less of the performance woes.  Best part is that Lr will respect and reflect the effort much more readily than any other RAW converter.  This is really the reason I have used other converters for special applications, but have never switch wholesale.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 13, 2016, 12:27:41 pm
The other reason is so that Lr and Br/ACR can sync with each other.   Br/ACR do not have access to the Lr catalog data about images and as such the XMP data is the common link.  And as Br/ACR do not suffer from near the performance issues that Lr does, there are just some tasks much more efficient in Br/ACR than Lr.  This also applies to any other program that can read the XMP file, now or in the future.

BTW, it is not only the SQL writes that cause issues.  It appears that Lr, reads slider position movements and hesitations much more literally than ACR.  As such, I find that it is much quicker to trigger screen updates to match the new slider positions than ACR.  The good is that on smaller images with a quick system, it is much more "interactive" than ACR.  On the downside with large images, on a 4K screen this triggers screen updating even when you just hesitate the slide and this really slows down.  ACR seems to take the stance "make up your mind and then I'll update"  while Lr seems to try to keep up with you.

I normally do not use ACR, but tried it and on my MacBook Pro I do not feel any difference in speed.

You continue to mention at there is an issue with the SQL write and reads, but there isn't! Run the performance monitor side by side running Lightroom and you will see there is not much I/O going on. You can also measure it. I cannot imagine that there is a difference between Windows and Mac regarding this. If you think about what Lightroom need to do with the database and you know something ab ut databases you should convince yourself that this is not where the issue is.

I continue to see issues with how the GPU is used. When I drag a slider I often see a glimpse of a previous version of the image flashing for a short moment. E.g. when converting a virtual copy to b&w I see the color version flashing for a short moment when dragging a slider. There is no doubt differences between the Mac and Windows in how the graphics is mapped. I do not know any of the details in how this is architected and coded for obvious reasons. But I would agree that Adobe has a serious amount of work to do in this area to improve performance.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 13, 2016, 12:31:50 pm
Exchange of data with programs like Bridge is not the other reason for having xmp writing enabled - it is the reason. Backup isn't, since xmp fails to include a swathe of the information and work that you do in Lightroom. Corrupt catalogues are very rare, and the safeguard is to routinely backup your catalogue.

As mentioned this is the reason I have xmp write enabled, so that in case a catalog gets corrupted I can save most of my image edits done since the last backup. Given that I never had a corrupt of lost catalog one can argue it is not needed. But since it does not cost anything really I still have it on. I the beginning using Lightroom I still used Bridge and Photoshop from Bridge and I need the xmp's for some automated scripts. It is, of course, the reason for why Adobe built the xmp write feature.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: dwswager on August 13, 2016, 12:49:06 pm
I normally do not use ACR, but tried it and on my MacBook Pro I do not feel any difference in speed.

You continue to mention at there is an issue with the SQL write and reads, but there isn't! Run the performance monitor side by side running Lightroom and you will see there is not much I/O going on. You can also measure it. I cannot imagine that there is a difference between Windows and Mac regarding this. If you think about what Lightroom need to do with the database and you know something ab ut databases you should convince yourself that this is not where the issue is.

I continue to see issues with how the GPU is used. When I drag a slider I often see a glimpse of a previous version of the image flashing for a short moment. E.g. when converting a virtual copy to b&w I see the color version flashing for a short moment when dragging a slider. There is no doubt differences between the Mac and Windows in how the graphics is mapped. I do not know any of the details in how this is architected and coded for obvious reasons. But I would agree that Adobe has a serious amount of work to do in this area to improve performance.

I'm on Windows so maybe there are some differences between IOS and Windows.  All I know is that when in develop module in LR, the sliders have significant hesitation some times.  I also have to wait some times for the scroll of the tools on the right side of the develop module window.  In ACR,  I can't say I ever get that type of slow down.  Bottom line, for develop, I am much more efficient in ACR than Lr.  Just last night, while developing some soccer game images, about half way through them, I jumped out of Lr and into Br to finish with ACR.  I have to admit that I much prefer the tabbed tools in ACR rather than scrolling forever to get from one end to the other of the tool bar in Lr.  And collapsing and opening the tools in Lr is no solution because it adds much additional time. From a Human Engineering perspective, it is much better to have the tools always in the same exact location than constantly moving around as Lr does it.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: john beardsworth on August 13, 2016, 01:10:08 pm
I have to admit that I much prefer the tabbed tools in ACR rather than scrolling forever to get from one end to the other of the tool bar in Lr.  And collapsing and opening the tools in Lr is no solution because it adds much additional time.

So right click a panel header and choose Solo Mode. This means that as soon as you click a panel header, that panel opens and other panels are closed. No more scrolling, no more collapsing and opening.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Tim Lookingbill on August 13, 2016, 03:21:16 pm
Actually my first reaction to John's suggestion was to complain about having to scroll down to get to the rest of the collapsed panels until I started clicking on each panel bar's arrow.

Great tip, John!

I just wished I would've found it reading LR4's User's Guide but since this app is a clutter of nested features and options I forgot about right click on every freakin' geometrically shaped interface object spread out across LR's interface design.

See! I complimented John at the same time complained! It's a first! Now that's multitasking!
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: john beardsworth on August 13, 2016, 03:56:26 pm
I'm not sure any manual can explain this feature in a way that makes its purpose obvious, Tim. My initial reaction would probably have been "Solo Mode, huh?"

By the way, you don't even need to click the arrow - anywhere in the panel header will do fine.

Unfortunately enabling Solo Mode is on a module by module basis, so if you want it in Library you'll have to set it there in both the left and right sides in Library too.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Tim Lookingbill on August 13, 2016, 04:51:25 pm
Don't need it in Library which I only use to view thumbnails.

Thanks again for the Solo Mode tip.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: donbga on August 13, 2016, 10:30:26 pm
I'm on Windows so maybe there are some differences between IOS and Windows.  All I know is that when in develop module in LR, the sliders have significant hesitation some times.  I also have to wait some times for the scroll of the tools on the right side of the develop module window.  In ACR,  I can't say I ever get that type of slow down.  Bottom line, for develop, I am much more efficient in ACR than Lr.  Just last night, while developing some soccer game images, about half way through them, I jumped out of Lr and into Br to finish with ACR.  I have to admit that I much prefer the tabbed tools in ACR rather than scrolling forever to get from one end to the other of the tool bar in Lr.  And collapsing and opening the tools in Lr is no solution because it adds much additional time. From a Human Engineering perspective, it is much better to have the tools always in the same exact location than constantly moving around as Lr does it.
Your conclusions about your experienced differences of performance betweeqn ACR and LR seem to me to be at best anecdotal.


Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: cortlander on August 15, 2016, 11:06:29 am
Have you considered Capture One as an alternative. It should be a an excellent alternative.

+1
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Hans Kruse on August 16, 2016, 02:50:29 am
I'm on Windows so maybe there are some differences between IOS and Windows.  All I know is that when in develop module in LR, the sliders have significant hesitation some times.  I also have to wait some times for the scroll of the tools on the right side of the develop module window.  In ACR,  I can't say I ever get that type of slow down.  Bottom line, for develop, I am much more efficient in ACR than Lr.  Just last night, while developing some soccer game images, about half way through them, I jumped out of Lr and into Br to finish with ACR.  I have to admit that I much prefer the tabbed tools in ACR rather than scrolling forever to get from one end to the other of the tool bar in Lr.  And collapsing and opening the tools in Lr is no solution because it adds much additional time. From a Human Engineering perspective, it is much better to have the tools always in the same exact location than constantly moving around as Lr does it.

The Mac runs OX X and from now on Mac OS, not IOS which is only for mobile Apple devices like Iphone and Ipad.

Regarding the develop panel compared to a tab'ed view. As mentioned Solo View is a solution but I find it much better to only have the sections open that I always use, which is: Basic, tone curve and effects. I can see them all on my big screen without scrolling. For the other panels I will click on the bar for the panel which will open it and then do the adjustment and then click on the bar an close it again. I find this even more convenient than a Tab'ed view. Just my opinion :) On my MacBook I need to scroll a little more and with two fingers in the trackpad I can do this very fast.

Notice, I'm not trying to say you are wrong or you should prefer Lightroom :) Just a hint if that makes a difference for you or somebody else from someone who uses Lightroom professionally and for years.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on August 16, 2016, 08:07:59 am
Have you considered Capture One as an alternative. It should be a an excellent alternative.

I have tried C1 and gave it a full 30 day trial. For my work and quantity of images I found it to be slower then LR, especially with export of finished images to JPEG etc, it also takes its time to render 1:1 images at 50mp from the Canon EOS 5Ds.  Undoubtedly C1 has some major positives with some tools, control of colour was very good, as is sharpening and noise reduction which are probably a bit better than LR, that being said I missed the lack of custom camera profiles as per LR and ACR.  In doing a whole set of comparisons with the same set of images I found that while I was able to get nice images out of C1 they slightly different interpretations than the LR versions, not better.

I have used LR for a long time and have a deep understanding of the software, using C1 was interesting but ultimately a less cohesive package for me than LR so I choose to stick with Adobe for now.

Still looking forward to On1 photo raw though! Hopefully another viable alternative in this market!
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Rendezvous on August 18, 2016, 12:19:14 am
I noticed I get significantly better performance if I (on mac) unmount the drive containing the photos, and edit using the smart previews. Things are a lot more snappy, but of course this solution isn't suitable for 1:1 sharpening etc.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: bbrantley on August 28, 2016, 11:46:33 pm
To the people who are seeing good Library mode 1:1 performance:  not on my 5k iMac, at least on MacOS.  There is still a painful lag.  Smart preview is 2880px wide, and a full rendering has to occur for each image.  Why the software can't precompute some more of this is a mystery to me.
Title: Re: Woeful Lightroom performance with EOS 5Ds
Post by: Ghibby on April 11, 2017, 10:05:02 am
Been a while since this topic has been posted on but though it would be of interest to some readers.   I have seen a continued improvement in the performance of LR, the use of smart previews to speed up develop module has been great. Its still a bit clunky when zoomed in but overall performance is acceptable with the 5Ds.  I have recently tried On1 Photo 2017 to see what it can offer…  Not ready yet IMHO. It looks promising but it makes Lightroom look fast, it is so slow on export and switching between modules, not useable for a professional edit yet.  Image quality is again different to Lightroom and while it is better at noise reduction due to the control on offer with shadow, mids or highlight protection it does not seem to be able to extract the same level of finesse and detail from the image as LR can.  Some of this is perhaps due to my comparative inexperience relative to LR but the speed of operation and basic set up is too painful for me to continue evaluating for now.  At the moment Adobe keeps my custom as I feel it offers the best all round package, including sheer performance (which it pains me to say in light of the original subject matter for this post) for my needs.  Still hoping for a big performance boost from LR in the future though.

Ben