Luminous Landscape Forum

Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Chris Kern on May 23, 2014, 07:00:11 pm

Title: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 23, 2014, 07:00:11 pm
I've been playing around with DxO Optics Pro 9.5's new trick: the ability to import a raw file from Lightroom and return a linear DNG.  If I understand this properly (as usual, the explanatory material on the DxO website is less than pellucid), by emitting a DNG, DxO Optics Pro allows a Lightroom user to take advantage of DxO's sophisticated optical testing and correction technology while still retaining all the functionality of LR—except, of course, Adobe's demosaicing algorithms.  For the moment, at least, I'm only interested in DxO's automated sensor-and-lens corrections, with all additional processing performed in LR.

I'm encouraged by the initial results.  I've attached a small center crop from a much larger image made with a Nikon D800E and Nikon's 24-120mm f/4 zoom.  The first attachment is a 1:1 JPEG of the raw file as demosaiced by LR after applying LR's automatic lens corrections.  The second attachment is the same raw file demosaiced by DxO after applying DxO's automatic corrections for the sensor and lens.  I didn't do any manual sharpening or make any other corrections with either tool since my objective is to compare how the two perform the raw conversion and correct for known optical imperfections in the capture.  I've only worked on a few images so far, and only those shot with this particular sensor-lens combination, so it's too soon to come to any definitive judgment.  But in every test I've made, the DxO image looked better than the LR equivalent—sometimes dramatically so.

My only concern is that I may be giving up some of Lightroom's functionality by having DxO perform the conversion and turn over the linear DNG to LR.  It appears LR can still properly modify the white balance of the DxO-emitted DNG.  Is there anything else I might be giving up, or should otherwise be concerned about?
Title: Re: Lightroom-DxO Round-Trip
Post by: Steve House on May 23, 2014, 07:25:55 pm
I too have been curious about this since 9.5 was announced.  Consider the workflow:  Shoot RAW (NEF).  Import into LR, converting to DNG on import. Export DNG to DxO and process.  Re-import DNG from DxO back to LR.  Export from LR to image file (JPG, TIFF, etc) or print directly from LR.  What engine is actually doing the demosaicing, Adobe's, DxO's, an additive combination of both?  I've always thought the DNG was itself a RAW file format, with the colour conversions only getting 'baked in' when a true image file was created from it or it was printed.  That would seem to suggest such a workflow would be using DxO's optical corrections but Adobe's colour processing.  Anyone really know???
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 23, 2014, 07:28:51 pm
In a nutshell, the Linear DNG isn't raw (it's far less raw). It's rendered but in a linear space. Could have been a TIFF too, don't let the DNG part out give you the impression it's the same as DNG in from Lightroom.
Title: Re: Lightroom-DxO Round-Trip
Post by: Steve House on May 23, 2014, 07:37:50 pm
So what happens if you do additional processing in LR after bringing it back from DxO (compared to never sending the DNG to DxO in the first place)?
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 23, 2014, 07:39:22 pm
In a nutshell, the Linear DNG isn't raw (it's far less raw). It's rendered but in a linear space. Could have been a TIFF too, don't let the DNG part out give you the impression it's the same as DNG in from Lightroom.

Understood, Andrew.  I'm aware the file is "far less raw."  But my understanding has been that, for example, a TIFF has the color balance baked-in, but that a linear DNG can still be color-corrected.  (And, indeed, I can't detect any difference when I fiddle with the color sliders in LR.)  Am I missing something here?  And even if I'm not, is there something else, perhaps unrelated to color, that I'm missing?  I don't understand enough about the different semantics of the TIFF and DNG file formats to know how to perform definitive controlled tests.
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 24, 2014, 11:42:36 am
I'd think of the linear DNG just as I would a linear TIFF as AFAIK, the data would be the same (high bit, linear). Different format container, still baked and rendered, less so of course than a JPEG from the camera or a TIFF processed by you in any other raw converter. I don't see how the baked linear DNG would provide any major capabilities over the same data in a TIFF.
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 24, 2014, 12:21:37 pm
I'd think of the linear DNG just as I would a linear TIFF as AFAIK, the data would be the same (high bit, linear). Different format container, still baked and rendered, less so of course than a JPEG from the camera or a TIFF processed by you in any other raw converter. I don't see how the baked linear DNG would provide any major capabilities over the same data in a TIFF.

Sigh.  I'm still struggling to understand this—and why, other than marketing hype, DxO is making such a big deal about the ability of the latest rev of Optics Pro to pass a linear DNG back to Lightroom.  (Of course, it may indeed just be marketing hype.)

I just did a web search, and found a couple of explanations by Eric Chan which I think may be relevant to this discussion.

From a 2011 discussion:

Quote
DNG is indeed very much like TIFF.  It’s actually a set of TIFF extensions, with tags to describe things like white balance, color profiles, and various calibration data needed for processing raw images (as opposed to already-rendered images like regular jpegs and tiffs). . . .

At its most essential level, DNG is just a container of image data and its associated metadata.  It can hold raw image data (a.k.a., scene-referred data), and it can also hold rendered image data (a.k.a., output-referred data).  For raw data, the data can be in the mosaic form, or it can be demosaiced (so-called “linear DNG”).

(Source: http://www.natcoalson.com/blog/2011/11/29/my-adobe-dng-chat-with-eric-chan/)

And from a 2008 post by Eric on the DPReview website:

Quote
A usual TIFF file that comes out the back end of a raw converter has already been rendered, i.e., it has been mapped to a standard color space, it has been tone mapped, white balancing has been done, etc. More technically, the image is output-referred.

In contrast, the linear DNG is still scene-referred and can still benefit from many of the operations typically performed by a raw converter, such as white balance, the application of a camera color profile, HDR compositing, etc.

So the underlying internal file format looks similar, but the actual image contents and the types of operations that can be applied to that image are quite different.

(Source: http://www.dpreview.com/forums/post/30148341)

Now I think what these mean is that while both TIFF and DNG are just containers, one thing that distinguishes a linear DNG from a TIFF is that it contains additional metadata which allow some processing that is usually only performed during the raw conversion process.  Does that sound right?  If so, it might account for the claims DxO is making—and give them some credence.
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 24, 2014, 01:10:02 pm
Metadata but more so, less processing hence the idea (and that's up to debate) that this is scene referred not output referred data. See:
http://www.color.org/ICC_white_paper_20_Digital_photography_color_management_basics.pdf
It's rendered so not raw, but minimally processed. It's certainly more scene referred than output referred. Some operations may be easier to conduct, white balance comes to mind. But it isn't raw and that's the big deal IMHO. Don't let the DNG part of the name give you an idea it's a digital negative as we think of a raw neg. You've baked those pixels but to a lesser degree than using other tools on the data.
Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 24, 2014, 01:28:23 pm
Anyone think there's a good reason why they don't return a proper DNG with their processing instructions added as metadata?
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 24, 2014, 01:29:56 pm
Anyone think there's a good reason why they don't return a proper DNG with their processing instructions added as metadata?
Outside their processing, how would it be used? But if they are supposed to do this, they most certainly should.
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 24, 2014, 01:45:52 pm
It's rendered so not raw, but minimally processed. It's certainly more scene referred than output referred. Some operations may be easier to conduct, white balance comes to mind. But it isn't raw and that's the big deal IMHO. Don't let the DNG part of the name give you an idea it's a digital negative as we think of a raw neg. You've baked those pixels but to a lesser degree than using other tools on the data.

Yup.  I understood that going in.  But to bring this back to the starting point, I've been favorably impressed for years with the results of DxO Optics Pro's automated optical corrections.  Lightroom's lens correction module works pretty well, but I've found DxO to be generally superior—at least with the sensor-lens combinations I've tried.

On the other hand, I prefer LR (with an occasional but increasingly rare visit to Photoshop) for other post-processing.  So I turn off all adjustments in DxO except the corrections directly related to the company's sensor-lens database, and use LR for everything else.

Until now, that has meant rendering in DxO Optics Pro and importing a TIFF into Lightroom.  Aside from the awkwardness of the extra manual step, there were things that definitely didn't work right when I worked on the DxO TIFF in Lightroom.  For example, white balance adjustments simply didn't match what I could do in LR with the raw file.

It seems (I'm not confident in the comprehensiveness of my initial judgment) that DxO's new ability to emit a DNG has solved both problems.  I can hand off the raw to Optics Pro directly from LR (DxO provides a new LR plug-in for that) and the Lightroom sliders seem to work pretty much the same way on the DxO-emitted DNG as they do on the raw file, modulo the inherent difference in the way the two programs perform the raw conversion.  While I may no longer be working on the raw file, it's still there if I'm not happy with the DxO result.  But mostly, the optical corrections made by DxO as part of its rendering process seem like a win to me.

Now the ideal arrangement from my perspective would be for Adobe to license DxO's technology and use it to make sensor-lens corrections to the raw file on-the-fly.  But short of that, half a half-backed loaf is better than a fully-baked loaf.  If that makes any sense.
Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 24, 2014, 02:09:37 pm
Outside their processing, how would it be used? But if they are supposed to do this, they most certainly should.

It would be for their processing, just like ACR adjustments. I'm surprised they didn't follow that route.
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 24, 2014, 03:36:28 pm
It would be for their processing, just like ACR adjustments. I'm surprised they didn't follow that route.

As far as I can determine, DxO Optics Pro operates directly on the sensor data from the raw file and makes adjustments on-the-fly, similarly to Lightroom (i.e., "non-destructive parametric editing"), then produces a rendered file only if the user wants it to emit one.  The major change from previous revisions of the product is that v. 9.5 now can produce linear DNG files as well as TIFFs and JPEGs.  Plus, they've added some functionality and a Lightroom plug-in to allow the user to (1) tell DxO which raw file(s) to operate on from within Lightroom (I'm not sure whether that offers any advantage over Adobe's exiting "Edit-In" facility) and (2) pass the DxO output file back to Lightroom.
Title: Re: Lightroom-DxO Round-Trip
Post by: Misirlou on May 24, 2014, 04:42:10 pm
It's just more convenient to work with this new DxO path, at least for me.

You might also consider noise reduction in DxO. Depending how noisy your image is, and what you intend to do with it, the DxO "prime" reduction process is pretty amazing. It takes a very long time to process, but I haven't seen better results with any other system. Useless for a wedding photographer, but if you've got a single image with a lot of noise, or a small group of them, it's worth the hassle.

I recently shot a set of HDR sequences for panorama stitching. After much experimentation, I determined that I got the most out of them by converting the raws, with lens correcting and denoising in DxO. Then I tried several flavors of HDR, with the NIK process (also launched from Lightroom) coming out the best with that particular set of photos.

I think a lot of the complaints about DxO white balance come down to their default camera profiles. You get an easier profile selection (or custom creation) with Adobe, and their recent default camera profiles are quite good (with my cameras at least). If you're willing to come up with better user profiles for DxO, you could probably do just about as well.

But no matter how good DxO gets, LR's DAM will always be important to me, as well as easier creative controls with LR's tonal adjustments.
Title: Re: Lightroom-DxO Round-Trip
Post by: Steve House on May 24, 2014, 04:42:53 pm
When I import a NEF camera raw file into Lightroom with LR converting it to DNG on import, is the resulting file a "linear DNG?" What other types of DNG are there?
Title: Re: Lightroom-DxO Round-Trip
Post by: dennbel on May 24, 2014, 05:05:16 pm
I've often wondered just how accurate the DXO correction is for MY lens and My sensor. How many do they test and do they average them? Same model lens and sensor from batch to batch have variances. So what they get, may not be entirely accurate for my lens and sensor. Just thinking out loud.
Title: Re: Lightroom-DxO Round-Trip
Post by: Schewe on May 24, 2014, 08:44:28 pm
Anyone think there's a good reason why they don't return a proper DNG with their processing instructions added as metadata?

Because DxO's image corrections need to alter pixels with their algorithms for lens corrections. ACR/LR could use DxO's algorithms if stored in metadata unless DxO gave (licensed) the algorithms to Adobe...so, DxO must render the raw pixels into RGB pixels to run the distortion/lens corrections on.

Linear DNG for all practical purposes is a semi-raw file format. The main difference is dealing with the camera color to processing profile conversions–with Linear DNG, you can't go upstream from that initial color transform.

That and the fact that a Linear DNG limits the full capability of PV 2012's highlight preservation...an L/DNG is really close but not exactly the same as raw.

I seriously doubt that Adobe (read Thomas) will open the ACR/LR processing pipeline to the extent that 3rd parties could plug into the pipeline...so, I suspect it'll always require a new file image be spawned off either as a rendered TIFF or L/DNG. The original raw will still be preserved and treated as read only.
Title: Re: Lightroom-DxO Round-Trip
Post by: Charlene McKinnon on May 25, 2014, 12:38:10 am
If you want to take advantage of all the features of DxO which normally require a raw file, except use Lightroom for rendering the raw image data, you can:


This will work perfectly as long as you don't bake distortion/lens corrections into the tiff beforehand, or crop, or make other corrections which are best reserved for DxO.

Obviously if you will be doing this a lot, a batch file or plugin or some way to automate repetitive steps would be invaluable.
Title: Re: Lightroom-DxO Round-Trip
Post by: Ligament on May 25, 2014, 04:29:10 am
Does DXO Optics Pro 9.5 *still* use Adobe RGB as its internal working colorspace? If so, you could loose a lot of valuable data exporting from lightroom (profoto working space) to DXO (adobe RGB) back to lightroom (now with an adobe RGB file).

This has always been a peeve of mine with DXO. Why on earth would they use Adobe RGB as the internal working space? Why throw all that data away?
Title: Re: Lightroom-DxO Round-Trip
Post by: Damon Lynch on May 25, 2014, 04:49:58 am
Thanks everybody for explaining linear DNG.

If you want to use LR but like DxO's optical corrections, there is also DxO Viewpoint.  It's so much cheaper. It is a real time-saver when it comes to correcting the volume deformation caused by wide angle lenses, particularly useful for photos with people in them.

I remain bemused by the apparent fact that in this multi-billion dollar industry, one set of people (a company's own optical engineers) know perfectly well the optimal algorithms to improve image quality in post-processing, yet the set of most important people in post-processing (Adobe, DxO, et al.) must themselves develop algorithms that only approximate the ideal corrections. Would companies like Canon and Leica suffer if these optical correction algorithms were mandated to be open access?
Title: Re: Lightroom-DxO Round-Trip
Post by: fdisilvestro on May 25, 2014, 08:39:18 am
Hi,

A few observations,

1) Linear DNG output from DXO Optics Pro is not new of 9.5. It has been there a long time ago, at least from version 6.

2) The new thing is the LR plugin that allows to export a raw file from LR to DXO (LR usually exports rendered files to external editors). In the past, DXO OP had the ability to read from LR catalogues, so you could use LR as DAM, select the images, and then open the catalogue from DXO and select a specific collection. This worked with DXO 6 and probably DXO 7 with LR up to version 3. This new plugin is even better than the old way to read LR catalogues.

3)If you convert your raw files to DNG without keeping the original raw, then you cannot use this plugin, since DXO does not accept DNG as input (except those created directly in camera, as few models do)

4)The output DNG from DXO is a linear DNG meaning that it has been demosaiced and interpolated, because this is a necessary step before applying optical corrections.

5) The output DNG is less processed than a TIFF. It has not been encoded to a color space (so you avoid the limitation of Adobe RGB if you output TIFF). Additionally you can apply your preferred DNG profile and you can even create a DNG profile with the Xrite tool from this linear DNG.

6)When to use this route: If you want the Prime noise reduction and/or DXO optical corrections, which for some camera / lenses combinations are superior to Adobe's in my own experience, especially the "lens softness".

7)Note if you go this route: Apply white balance in DXO if you have blown out highlight. The explanation for this is long and has been posted in this site a while ago. If you do not do this and wait until returning to LR to apply white balance, you might end with color casts in those highlights.

8) You may try DXO for free for a month
Title: Re: Lightroom-DxO Round-Trip
Post by: Charlene McKinnon on May 25, 2014, 09:48:11 am
you could loose a lot of valuable data

Theoretically, I suppose, although in practice - I can't tell there is any color loss / degradation.
Title: Re: Lightroom-DxO Round-Trip
Post by: Charlene McKinnon on May 25, 2014, 09:57:32 am
there is also DxO Viewpoint.

Thanks for the tip. I just tried it - notes:


Anyway, I agree - if those are the corrections you need (and you want raw conversion by Lightroom), then it may not be a bad way to go...
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 25, 2014, 11:00:28 am
When I import a NEF camera raw file into Lightroom with LR converting it to DNG on import, is the resulting file a "linear DNG?" What other types of DNG are there?
No, probably not linear but the actual raw data. Linear is an option, off by default.
Title: Re: Lightroom-DxO Round-Trip
Post by: JimAscher on May 25, 2014, 11:19:00 am
Hi,

A few observations...

3)If you convert your raw files to DNG without keeping the original raw, then you cannot use this plugin, since DXO does not accept DNG as input (except those created directly in camera, as few models do)....


To my disappointment, and following a long email correspondence with the very cordial DxO customer service people, I learned that DxO does not support all DNG's created directly in camera.  It apparently does for the later model Leica's but not for my Ricoh GXR M-module, reportedly for the reason that that specific camera has been discontinued by Ricoh.  So, I can use DxO for converting raw from other of my cameras (with the exception also of my Foveon-sensor Sigma), but not DNG's from my Ricoh GXR. 
Title: Re: Lightroom-DxO Round-Trip
Post by: Damon Lynch on May 25, 2014, 12:09:25 pm
Hi Charlene,

I find the volumetric anamorphosis correction the most useful feature in Viewpoint, especially for faces near or at the edge of the frame. When you open a TIFF in Viewpoint via Photoshop from LR, and it's asking for the original RAW, that's a bug. I find the CA removal in LR to be sufficient for my needs.

Damon
Title: Re: Lightroom-DxO Round-Trip
Post by: Charlene McKinnon on May 26, 2014, 12:22:09 am
Hi Damon,

I find the volumetric anamorphosis correction the most useful feature in Viewpoint, especially for faces near or at the edge of the frame.

I too find it useful in some cases. It's just that a lot of cases I try it, and it make the photo look better in some ways but worse in others - no net improvement, or net negative...

Quote
When you open a TIFF in Viewpoint via Photoshop from LR, and it's asking for the original RAW, that's a bug.

Could be, but it needs access to the raw file for exif metadata, in order to do module-based corrections to jpegs etc.

Quote
I find the CA removal in LR to be sufficient for my needs.
Same here - Lr's CA removal seems to work quite well so far...


Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 27, 2014, 04:47:45 am
Because DxO's image corrections need to alter pixels with their algorithms for lens corrections.

What I was actually suggesting, Jeff, was that DXO might have chosen to output a regular DNG rather than a linear one, storing their adjustment parameters as metadata in the same way as LR/ACR but to DXO's own namespace. LR would certainly preserve that metadata in the file, even if it doesn't do anything with it (it already preserves other unknown metadata). I think the DNG spec also allows DXO to write their own embedded previews and speed up rendering each time the DNG is reopened in that program.
Title: Re: Lightroom-DxO Round-Trip
Post by: eliedinur on May 27, 2014, 06:37:59 pm
What I was actually suggesting, Jeff, was that DXO might have chosen to output a regular DNG rather than a linear one, storing their adjustment parameters as metadata in the same way as LR/ACR but to DXO's own namespace. LR would certainly preserve that metadata in the file, even if it doesn't do anything with it (it already preserves other unknown metadata). I think the DNG spec also allows DXO to write their own embedded previews and speed up rendering each time the DNG is reopened in that program.
John, I don't understand what would be the point of their doing that. You would have a fully editable DNG with part of its metadata useless to LR and part of it useless to DxO. How would you go from there to a single viable RGB image file that incorporates contributions from both converters?
Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 29, 2014, 04:11:20 am
John, I don't understand what would be the point of their doing that. You would have a fully editable DNG with part of its metadata useless to LR and part of it useless to DxO. How would you go from there to a single viable RGB image file that incorporates contributions from both converters?

The value would be the fully-editable DNG would contain all its metadata, as it is designed to do (metadata not understood by LR isn't "useless"). I guess I am assuming that one would use one or other converter, not make adjustments in both. Do people really want to do that?
Title: Re: Lightroom-DxO Round-Trip
Post by: fdisilvestro on May 29, 2014, 04:36:31 am
The value would be the fully-editable DNG would contain all its metadata, as it is designed to do (metadata not understood by LR isn't "useless"). I guess I am assuming that one would use one or other converter, not make adjustments in both. Do people really want to do that?

Yes, that's the whole purpose, you might perform optical corrections and noise (prime) corrections in DXO and the rest of the edits in LR (color, tone, etc).
One specific example: for the nikon 70-200 VRI, the optical correction from LR does not compensate uniformly for vigneting, leaving one or two corners darker than the rest. The corresponding module of DXO works just fine.

Regards
Title: Re: Lightroom-DxO Round-Trip
Post by: Fine_Art on May 29, 2014, 09:53:57 am
I've been playing around with DxO Optics Pro 9.5's new trick: the ability to import a raw file from Lightroom and return a linear DNG.  If I understand this properly (as usual, the explanatory material on the DxO website is less than pellucid), by emitting a DNG, DxO Optics Pro allows a Lightroom user to take advantage of DxO's sophisticated optical testing and correction technology while still retaining all the functionality of LR—except, of course, Adobe's demosaicing algorithms.  For the moment, at least, I'm only interested in DxO's automated sensor-and-lens corrections, with all additional processing performed in LR.

I'm encouraged by the initial results.  I've attached a small center crop from a much larger image made with a Nikon D800E and Nikon's 24-120mm f/4 zoom.  The first attachment is a 1:1 JPEG of the raw file as demosaiced by LR after applying LR's automatic lens corrections.  The second attachment is the same raw file demosaiced by DxO after applying DxO's automatic corrections for the sensor and lens.  I didn't do any manual sharpening or make any other corrections with either tool since my objective is to compare how the two perform the raw conversion and correct for known optical imperfections in the capture.  I've only worked on a few images so far, and only those shot with this particular sensor-lens combination, so it's too soon to come to any definitive judgment.  But in every test I've made, the DxO image looked better than the LR equivalent—sometimes dramatically so.

My only concern is that I may be giving up some of Lightroom's functionality by having DxO perform the conversion and turn over the linear DNG to LR.  It appears LR can still properly modify the white balance of the DxO-emitted DNG.  Is there anything else I might be giving up, or should otherwise be concerned about?

Why does the LR version look so mushy compared to the DxO version?
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 29, 2014, 10:10:05 am
The value would be the fully-editable DNG would contain all its metadata, as it is designed to do (metadata not understood by LR isn't "useless"). I guess I am assuming that one would use one or other converter, not make adjustments in both. Do people really want to do that?
How is metadata that LR can't understand anything but 'useless' to it and the end user unless they render? How can one move from raw converter to raw converter with raw data and two differing sets of instructions only one converter can understand?
Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 29, 2014, 10:29:30 am
Imperfectly, but possibly of value, as C1 has shown by reading LR adjustments.

Storing more than one raw converter's adjustments in the DNG metadata is what the X in xmp is all about.
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 29, 2014, 11:31:27 am
Why does the LR version look so mushy compared to the DxO version?

I'm not sure I understand your question. One of the points I made in the post that started this thread was that the rendering with default DxO optical corrections seemed to produce a better initial image than the Lightroom rendering with Lightroom's lens corrections.

In both cases, the sample images were produced without any manual capture sharpening.  Even though the D800E effectively has no AA filter, my experience has been that some capture sharpening is still necessary.  The reason I want to emphasize that I performed no manual capture sharpening is that one of the mainstays of the DxO optical correction suite is a correction for "lens softness."  I don't know whether Lightroom's lens correction module for the lens I was using includes a similar property.  The samples I posted represented a small center crop of a much larger capture file (see attachment) so small differences in edge acuteness are noticeable.

Again, my experience is that DxO often seems to provide a better starting point than Lightroom for the camera sensors and lenses I use.  That especially seems to be true with the D800E, perhaps because the sensor's high resolution reveals optical defects in a lens that wouldn't otherwise be as objectionable and DxO's sensor-lens correction model does a superior job of neutralizing those defects.  I haven't attempted to perform any rigorous testing to determine whether additional processing with Lightroom would compensate for DxO's initial benefits—and I'm not sure any generalization is possible because there are so many variables that might need to be considered: output format (i.e., print or screen), degree of cropping, size of final image, etc.  I'm working on the assumption that this is something that needs to be determined on an image-by-image basis.

What I've tentatively concluded after reading all the responses in this thread is that I'm not giving up much by having DxO perform the initial raw conversion and then continuing to process the DxO-emitted linear DNG in Lightroom.  And of course the original raw file is always available, so I can still have the option of Lightroom-only processing if I'm not satisfied with what I get from DxO.
Title: Re: Lightroom-DxO Round-Trip
Post by: digitaldog on May 29, 2014, 11:43:54 am
Imperfectly, but possibly of value, as C1 has shown by reading LR adjustments.
Storing more than one raw converter's adjustments in the DNG metadata is what the X in xmp is all about.

I'm all for saving all the data, but what do you do with it? I have a raw file inside a DNG with two sets of instructions unique to each product which renders the data. I've got C1 and LR xmp. Now what and in what order to utilize both on the raw data?
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 29, 2014, 01:35:56 pm
Storing more than one raw converter's adjustments in the DNG metadata is what the X in xmp is all about.

I'm not certain if this accomplishes what you're proposing, but DxO Optics Pro writes a sidecar file containing its proprietary metadata into the directory where the raw image is stored.  The file doesn't have a dot-x-m-p extension, presumably to avoid overwriting any sidecar for the same image created by Lightroom.  (It's saved as <raw file prefix>.dop.)  The purpose of the sidecar apparently is to permit you to resume processing in DxO wherever you suspended it in a previous session.  In other words, DxO doesn't apply its default adjustments to the raw image, but rather the adjustments you were using when the previous session ended as documented in the sidecar.  The only purpose of DxO's linear DNG, as far as I can tell, is to give Lightroom a partially-baked file to use when you continue processing in LR.  DxO doesn't use the linear DNG at all; it always operates on the raw image.
Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 29, 2014, 02:07:21 pm
DNG files don't need sidecars - it's an advantage of the format. Instead DXO could write to their own section of xmp inside the file itself, and read it when resuming processing. But this capability is different from the desire to process Dxo results in Lightroom.
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on May 29, 2014, 02:37:45 pm
DNG files don't need sidecars - it's an advantage of the format. Instead DXO could write to their own section of xmp inside the file itself, and read it when resuming processing. But this capability is different from the desire to process Dxo results in Lightroom.

DxO doesn't use the linear DNG.  It operates on the original raw file.  I guess it could read metadata from the linear DNG instead of the sidecar, but I'm not sure what the advantage of that would be, since if and when processing resumed, DxO would again be applying its corrections to the original raw.
Title: Re: Lightroom-DxO Round-Trip
Post by: john beardsworth on May 29, 2014, 03:05:05 pm
I think we're going round in circles here, Chris.
Title: Re: Lightroom-DxO Round-Trip
Post by: tnargs on July 15, 2014, 06:25:37 pm
7)Note if you go this route: Apply white balance in DXO if you have blown out highlight.

On a related issue, when I work on an image's colour in DxO, then export to LR as a DNG, the image is not even near to the same colour balance when I view it in LR compared to the colour balance I was happy with in DxO.

(Yet, if I export to LR as a JPEG, the colour balance is a match when viewed in either program.)

Why is this happening and, more importantly, what can I do to get DNG files exported from DxO to LR to retain the nice colour balance I got in DxO?

Please help.
Title: Re: Lightroom-DxO Round-Trip
Post by: Dr Tone on July 15, 2014, 09:16:02 pm
If you are applying color rendering adjustments in DxO you need to export as TIFF to retain your color in Lightroom.  DxO mentions this in their Lightroom plug in tutorial.
Title: Re: Lightroom-DxO Round-Trip
Post by: tnargs on July 15, 2014, 09:41:33 pm
I have seen that but why?  ??? What fundamental reason prevents accurate transfer of colour using DNG files between DxO and LR?

Is it something DxO is doing? (in which case the colour will not transfer well from DxO DNG files to any other software, not just LR. Why would DxO want to do that? Why even offer DNG if it can't preserve colours?)

Or is it something LR is doing? (in which case there might be a 'cure' or 'compensation' possible, either in DxO or in LR, and I would like to know how to do that.)

BTW, using TIFF has other tradeoffs that don't suit my needs.
Title: Re: Lightroom-DxO Round-Trip
Post by: Dr Tone on July 15, 2014, 10:23:26 pm
I think you'll find your answer in the second post of the following thread:

http://www.luminous-landscape.com/forum/index.php?topic=68829
Title: Re: Lightroom-DxO Round-Trip
Post by: fdisilvestro on July 15, 2014, 11:11:12 pm
Hi,

The reason is that DXO demosaices and interpolates the file in order to apply the optical corrections. By doing this, you don't have the original RAW anymore but a linear DNG where the individual channel values have been modified. For the rest of edits, DXO uses its own metadata (as to instructions to apply to the file, similar to the LR metadata) which are not compatible with LR. If you export the DNG to LR, you loose the DXO metadata (the color and tone edits) and you start with a quasi-Raw file in LR.

Now,  why do I say you have to white balance in DXO if you have blown out highlights? Well, for some reason, DXO produces 16 bit per channel unsigned integer values in the DNG it produces. The curious thing is that all non-clipped (from the original raw) data is spread out in 15 bits (0 -32,677) and blown out data gets a much higher value that needs the 16 bits, but it is not the maximum value (65535) Why it does it I have no clue, but what happens is that LR does not consider it as a "blown out value" so if you modify the white balance in LR later on, you will get non-neutral values in those highlights. You might correct them with local edits, but in some cases it might require a lot of effort,
Title: Re: Lightroom-DxO Round-Trip
Post by: tnargs on July 16, 2014, 01:31:26 am
If you export the DNG to LR, you loose the DXO metadata (the color and tone edits) and you start with a quasi-Raw file in LR.

That is not my (very limited, new to DxO) experience.

Example: I start with a RAW file that looks too blue in LR. I export to DxO and correct the colour, then export the file to LR as DNG.

If you are right, it should now look blue again when I lose the colour edits. But.... it actually looks too yellow!
Title: Re: Lightroom-DxO Round-Trip
Post by: fdisilvestro on July 16, 2014, 01:49:58 am
That might be due to white balance, which does get "cooked" into the DNG. White balance =/= color edits. White balance is basically a scalar multiplication of the raw values by a factor in each channel.

Additionally, you have to verify which camera (DCP/DNG) profile is applied to the file in LR, since what you have before going to DXO might be different than what is applied when you return back, and it can make the image look different.
Title: Re: Lightroom-DxO Round-Trip
Post by: tnargs on July 16, 2014, 05:34:11 am
My camera profile is not in LR so that's not it. LR uses Adobe Standard profile under the Camera Calibration tab.
Title: Re: Lightroom-DxO Round-Trip
Post by: Denis de Gannes on July 16, 2014, 06:10:23 am
My camera profile is not in LR so that's not it. LR uses Adobe Standard profile under the Camera Calibration tab.

Each and every camera model that is supported by Lightroom/ ACR has its own specific "Adobe Standard profile".
By default this is the one selected, and is specific to your camera. Some cameras only have an Adobe Standard profile others have like  "Camera Vivid, Camera Portrait etc which are additional profiles created by Adobe to emulate the particular special settings you may have chosen when shooting with your camera.
Title: Re: Lightroom-DxO Round-Trip
Post by: tnargs on July 16, 2014, 08:17:25 am
That's good to know, and thanks for explaining, though in my newness to this topic, I can't see why it makes sense for Adobe LR to apply a camera profile to a linear DNG file that has been through the DxO camera and lens profiles.
Title: Re: Lightroom-DxO Round-Trip
Post by: fdisilvestro on July 16, 2014, 08:28:24 am
Hi,

It is because the only thing that has happened so far is demosaicing (e.g. estimating the missing R,G or B values so that each for each pixel we get rgb coordinates, and the optical correction, which is just a geometric transformation. In essence, it is not much different than a raw, the color space has not been encoded yet nor any tone curve. It is not just LR that can open and apply a DNG profile, you could actually build a DNG profile out of one of those DNG's (if the image contains a colorchecker). The file is still in "scene referred" mode
Title: Re: Lightroom-DxO Round-Trip
Post by: Chris Kern on July 16, 2014, 07:50:04 pm
It is because the only thing that has happened so far is demosaicing (e.g. estimating the missing R,G or B values so that . . . for each pixel we get rgb coordinates, and the optical correction, which is just a geometric transformation. In essence, it is not much different than a raw, the color space has not been encoded yet nor any tone curve. . . . The file is still in "scene referred" mode

If you're correct, you have finally provided the definitive answer—and very succinctly, at that—to the question which prompted me to start this thread.  Moreover, and not coincidentally, your explanation is consistent with the description of linear DNG by Eric Chan cited above.  Thanks.
Title: Re: Lightroom-DxO Round-Trip
Post by: tnargs on July 16, 2014, 08:49:07 pm
Excellent and thank you.