Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: meyerweb on September 18, 2011, 09:53:04 pm
-
OK, I know that, in general, I should do as much editing as possible against the raw file (using LR3) and convert to another format only if exporting for some other use. BUT...
Many plug-ins, including the NIK plug-ins that I really like, output a new file as Tiff or PSD. So yes, I do as much editing in LR3 before using the plug-in.
But I noticed recently that Nik suggests a workflow for Dfine that calls for doing NR before doing any other edits. Now then, Dfine might not be all that much better than the NR in Lightroom, but let's assume this is an example of a more general case.
So if I run Dfine first, I then I have to apply all my edits to a Tiff or PSD file. What do I lose compared to doing them on a raw file?
I'm assuming that if I edit an image in a plug-in, LR applies all the edits prior to passing the data to the plug-in.
Thanks.
-
I'm glad you asked this question. It's been the elephant in the room. I'm waiting as well for the answer from one of our experts.
-
Simple. Use the MUCH improved noise reduction in LR3!
-
Basically none of these third party plug-ins do anything specifically useful in LR. They render out the raw and apply them as if you exported the data and used their siblings in Photoshop.
So its about workflow. When do you want to draw the line in the sand and stop instruction based (parametric), totally non destructive editing workflow? I can’t answer that for you. I can say I see nothing at all useful in these products because they may force me to render data before I want to. I might use them if I’m 100% sure I’m done processing the raw data and ready to render. I understand that as the raw processors improve (we just saw a major quality leap in PV 2010 over PV2003), I may want to re-render the data which means any editing thereafter has to be redone. Once you render data, the raw processing capabilities are over, you can’t get that toothpaste back into the tube. Its fast (instruction based), high bit, wide gamut, applied in an optimal order, allows iterations (Virtual Copies) with virtually no overhead etc. For those reasons, I use as much of the raw tools as possible.
-
As I understand things, it's a mistake to think that you are ever actually editing the RAW file. A raw file does not contain an image, it contains camera sensor data that can be processed (rendered) to create an image. So, when you view a raw file in LR, what's really happening is that LR is using it's default rendering settings to process the raw data to create the image you see on screen. When you "edit" a raw file in LR, the raw file is not changed, but the processing instructions are changed (and thus the on-screen image changes).
Bottom line, then, is that when you do NR in LR, you are not working on the raw file but on the current, on-screen rendering of it. So, once you understand this, I think you can see that your question does not really make sense. LR's NR is really good, you - or at least I - rarely need anything more.
-
Thanks to all. Yes, I understand the LR3's NR is very good. That's why I said let's look at it as a more general case regarding plug-ins. I understand what's been said above, and I understand the workflow advantages of parametric editing, but still have the basic question:
In terms of flexibility, capability, range of adjustment, quality of results, etc., when changing things like color temp, exposure, curves, color balance, lens corrections, etc. are the results going to be the same (or at least comparable) when working on a tiff file as when working with raw? Or does working on a tiff file impose limits on what can be done to an image in LR or the quality of the output? Is the benefit to working with raw files in LR primarily workflow, or is there a benefit in terms of image quality, too?
If I apply a plug-in, from Nik or anyone else, create a tiff with the output of that plug-in, and then proceed to do more edits in LR3, will that impact what I can do with the image (not the workflow).
I'm not trying to justify not using raw files; I'm just trying to understand how LR3 works with non-raw files compared to raw.
Thanks again.
-
As I understand things, it's a mistake to think that you are ever actually editing the RAW file. A raw file does not contain an image, it contains camera sensor data that can be processed (rendered) to create an image.
A raw file contains an image just as does a TIFF. The raw image contains data in a mosiac format, whereas the TIFF uses a separate channel for each of the RGB components. Shown below is a straight dump of a raw image as shown by Rawnalize--it looks to me like an image. However, it needs to be demosiaced by a raw converter just as the three separate channels of a TIFF need to be combined in Photoshop.
Regards,
Bill
(http://bjanes.smugmug.com/Photography/Raw/i-Lk6qNv5/0/M/NEFDump-M.png)
If you look at a lower magnification, the image outlines can be better seen:
(http://bjanes.smugmug.com/Photography/Raw/i-d5mMGgk/0/O/NEFDumpLowPower.png)
-
The workflow aspects aside, if you generate a tiff with a plug-in, you can continue to work on that further in Lightroom using the same tools that you would have used if you were still working with the converted RAW.
In terms of quality when working on the tiff, it's difficult to give a definite answer because it's not possible to know what you might do to an image before generating the tiff; and not being familiar with the NIK plug-ins, I don't know if they generate 8 or 16 bit tiffs. If they are producing 16 bit depth tiff's, then in general, you should probably be ok, but you never know, there might be something you want to do to the image and it breaks.
For the other aspects, it's not as flexible or capable and you won't have as much latitude for adjustments as with the original RAW data.
-
In terms of flexibility, capability, range of adjustment, quality of results, etc., when changing things like color temp, exposure, curves, color balance, lens corrections, etc. are the results going to be the same (or at least comparable) when working on a tiff file as when working with raw?
In a word, no.
-
So its about workflow. When do you want to draw the line in the sand and stop instruction based (parametric), totally non destructive editing workflow? I can’t answer that for you. I can say I see nothing at all useful in these products because they may force me to render data before I want to. I might use them if I’m 100% sure I’m done processing the raw data and ready to render. I understand that as the raw processors improve (we just saw a major quality leap in PV 2010 over PV2003), I may want to re-render the data which means any editing thereafter has to be redone. Once you render data, the raw processing capabilities are over, you can’t get that toothpaste back into the tube. Its fast (instruction based), high bit, wide gamut, applied in an optimal order, allows iterations (Virtual Copies) with virtually no overhead etc. For those reasons, I use as much of the raw tools as possible.
Andrew, thank you! I have a gut feeling of unhappiness whenever I render a shot, for whatever reason, and find myself fiddling with it back in LR. I'm pleased to hear that it might not be entirely fanciful.
Jeremy
-
Is the benefit to working with raw files in LR primarily workflow, or is there a benefit in terms of image quality, too?
Maybe a few tests will help answer the question. Make a copy of a RAW file. Develop both copies in LR with all adjustments zeroed out. Render one copy as a Tiff (ProPhoto 16 bit). Then, make identical adjustments to the RAW file and the Tiff in LR. Such a test should give some indication whether the end results of LR adjustments are different for RAWs versus Tiffs.
-
Maybe a few tests will help answer the question. Make a copy of a RAW file. Develop both copies in LR with all adjustments zeroed out. Render one copy as a Tiff (ProPhoto 16 bit). Then, make identical adjustments to the RAW file and the Tiff in LR. Such a test should give some indication whether the end results of LR adjustments are different for RAWs versus Tiffs.
It's not really a test that works that well, because you can't extrapolate the results beyond the test images you use. The results only tell you about the images you test, nothing more. Further edits might be OK in one image with one set of adjustments, but not be OK on another image or with another set of adjustments.
And that's not to say you should never render a tiff or psd out of Lightroom. If you have to, you have to. LR is not a complete replacement for Photoshop and there are many legitimate reasons why you might want to edit at the pixel level. It's just, if you can achieve something as easily in LR working with the RAW data, then that is the most flexible and least destructive approach to your workflow.
-
It's not really a test that works that well, because you can't extrapolate the results beyond the test images you use. The results only tell you about the images you test, nothing more. Further edits might be OK in one image with one set of adjustments, but not be OK on another image or with another set of adjustments.
Peter, what you say is true, obviously. However, as an analogy, it's not possible to prove a scientific theory, yet people do experiments all the time. For example, no one has proved the theory of gravity. No one has disproved it, either, but no one can be certain that another experiment might some day disprove the theory.
My point being that a few test images could prove that LR handles RAWs differently than Tiffs. Or the tests may show no difference, in which case I'd say you still learn something. If you do more tests and still see no difference, you learn more, but of course such tests can never prove that there will never ever be any difference. ;)
-
This thread makes me wonder whether I've been doing what I thought I was doing in my workflow.
I use DxO Optics Pro as a preprocessor to perform its automagical measured optical corrections for my various sensor-lens combinations, and have it configured to emit a DNG file, which I then open in Lightroom. Presumably, Lightroom invokes the Adobe Camera Raw module to render it. (If I open the DNG produced by DxO in Photoshop, the initial display is ACR.)
My assumption was that I wasn't losing any of the raw file's luminance data along the way. Obviously, some of the information must be shifted around a bit as part of the DxO optical munging, but I haven't been thinking of that as a loss so much as compensation for the optical deficiencies of the camera.
I realize that without knowing the internals of the DxO product, it probably is impossible to determine whether any useful data are being dropped on the floor. But assuming DxO actually is doing only what I have configured it to do -- i.e., just the optical corrections and no other image processing -- is there any reason to suspect that I am providing Lightroom with any less information than it would have if it was parsing the camera manufacturer's raw file directly?
Chris
-
Peter, what you say is true, obviously. However, as an analogy, it's not possible to prove a scientific theory, yet people do experiments all the time. For example, no one has proved the theory of gravity. No one has disproved it, either, but no one can be certain that another experiment might some day disprove the theory.
My point being that a few test images could prove that LR handles RAWs differently than Tiffs. Or the tests may show no difference, in which case I'd say you still learn something. If you do more tests and still see no difference, you learn more, but of course such tests can never prove that there will never ever be any difference. ;)
Totally agree with the basic principle. To take the scientific argument further, in testing a hypothesis, you don't try to prove your own theory correct, but generally try to show that it fails, or that the null hypothesis (opposite view) is correct. By eliminating the alternatives, you draw a stronger argument to show that your original hypothesis has general application as a theory.
The only difference I would have is that I wouldn't view a few images as being a sufficiently large sample size, particularly given the variables involved. If you want to draw some general conclusions, you would have to test lots of images.
But that's taking the thread totally off topic.
I will still premise my view with the fact that it's quite possible to work with tiffs and psds without breaking them. Even before smart objects came to Photoshop, we used to work on tiffs and psds without breaking them. But, it's less flexible and the files may have less latitude for adjustment, so if you can do something equally well in LR with it's tools, there's not much need to produce a tiff. If you can't do it as well in LR with its tools, then you might not have a choice but to use a plug-in.
-
I use DxO Optics Pro as a preprocessor to perform its automagical measured optical corrections for my various sensor-lens combinations, and have it configured to emit a DNG file, which I then open in Lightroom. Presumably, Lightroom invokes the Adobe Camera Raw module to render it. (If I open the DNG produced by DxO in Photoshop, the initial display is ACR.)
My assumption was that I wasn't losing any of the raw file's luminance data along the way.
That DNG is rendered (demosaiced) data. So no, you are not sending the actual raw data from DxO to LR in the same way as simply using the original data (or converted DNG) in LR. Keep in mind that a DNG can contain rendered data. No different from a TIFF.
-
is there any reason to suspect that I am providing Lightroom with any less information than it would have if it was parsing the camera manufacturer's raw file directly?
Chris
DNG files produced b DxO Optics Pro are not really Raw files as the ones that come out of the camera. First, they have been interpolated and demosaiced, since that's the only way for DxO to apply the optical corrections.
These DNGs still are not completely rendered files, that's why they open in ACR. Compared to Tiff's, they are not color space encoded nor gamma corrected.
You could assign a camera profile and perform white balance and color space encoding as in a RAW file in LR/ACR.
An important note is the way blown out highlights are treated when using this workflow: the resulting DNG don't have those highlight at saturation (equivalent to 255 in 8 bits) meaning that if you perform white balance in LR/ACR and then apply highlight recovery, your highlights may result non neutral. In this case you should perform white balance in DxO before producing the DNG.
More on this thread (http://www.luminous-landscape.com/forum/index.php?topic=54284.0)
-
... it's not possible to prove a scientific theory, yet people do experiments all the time. For example, no one has proved the theory of gravity. No one has disproved it, either, but no one can be certain that another experiment might some day disprove the theory....
So, from a practical standpoint, you are saying that some day some guy might jump from a tall building and... not fall to the ground?
-
Basically none of these third party plug-ins do anything specifically useful in LR. They render out the raw and apply them as if you exported the data and used their siblings in Photoshop.
So its about workflow. When do you want to draw the line in the sand and stop instruction based (parametric), totally non destructive editing workflow? I can’t answer that for you. I can say I see nothing at all useful in these products because they may force me to render data before I want to. I might use them if I’m 100% sure I’m done processing the raw data and ready to render. I understand that as the raw processors improve (we just saw a major quality leap in PV 2010 over PV2003), I may want to re-render the data which means any editing thereafter has to be redone. Once you render data, the raw processing capabilities are over, you can’t get that toothpaste back into the tube. Its fast (instruction based), high bit, wide gamut, applied in an optimal order, allows iterations (Virtual Copies) with virtually no overhead etc. For those reasons, I use as much of the raw tools as possible.
I want a plugin api in lightroom that allows 3rd party plugins to "hookin" to the pipeline at any given point, and do its thing on the fly.
This might mean that re-rendering in Lightroom is slowed down for certain plugins, and that one LR database cannot be exported to another LR installation without having the same plugins installed, but I can live with that...
-h
-
I want a plugin api in lightroom that allows 3rd party plugins to "hookin" to the pipeline at any given point, and do its thing on the fly.
So do I (and an anti gravity machine). I don’t expect I’ll get my wish anytime soon.
-
So, from a practical standpoint, you are saying that some day some guy might jump from a tall building and... not fall to the ground?
Been done... Arthur Dent in 'The Hitchhiker's Guide to the Galaxy' series... book 4(?) Actually he tripped and forgot to fall, so there's a different set of parameters again! ;D
Mike.
(Just injecting a little levity)
-
So, from a practical standpoint, you are saying that some day some guy might jump from a tall building and... not fall to the ground?
I wouldn’t bet on it, and I wouldn’t want to be the guy doing the experiment, but ya never know. :-\
If I lived in the 1800’s and found someone willing to bet that Newton’s theory of gravity was wrong, I’d have bet everything I own. Einstein would have put me in the poor house! Ya never know.
-
DNG files produced b DxO Optics Pro are not really Raw files as the ones that come out of the camera. First, they have been interpolated and demosaiced, since that's the only way for DxO to apply the optical corrections.
[ ... ]
An important note is the way blown out highlights are treated when using this workflow: the resulting DNG don't have those highlight at saturation (equivalent to 255 in 8 bits) meaning that if you perform white balance in LR/ACR and then apply highlight recovery, your highlights may result non neutral. In this case you should perform white balance in DxO before producing the DNG.
More on this thread (http://www.luminous-landscape.com/forum/index.php?topic=54284.0)
Thanks for the reply and also to the link to your very interesting posts in the referenced thread. (And to Andrew Rodney, who also pointed out that the DNGs produced by DxO don't encapsulate raw data.)
Chris
-
The DxO folks don’t want to make that fact anything close to obvious <g>.
-
I want a plugin api in lightroom that allows 3rd party plugins to "hookin" to the pipeline at any given point, and do its thing on the fly.
If you mean plug-in to ACR processing pipeline, you'll have a really long wait. The ACR/LR works as well as it does because it's an unbroken pipeline completely under the control of the engineers. To open the pipeline up would hand over control of certain parameters to a 3rd party and I'm pretty darn sure Thomas Knoll would never do that.
-
If you mean plug-in to ACR processing pipeline, you'll have a really long wait. The ACR/LR works as well as it does because it's an unbroken pipeline completely under the control of the engineers. To open the pipeline up would hand over control of certain parameters to a 3rd party and I'm pretty darn sure Thomas Knoll would never do that.
I have no idea how mr Knoll and his collegues thinks about this.
But I fail to see why opening up the pipeline would be such a disaster from a technical/ergonomical point of view. From a sales/economic point of view it might, but that is not something that I am occupied with.
It could be made quite simple initially. Only one entry-point. Some well-defined format for i/o. If the right plugin is not found on the computer, it will simply be by-passed (and a more or less incompletely developed image is seen).
If you record music on your PC, there are standards for how plugins will work, and they are indeed "parametric" in that you can usually re-render the music in real-time at any time changing the parameters.
I do not see Lightroom as the ultimate raw processing pipeline, but a stable, intuitive, reasonably fast database that I have the confidence to spend hours importing my images into.
-h
-
A TIFF file is a rendered file as is a JPEG.
It's moderately well known that you cannot do white balance correction as effectively with a JPEG file as you can with a RAW file.
It therefore seems reasonable that a basic example of the limitation of TIFF vs RAW would be in white balance correction.
But that's something that you can do - once - prior to exporting the image.
Where you will lose editing quality is if you darken the RAW image at all in creating the TIFF.
The result of that will mean that colour information is thrown away that can never be reclaimed.
However the same holds true for generating any picture from data - once the data is transformed to make the image darker, you must go back to the original data to again lighten the image, you cannot use the rendered result even if the format stays the same (eg JPEG to JPEG.)
-
I have no idea how mr Knoll and his collegues thinks about this.
But I do and I'm telling you, the engineering required to open up the ACR/LR pipeline is simply not something that is likely to happen. Thomas and the boys are all about making the processing better within the pipeline but trying to hook up some other processing routine and not breaking the pipeline is a major deal. Everything that is processed within the pipeline is completely dependent upon the steps of the pipeline. Adding a 3rd party routine–and truth be told I've not yet heard a compelling reason yet–would make a complex pipeline even more complex and fragile...nope, sorry, not something I would want to see.
-
Some tools work equally well and similarly in behavior between raw and non-raw; examples include tone curves, HSL tweaks, post crop vignette, and split tone. But others are really optimized for raw or the processing is of a nature such that the adjustment is fundamentally best done in raw space; examples includ white balance, color profiles, sharpening, noise reduction, and lens corrections. So, there is still useful capabilities when adjusting non-raw files like jpegs but overall, the more you can take care of starting from th raw file, the better.
-
Thanks folks. An interesting discussion. I may try that experiment when I get some time. I know a few images alone won't be proof, but I figure if I try a high-key image, a low-key image, and a high-contrast image, and a variety of edits on each, it may tell me something.
-
So, there is still useful capabilities when adjusting non-raw files like jpegs but overall, the more you can take care of starting from th raw file, the better.
I would also add that a lot of the processing in ACR/LR for non-raw files really depends on the source of the file and what color space and gamma the rendered images are in. A camera JPEG that is sRGB or Adobe RGB will not fare as well in processing as a Photoshop 16 bit RGB image. The ACR/LR processing pipeline is based on doing the processing in it's own color space which is essentially ProPhoto RGB colors with a linear gamma in 16 bit. So the closer your rendered file would be to the pipeline, the better the processing of the image will be.
Look, processing an image inside of ACR/LR that started life elsewhere is still a useful strategy...arguably better than processing the file in Photoshop. But the closer that image is to raw, the better potential for corrections down the road.
If I were to rank the alternatives to raw, the 1st best alt would be a linear DNG. It's not restricted to a fixed color space and the linear gamma gives a lot of head room editing the extreme highlights (a benefit to raw). The next best would be a ProPhoto RGB in linear gamma, but that's not something that's easy to do...a ProPhoto RGB 16 bit would be a slight step down from ProPhoto Linear. Less good would be Adobe RGB 16 bit...the ARGB color space would clip potential colors and the gamma is already locked in at 2.2 so you would lose precision going from 2.2 to linear.
The worst source of files for editing in ACR/LR would be sRGB or Adobe RGB camera JPEGs...and it would be pretty low on the ladder compared to raw. But compared to opening an sRGB camera JPEG in Photoshop for editing, ACR/LR would still be a better processing pipeline.
I don't disagree that some 3rd party Lightroom plug-ins that render the raw file into a TIFF can be useful. The key point is at what point you go into the render format and why. If you've already done all the image corrections in ACR/LR and need a processing routine you can only get via the plug-in, go for it. You'll still have your original raw file you can go back to.
The vast majority of my images always remain in raw format. The only time I render a raw to a 16 bit ProPhoto RGB is to do those things that can only be done in Photoshop–and I always do this so I can soft proof and do final retouching prior to making serious prints. Note however, once I've rendered the image into Photoshop, I rarely go back and apply any Develop module image adjustments...
-
If I were to rank the alternatives to raw, the 1st best alt would be a linear DNG. It's not restricted to a fixed color space and the linear gamma gives a lot of head room editing the extreme highlights (a benefit to raw). The next best would be a ProPhoto RGB in linear gamma, but that's not something that's easy to do...a ProPhoto RGB 16 bit would be a slight step down from ProPhoto Linear. Less good would be Adobe RGB 16 bit...the ARGB color space would clip potential colors and the gamma is already locked in at 2.2 so you would lose precision going from 2.2 to linear.
Linear working spaces are used in Lightroom and ACR because white balance, demosaicing, and other processes performed by raw converters are best performed in a linear space. However, linear integer encoding is inefficient, since it wastes most of its range at the high end with a superfluous number of levels. Most of those levels can be thrown away with no perceptible damage to image quality, as is done with the Nikon compressed NEFs. The theoretical extra highlight headroom to which you refer is not attainable because the noise in the image is much greater than the quantization step (see Emil Martinec (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html)). The shadows in a linear integer encoding are impoverished and have a large quantization error as demonstrated by Figures 2 and 3 in Greg Ward's HDR paper (http://www.anyhere.com/gward/hdrenc/). Raw files are not posterized, but excessive editing of shadow areas can result in posterization if the effect is not masked by noise. Some raw converters now use 32 bit floating point in their working spaces.
As Bruce explains and is quoted below, the size and shape of the gamut is not affected by gamma, but the quantization is. Bruce has determined that a gamma of 2.12 gives the least quantization error, and this is reasonably close to the gamma 2.2 in AdobeRGB and the average gamma in sRGB. A linear space is not ideal for editing, but using 16 bits does keep the quantization error relatively low.
Regards,
Bill
Bruce Lindbloom:
Gamma does not affect the size or shape of the gamut. It does affect the distribution of RGB points within the gamut. Therefore, it is an important consideration for controlling quantization. I have previously analyzed the optimal gamma for the grayscale only (you can see this analysis here). This result was about 2.2 (actually either 2.1723 or 2.3243 depending upon the error metric used).
For Beta RGB, I additionally performed a three-dimensional analysis, where I looked at the ΔE produced by tiny perturbations in RGB space, measured at each of the locations in the color set. For each color, the RGB value was perturbed a distance of one-percent in each of 20 different directions (I used the centers of the 20 faces of an icosahedron to evenly distribute the directions in three-space). Minimizing the RMS ΔE resulted in a gamma of 2.12. This was sufficiently close to 2.2 that I did not feel a deviation from a "standard" 2.2 value was warranted.
-
So, from a practical standpoint, you are saying that some day some guy might jump from a tall building and... not fall to the ground?
Yes.....he might fall in the sea or a river. ;) ;D
-
The theoretical extra highlight headroom to which you refer is not attainable because the noise in the image is much greater than the quantization step (see Emil Martinec (http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html)).
Thank you for that link, it was an interesting read.
-h
-
Bill, fortunately the so-called linear DNG images don't actually have to be stored using linear integer representation. They can be stored using non-linear encoding if desired. (There are also some extensions in the works that will provide extra flexibility.)
-
I wouldn’t bet on it, and I wouldn’t want to be the guy doing the experiment, but ya never know. :-\
If I lived in the 1800’s and found someone willing to bet that Newton’s theory of gravity was wrong, I’d have bet everything I own. Einstein would have put me in the poor house! Ya never know.
Off topic for the thread, but a good example of "ya never know": "Physics rule broken? European scientists claim neutrinos measured traveling faster than light." http://www.washingtonpost.com/world/europe/absolute-speed-barrier-broken-cern-claims-neutrinos-clocked-traveling-faster-than-light/2011/09/22/gIQA5Sn9nK_story.html?hpid=z2