Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: ChuckZ on December 11, 2009, 01:58:31 pm
-
Per a posting I read somewhere recently, they said that even if you are shooting raw, the camera histogram is based on the result you would get if you are shooting jpg. They went on to say that you should setup your camera for the color space with the widest gamut (AdobeRGB for my D300) and set the contrast at a minimum (in Set Picture Control in D300) to get a histogram that most closely represents the raw image. I just returned from Death Valley and am currently processing the shots. Surely enough, while the histogram in Lightroom is not exactly what I see on the camera display, it is pretty close and definitly closer than when I was using the camera defaults.
-
Per a posting I read somewhere recently, they said that even if you are shooting raw, the camera histogram is based on the result you would get if you are shooting jpg. They went on to say that you should setup your camera for the color space with the widest gamut (AdobeRGB for my D300) and set the contrast at a minimum (in Set Picture Control in D300) to get a histogram that most closely represents the raw image. I just returned from Death Valley and am currently processing the shots. Surely enough, while the histogram in Lightroom is not exactly what I see on the camera display, it is pretty close and definitly closer than when I was using the camera defaults.
1. Raw images have no color space.
2. Yes, when shooting raw, the jpg on the back of camera is what the histo is based on, and this jpg reflects the set up parameters you chose.
3. However, the actual raw image has no color space, nor will it necessarily reflect the look of the corresponding jpg.
4. Raw's will have a bit wider DR, so what looks blocked of blown on the jpg, may not necssarily be the case for the raw file.
5. I would nutralize/turn off all jpg effects so that it looks closer to how the corresponding raw looks.
6. You can set the camera to any color space you want, but the raw has no color space, so it don't matter.
-
I leave my 1DS MKIII in all neutral or "standard" and the histogram is very close in all ways.
-
1. Raw images have no color space.
2. Yes, when shooting raw, the jpg on the back of camera is what the histo is based on, and this jpg reflects the set up parameters you chose.
3. However, the actual raw image has no color space, nor will it necessarily reflect the look of the corresponding jpg.
4. Raw's will have a bit wider DR, so what looks blocked of blown on the jpg, may not necssarily be the case for the raw file.
5. I would nutralize/turn off all jpg effects so that it looks closer to how the corresponding raw looks.
6. You can set the camera to any color space you want, but the raw has no color space, so it don't matter.
I don't believe anything in the original post implied that the raw has a color space. What it did say is to offset the effect of #4 in your statement, the OP felt that setting the camera to aRGB and using a preset with low contrast provided a histogram that was closer to what the raw histogram would look like. I believe many fiddle with the jpeg settings in an effort to achieve this goal. The biggest challenge of using ETTR is knowing when you are actually clipping the raw data.
-
1. Raw images have no color space.
That assertion is questionable. Thomas Knoll and other experts do regard raw files as having a color space.
See this thread (http://luminous-landscape.com/forum/index.php?showtopic=22471) for a spirited discussion on this topic.
-
I also now have my 5DII set to Neutral, with the contrast as low as possible and the colorspace set to aRGB. BIG IMPROVEMENT! It doesn't affect the RAW file in any way, shape or form, but what it does do is give you a camera histogram that shows a much more accurate representation of how much latitude you have. When I had the contrast setting at the default setting, the histogram would frequently show blown highlights when there weren't any. Which, of course, led me to underexpose unnecessarily.
Good exposure is critical. These settings really do help ensure it.
-
If you want your camera's histogram become as close as possible to the RAW data, all those tricks you mention have much less effect than neutralising the camera white balance, which is the main difference between JPEG and RAW exposure.
More info in this thread: Camera's histogram reliable to the RAW data (http://luminous-landscape.com/forum/index.php?showtopic=22250) and in this article: UNIWB. MAKE CAMERA DISPLAY RALIABLE (http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm).
The article appeared some weeks ago in LL was a bit surprising to me, it seems Michael doesn't look too carefully at the discusions hold in his own forum (I know I know, you are very busy Michael).
Regards
-
I don't believe anything in the original post implied that the raw has a color space. What it did say is to offset the effect of #4 in your statement, the OP felt that setting the camera to aRGB and using a preset with low contrast provided a histogram that was closer to what the raw histogram would look like. I believe many fiddle with the jpeg settings in an effort to achieve this goal. The biggest challenge of using ETTR is knowing when you are actually clipping the raw data.
Read the OP's initial post again...the implication that raw has a color space is there.
I think it would be best for him to nuetralize all effects that effect the jpg so that the resulting histogram on the back will more closely match the corresponding raw image.
-
That assertion is questionable. Thomas Knoll and other experts do regard raw files as having a color space.
See this thread (http://luminous-landscape.com/forum/index.php?showtopic=22471) for a spirited discussion on this topic.
I read that thread, and I will continue to maintain that raw files do not in fact have color space.
A raw image is just that. Raw, lacking post processing by the camera.
A color space is just a multi-plier (or shift) to the digital data during conversion in PhotoShop (or other processing program).
You can shoot in all the color spaces your camera provides, but doing so will not change the actual digital raw data when shooting the same exact composition for each color space.
That thread starts off asserting that a raw image is not color. This is not true. If that were true then by that flawed logic, jpg files are not color either.
All digital pictures provide digital values that represent the three primary colors, and mixtures of them.
Just because someone wrote a book should not guarentee they know what they're talking about.
Raw has no color space.
-
Read the OP's initial post again...the implication that raw has a color space is there.
No, it isn't. I don't think you understood what he wrote. Most cameras have an option for color space selection, and he mentioned setting it to Adobe RGB. He never said anything about it affecting the RAW data, what he said was that the in-camera histogram now better reflects the RAW data. And this is true, because the in-camera colorspace setting affect JPEG's, and the in-camera histogram is based on the camera's JPEG engine and the processing it does.
-
I will continue to maintain that raw files do not in fact have color space.
A raw image is just that. Raw, lacking post processing by the camera
Yes, it is raw, more or less, depending on the camera and options). What exactly does that have to do with color space?
A color space is just a multi-plier (or shift) to the digital data during conversion in PhotoShop (or other processing program)
This is not so, though it describes one possible way of the white balance application.
You can shoot in all the color spaces your camera provides, but doing so will not change the actual digital raw data when shooting the same exact composition for each color space
That's right. The color space is always the same, namely the color space of that camera.
That thread starts off asserting that a raw image is not color
The raw image IS color, but not RGB.
All digital pictures provide digital values that represent the three primary colors, and mixtures of them
This is incorrect. Examples: CMYK, Lab, and raw images.
Raw has no color space
I'm afraid you need some more effort to explain, why not.
-
If you want your camera's histogram become as close as possible to the RAW data, all those tricks you mention have much less effect than neutralising the camera white balance, which is the main difference between JPEG and RAW exposure.
More info in this thread: Camera's histogram reliable to the RAW data (http://luminous-landscape.com/forum/index.php?showtopic=22250) and in this article: UNIWB. MAKE CAMERA DISPLAY RALIABLE (http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm).
The article appeared some weeks ago in LL was a bit surprising to me, it seems Michael doesn't look too carefully at the discusions hold in his own forum (I know I know, you are very busy Michael).
Regards
Question about the method detailed in the above article. From what I understand from a practical point of view you need to take the file and load it up into the camera as a custom WB using the usual method. This does mean of course that the file needs to be on each and every card (replaced after every format) and you would have to reload the custom WB from the card every time you put a new card in. As most pro's do not use just one large card (I shoot weddings with multiple smaller cards) nor do they have time to replace a file after every format or remember to upload it to the camera system, I would suggest that practically this method might work best with a 1 series where the SD card slot is either not used (the file can reside there permenantly) or where a large SD card acts as backup to the main card which would be used more frequently thereby having the file remain available to the camera throughout a long shooting session even when the main card is being replaced frequently.
I'm going to try the generated file on my 5D, question before I begin, often in cases when using severe filtration the correction available in ACR or C1 is limited. I have files all shot under a blue awning which are impossible to correct sufficiently. Heck even not having the ability to change the WB +/- 1500K could be a real problem in real world use. Do you find that to be an issue using this custom WB image you've invented?
Many thanks
-
you need to take the file and load it up into the camera as a custom WB using the usual method. This does mean of course that the file needs to be on each and every card
No, you just need to load that particular WB on your custom WB bank in the camera. After that the file can be deleted from the card, and the UniWB can be used no matter which card you are using just by selecting that custom WB.
Anyway I think UniWB is perhaps not the best method for a wedding photographer. You are using controlled lighting (flash) and you need to be very quick so there is no time for perfect ETTR. Also perhaps some of the guests want to see the images straight from your camera and the green tint can be confusing.
I'm going to try the generated file on my 5D, question before I begin, often in cases when using severe filtration the correction available in ACR or C1 is limited. I have files all shot under a blue awning which are impossible to correct sufficiently. Heck even not having the ability to change the WB +/- 1500K could be a real problem in real world use. Do you find that to be an issue using this custom WB image you've invented?
No, the UniWB is irrelevant at RAW development time, it is just a _capture_ time trick. It doesn't alter in any way the RAW data produced, only the WB metadata. You are not filtering the image at all.
I.e. if you shoot a scene at 1/200@f/8 with UniWB and next with the Daylight WB preset, the RAW data will be _exactly_ the same. Your image will open green in ACR just because the RAW developer tries to mimic that 'strange' WB you applied on the camera, but as soon as you set ACR's Tungsten WB the resulting image will be undistinguishable from the shot done with the Daylight preset once Tungesten is set in ACR. Try it.
Regards
-
I read that thread, and I will continue to maintain that raw files do not in fact have color space.
That thread starts off asserting that a raw image is not color. This is not true. If that were true then by that flawed logic, jpg files are not color either.
All digital pictures provide digital values that represent the three primary colors, and mixtures of them.
Just because someone wrote a book should not guarentee they know what they're talking about.
Raw has no color space.
Lovel,
You need to read and think critically before engaging your mouth. No way did the OP suggest that the file had a color space. He merely pointed out that one had the option of rendering the raw files into various color spaces supported by the camera's JPEG engine.
The thread you referenced did start off suggesting that the raw file contained color information, the same as an sRGB rendered image. Others maintained that the raw was monochrome and not color. Color itself is a perception that takes place in the eye and brain. The camera merely records tristimulus information that relates to color.
You sum up by reiterating without supporting facts or reasoning that raw has no color space and denigrating authors of unnamed books. Since your thinking is rambling, noncritical, and unsupported by any reasoning or factual data, I could care less whether you consider that raw has a color space or not.
-
No, you just need to load that particular WB on your custom WB bank in the camera. After that the file can be deleted from the card, and the UniWB can be used no matter which card you are using just by selecting that custom WB.
Anyway I think UniWB is perhaps not the best method for a wedding photographer. You are using controlled lighting (flash) and you need to be very quick so there is no time for perfect ETTR. Also perhaps some of the guests want to see the images straight from your camera and the green tint can be confusing.
No, the UniWB is irrelevant at RAW development time, it is just a _capture_ time trick. It doesn't alter in any way the RAW data produced, only the WB metadata. You are not filtering the image at all.
I.e. if you shoot a scene at 1/200@f/8 with UniWB and next with the Daylight WB preset, the RAW data will be _exactly_ the same. Your image will open green in ACR just because the RAW developer tries to mimic that 'strange' WB you applied on the camera, but as soon as you set ACR's Tungsten WB the resulting image will be undistinguishable from the shot done with the Daylight preset once Tungesten is set in ACR. Try it.
Regards
My bad, I didn't realise the camera retained the data indefinately as the custom WB, I thought it had to have the file to reference otherwise it would disappear. Does make life easier.
I've personally always calibrated the jpg preview (and by definition the histo) to match my ACR defaults. Gives me a better idea of what I have. I don't specifically shoot ETTR at weddings though although I tend towards hot rather than underexposed these days given the amount of information I can trust ACR to retain in the dresses, using the dodge and burn tools, etc. Didn't used to be like that in the older raw converter days when you held the whites in camera and tried to ignore the noise as you pulled the faces back up.
I did some trial shots, the histogram still looked nothing like the one in ACR and I couldn't see what was going on with the screen as well, bit confusing. Is there a specific colour space I need to use? Will the histo only match ProPhoto and 16 bit for example? That would be useless for me, I work in srgb and 8 bit because that is what my output space and depth is. Haven't got time when processing hundreds of images for the lab for colourshifts or suprise blown highlights when I convert down a colour space and bit depth at the end of the processing. As you said, this isn't really for me...
-
I tend towards hot rather than underexposed these days given the amount of information I can trust ACR to retain in the dresses
This is a common mistake. ACR (or any RAW developer) doesn't retain any information, it just displays the information present in the RAW file. There is no merit at all in doing that. The general belief is that RAW can help us to _recover_ information. Recover? where and when was that information lost that we have to recover it now?.
RAW doesn't recover anything, it was camera's JPEG that eliminated a lot of information from the RAW, specially highlights information and basically because of the white balance process. Opening the RAW file in a RAW developer doesn't mean recovering anything, it just means _not losing_ what it was lost when the camera built the JPEG file.
That is why UniWB makes sense: until camera manufacturers take into account RAW shooters, and decide to help us providing tools that allow to know how good resulted the data collected in the RAW file (for example providing RAW histograms in the camera, instead of cooked overexposed JPEG histograms), UniWB can help to make camera histograms get closer (never exactly the same) to a RAW histogram.
I did some trial shots, the histogram still looked nothing like the one in ACR and I couldn't see what was going on with the screen as well, bit confusing.
You seem to be believing the ACR histogram is the RAW histogram. It is not, ACR's histogram is a cooked version of the RAW histogram, i.e. a cooked version of what you really have in the RAW file. Among others the processes of white balance, colour interpolation and output colour profile conversion took place. The good thing of ACR is that it allows you to set a negative exposure compensation that brings again to life all that information that the white balance (which is a positive exposure compensation for each indivigual RGB channel) was blowing.
So ACR is OK to find out how much information the RAW file really has, but still it is not a RAW histogram. So don't expect camera's histogram + UniWB be close to what you will see in ACR. The goal of UniWB is to make camera's histogram closer to a genuine RAW histogram.
-
This does mean of course that the file needs to be on each and every card (replaced after every format) and you would have to reload the custom WB from the card every time you put a new card in
This is a different issue, but
1. one can make a file read-only on the card,
2. there is no reason to format the card every time, erase all is good enough - and this does not delete the read only files.
Thus the WB template image can remain on the card for ever. This is how I am doing it, though I have to admit, that I don't really need it, for I always use that WB template.
-
2. there is no reason to format the card every time, erase all is good enough - and this does not delete the read only files.
While technically true, (quick) format takes one second on Canon cameras, but delete all images takes a lot longer.
Also, after long use file fragmentation might become a problem and impact write/read speeds, but I have a feeling that is might be mostly an academic problem on flash cards.
-
Per a posting I read somewhere recently, they said that even if you are shooting raw, the camera histogram is based on the result you would get if you are shooting jpg. They went on to say that you should setup your camera for the color space with the widest gamut (AdobeRGB for my D300) and set the contrast at a minimum (in Set Picture Control in D300) to get a histogram that most closely represents the raw image. I just returned from Death Valley and am currently processing the shots. Surely enough, while the histogram in Lightroom is not exactly what I see on the camera display, it is pretty close and definitely closer than when I was using the camera defaults.
ChuckZ,
Rather in engage in the theocratic argument other members are having I just want to point out that you are correct in what you are doing and seeing.
-
Rather in engage in the theocratic argument other members are having I just want to point out that you are correct in what you are doing and seeing.
As I too have posted in this thread, I feel shocked by your assessment, that the argumentation here is theocratic. Aside from that, ChuckZ has thoroughly misunderstood the entire subject (like you did), because his intention was to see a histogram that most closely represents the raw image; but what he sees in Lightroom is the histogram of the JPEG image created by either the camera or by LR.
Anyway, Happy Christmas.
-
Per a posting I read somewhere recently, they said that even if you are shooting raw, the camera histogram is based on the result you would get if you are shooting jpg.
Surely enough, while the histogram in Lightroom is not exactly what I see on the camera display, it is pretty close and definitly closer than when I was using the camera defaults.
WB changes the results more than any other setting. Research uni-WB and install it on your D300 for the most accurate histogram and determining whether or not you are attaining optimum exposures.
-
Per a posting I read somewhere recently, they said that even if you are shooting raw, the camera histogram is based on the result you would get if you are shooting jpg.
Surely enough, while the histogram in Lightroom is not exactly what I see on the camera display, it is pretty close and definitly closer than when I was using the camera defaults.
WB changes the results more than any other setting. Research uni-WB and install it on your D300 for the most accurate histogram and determining whether or not you are attaining optimum exposures.
While the WB does have a huge effect on the histogram, one should not ignore the tone curve, either in the JPEG review or the rendering in ACR or Lightroom. With most subjects containing a balance of colors, the Green component of the camera RGB histogram will give the best estimate of ETTR exposure, since the green white balance multiplier is unity, while the red and blue multipliers are greater than unity. If the subject contains saturated red and blue (such as in pictures of some flowers), these channels may appear blown in the camera histogram while they are intact in the raw file because the red or blue multiplier results in overflow in those channels.
A hot camera tone curve may also cause the histogram to be too far to the right, and this effect can be more important than WB in some cases. For example, here are snapshots taken in my back yard with the Nikon D3 using the Standard Picture Control (normal contrast and saturation) rendering into Adobe RGB with daylight WB and UniWB. In the daylight WB shot, the exposure appears reasonably to the right. Since it was overcast, the picture is blueish. The UniWB histogram gives a better idea of the raw values for the red and blue channels, but this information does not aid in ETTR since the red and blue are to the left of green.
Daylight WB:
[attachment=18916:DaylightWB.jpg]
UniWB:
[attachment=18917:UniWB.jpg]
The ACR histogram with default settings appears reasonably exposed:
[attachment=18918:ACR_daylight.png]
However, if we look at the raw histogram with Rawnalize, the image is about one stop underexposed:
[attachment=18919:Rawnalize.png]
Both the camera and ACR are using a hot tone curve. In ACR an exposure offset of +0.5 stops is used, moving the histogram to the right by this amount. To get an ACR histogram better reflecting the raw data, one should set the tone curve to linear (all sliders on the main tab set to zero and the point curve set to linear) and look at the UniWB shot using -0.5 EV expose to cancel out the exposure offset that ACR uses. This histogram is still to the right of the Rawnalize histogram, since the ACR is using a gamma of 2.2.
[attachment=18920:ACR_as_shot.png]
For a better comparison, we need to convert the gamma to 1.0 and look at the histogram in Photoshop:
[attachment=18921:PS_linea...omposite.png]
-
While the WB does have a huge effect on the histogram, one should not ignore the tone curve,
Naturally, settings should be set to zero and a linear curve installed. I wasn't arguing that they were mutually exclusive.
A hot camera tone curve may also cause the histogram to be too far to the right, and this effect can be more important than WB in some cases. For example, here are snapshots taken in my back yard with the Nikon D3 using the Standard Picture Control (normal contrast and saturation) rendering into Adobe RGB with daylight WB and UniWB. In the daylight WB shot, the exposure appears reasonably to the right. Since it was overcast, the picture is blueish. The UniWB histogram gives a better idea of the raw values for the red and blue channels, but this information does not aid in ETTR since the red and blue are to the left of green.
It does aid some since you have about a half a stop more headroom indicated in the histogram (green channel's distance from the right side) in the uni-WB shot compared to the Daylight WB shot (where the blue channel is all the way to the right). Lowering contrast, saturation, and even sharpening (sharpening can add as much as a third of stop) all are important to getting the histogram as close as possible to accurately reflecting the RAW data. Also, the Standard Picture Control color profile is problematic, I use Neutral Picture Control on my D300.
Both the camera and ACR are using a hot tone curve. In ACR an exposure offset of +0.5 stops is used, moving the histogram to the right by this amount. To get an ACR histogram better reflecting the raw data, one should set the tone curve to linear (all sliders on the main tab set to zero and the point curve set to linear) and look at the UniWB shot using -0.5 EV expose to cancel out the exposure offset that ACR uses. This histogram is still to the right of the Rawnalize histogram, since the ACR is using a gamma of 2.2.
Generally with moderate contrast scenes, if you are not applying negative EC in the RAW converter and/or reducing the Brightness slider, you are working with an underexposed NEF file.
-
Naturally, settings should be set to zero and a linear curve installed. I wasn't arguing that they were mutually exclusive.
You can set a linear tone curve with ACR, but with the D3 camera there is no way to set a linear tone curve on the camera. At the lowest contrast (minus 3), the camera still applies a strong S curve to the data. A linear TRC would appear very flat and unattractive. Here is the lowest contrast TRC on the D3. Note that the three quarter tones are lifted upwards towards clipping.
[attachment=18923:D3_Stouf...d_Step_3.png]
Lowering contrast, saturation, and even sharpening (sharpening can add as much as a third of stop) all are important to getting the histogram as close as possible to accurately reflecting the RAW data.
Lowering the contrast does not affect the extreme highlights, but it does affect the quarter tones and three quarter tones. If you change the contrast and the highlights are affected, you are looking at the upper three quarter tones and not the highlights themselves. As shown in the above characteristic curve at the weakest contrast for the D3, the three quarter tones are not far from the clipping point. The effect of a strong contrast curve in ACR is shown below. The three quarter tones are lifted and the quarter tones are lowered, but the clipping point of the highlights and blacks is not affected. At the clipping point, input of 255 equals the output of 255. Sharpening can lift the highlights above clipping along a transition, but would not affect flat areas such as shown here. If your sharpening halos lift the highlights by 0.3 EV, you are likely oversharpening.
[attachment=18929:05_AC.PNG]
Generally with moderate contrast scenes, if you are not applying negative EC in the RAW converter and/or reducing the Brightness slider, you are working with an underexposed NEF file.
With ACR that is false if you are using the appropriate baseline offset and a linear tone curve. Here is a Stouffer wedge with highlight clipping in the green channel at step 4 as shown by Rawnalize:
[attachment=18924:O5_rawValues.png]
And here is the ACR view with the BaselineExposure offset of +0.5 taken into account by setting exposure to -0.5 EV and setting the TRC to linear. The clipping point is dead on.
[attachment=18925:05_nominal_linear.png]
Setting the contrast to Minus 50 and Plus 100 does not change the clipping point.
Contrast Minus 50:
[attachment=18926:05_contr_minus90.png]
Contrast +100:
[attachment=18927:05_contr_plus100.png]
I think you need to study the behavior of ACR and your camera JPEG engine in a bit more detail. I will leave it to you to demonstrate the behavior of the camera JPEG engine. My own studies indicate that the D3 uses a hot tone curve.
-
You can set a linear tone curve with ACR, but with the D3 camera there is no way to set a linear tone curve on the camera. At the lowest contrast (minus 3), the camera still applies a strong S curve to the data. A linear TRC would appear very flat and unattractive.
You are mistaken. You can apply a linear curve to the D3 using the Picture Control editing utility in NX. Many of us have such a linear curve installed in our Nikon cameras.
Generally with moderate contrast scenes, if you are not applying negative EC in the RAW converter and/or reducing the Brightness slider, you are working with an underexposed NEF file.
With ACR that is false if you are using the appropriate baseline offset and a linear tone curve.
ACR is not the only RAW converter, nor would I consider it the best; in fact, it is my least used choice for NEF files. Also, you refer to "offset and linear tone curve" which I presume means you are not using the Default ACR settings, which do require adjustment on optimally exposed NEF files:
(http://photos.imageevent.com/tonybeach/mypicturesfolder/sharing//Untitled-3%20copy.jpg)
And here is the ACR view with the BaselineExposure offset of +0.5 taken into account by setting exposure to -0.5 EV and setting the TRC to linear. The clipping point is dead on.
Then you agree that setting EC to a negative value is appropriate, even with ACR.
Setting the contrast to Minus 50 and Plus 100 does not change the clipping point.
Changing contrast in the Picture Controls changes the histogram; that is a distinctly separate issue from how to adjust the files using ACR (or any other RAW converter). This was supposedly a discussion about optimal exposure settings in-camera, not about RAW conversion settings.
I think you need to study the behavior of ACR and your camera JPEG engine in a bit more detail. I will leave it to you to demonstrate the behavior of the camera JPEG engine. My own studies indicate that the D3 uses a hot tone curve.
I'm intimately familiar with the JPEG engine in Nikon's DSLRs, and how that effects the histograms derived therefrom. I'm really not interested in doing a tutorial right now (maybe later), you might want to Google this topic and research it some more. Here for instance, is a quick explanation of how to install the linear curve on your D3 (or other Nikon DSLR that uses Picture Controls): http://www.clcarder.com/Tutorials/NikonCur...structions.html (http://www.clcarder.com/Tutorials/NikonCurveInstallationInstructions.html)
-
You are mistaken. You can apply a linear curve to the D3 using the Picture Control editing utility in NX. Many of us have such a linear curve installed in our Nikon cameras.
Yes, I am familiar with the Picture Control Utility. Thanks for the reference. However, concerning its effect, see below. Fortunately, one does not have to use NX to access the Picture Control Utility (See here (http://forums.dpreview.com/forums/read.asp?forum=1039&message=26531692)).
ACR is not the only RAW converter, nor would I consider it the best; in fact, it is my least used choice for NEF files. Also, you refer to "offset and linear tone curve" which I presume means you are not using the Default ACR settings, which do require adjustment on optimally exposed NEF files:
Opinions vary on NX and NX2. While it produces excellent results, the interface is awkward and it is quite slow. I had NX but did not update to NX2. When I migrated to 64 bit Win 7, I did not bother trying to install NX. I find ACR best for my needs.
Then you agree that setting EC to a negative value is appropriate, even with ACR.
Correcting for the BaselineExposure (DNG Specification (http://www.adobe.com/products/dng/pdfs/dng_spec.pdf)) is not negative exposure compensation. See the PDF for an explanation of BaselineExposure. The proper setting for exposure without compensation produces a rendered pixel value for the highlights of 255 in 8 bit notation. With ACR one must use -0.5 EV and a linear TRC to get the proper highlight value, i.e. a 14 bit raw value at saturation (around 16000) is 255 in the rendered file. Negative exposure correction, properly speaking, is highlight recovery.
Changing contrast in the Picture Controls changes the histogram; that is a distinctly separate issue from how to adjust the files using ACR (or any other RAW converter). This was supposedly a discussion about optimal exposure settings in-camera, not about RAW conversion settings.
A contrast curve in a raw converter works the same whether one is using the EXPEED JPEG engine, ACR, or NX2. I used ACR because that is what I use for my work.
I'm intimately familiar with the JPEG engine in Nikon's DSLRs, and how that effects the histograms derived therefrom. I'm really not interested in doing a tutorial right now (maybe later), you might want to Google this topic and research it some more. Here for instance, is a quick explanation of how to install the linear curve on your D3 (or other Nikon DSLR that uses Picture Controls):
Perhaps not so familiar as you think. The "Linear Tone Curve" is applied on top of the base Picture Control from which it is derived. For example, here is a linear tone curve derived from the Neutral Picture Control:
[attachment=18933:PictureControl.png]
I used this "Linear" curve to photograph the Stouffer wedge and rendered to JPEG in the camera with this setting.
[attachment=18934:ViewNX_SCR.png]
And here is the resulting tone curve as shown by Imatest. It is hardly linear, but is merely the Neutral Picture control with no adjustments.
[attachment=18935:D3_0008_Step_3.png]
-
Yes, I am familiar with the Picture Control Utility. Thanks for the reference. However, concerning its effect, see below. Fortunately, one does not have to use NX to access the Picture Control Utility (See here (http://forums.dpreview.com/forums/read.asp?forum=1039&message=26531692)).
NX is the term Nikon uses for all of their NEF software, so it can be ViewNX or CaptureNX.
I'm intimately familiar with the JPEG engine in Nikon's DSLRs, and how that effects the histograms derived therefrom. I'm really not interested in doing a tutorial right now (maybe later), you might want to Google this topic and research it some more. Here for instance, is a quick explanation of how to install the linear curve on your D3 (or other Nikon DSLR that uses Picture Controls)...
Perhaps not so familiar as you think.
I'm plenty familiar with it. Enough to know how to install a linear curve, create my own uni-WB, and to set the Picture Controls to accurately reflect the RAW data. I think it is ironic that for all your examples and arguing, your test shot is underexposed and mine is not.
-
I'm plenty familiar with it. Enough to know how to install a linear curve, create my own uni-WB, and to set the Picture Controls to accurately reflect the RAW data. I think it is ironic that for all your examples and arguing, your test shot is underexposed and mine is not.
In my example, I made no attempt to expose fully to the right, but merely was interested in comparing the camera histogram to the raw histogram and used the exposure indicated by matrix metering.
In your example, the Rawnalize histogram demonstrates one stop underexposure and yet you claim to have proper exposure. The raw histogram and the rendered histograms do not match.
(http://photos.imageevent.com/tonybeach/mypicturesfolder/sharing//Untitled-3%20copy.jpg)
-
Has anyone tried ETTR with a picture style, which contains a custom curve, roughly the inverse of the sRGB mapping (or Adobe RGB, not a big difference from this point), in order to get closer to the raw histograms? I did try that with my Canon 40D and I was not happy about the result, but honestly, I forgot in the meantime why. Of course, the color space conversion can not be disabled. It would be nice to have the option of loading color space conversion matrixes, like in DNG.
-
In your example, the Rawnalize histogram demonstrates one stop underexposure and yet you claim to have proper exposure.
(http://photos.imageevent.com/tonybeach/mypicturesfolder/sharing/Untitled-2%20copy.jpg)
At 1/3 more of a stop some highlights would become unrecoverable, at 2/3 of a stop the colors in the sky, the tree, and the background highlights would shift and become inaccurate.
The raw histogram and the rendered histograms do not match.
There is no gamma applied to the RAW histogram.
-
At 1/3 more of a stop some highlights would become unrecoverable, at 2/3 of a stop the colors in the sky, the tree, and the background highlights would shift and become inaccurate.
What does this Rawnalize screen capture have to do with the previous one where the highlights were 1 stop from clipping?
There is no gamma applied to the RAW histogram.
Of course, the gamma affects the scaling of the X axis as I demonstrated above, but one stop is still one stop. One must take the scaling into account when evaluating histograms. Shown below are two histograms from Photoshop demonstrating a the green channel of Stouffer wedge rendered at gamma 2.2 and gamma 1.0. Unfortunately, the x-axis is not scaled. The patches vary by 0.1 density units, so three of them is about 1 f/stop. In the gamma 2.2, the steps are nearly equal on the right but draw together on the left. With gamma 1.0 (bottom) the steps are expanded on the right and squished together on the left (like the scaling on a slide rule).
[attachment=18941:Iris_Gammas.png]
In my opinion, the best histogram has a log scale as shown below by Guillermo's Histogrammar. A log scale is more perceptually uniform and photographers think in f/stops. The program is free, but if you use it regularly, you should make a donation via Paypal.
[attachment=18942:histogrammar.png]
-
What does this Rawnalize screen capture have to do with the previous one where the highlights were 1 stop from clipping?
That screenshot was from the same file as the one used in the previous screenshot, and shows that the highlights are not one stop from clipping, some of them are as little as 1/6 of a stop from clipping, and most of them are within 2/3 of a stop of clipping.
The irony here is that if the shot was exposed a full stop more to the right (which would have disastrous results throughout the image), you would need to set even more negative EC in the RAW converters than what I showed. This proves my point that if you are not using negative EC for relatively low contrast scenes then you are underexposing them.
-
Has anyone tried ETTR with a picture style, which contains a custom curve, roughly the inverse of the sRGB mapping (or Adobe RGB, not a big difference from this point), in order to get closer to the raw histograms? I did try that with my Canon 40D and I was not happy about the result, but honestly, I forgot in the meantime why. Of course, the color space conversion can not be disabled. It would be nice to have the option of loading color space conversion matrixes, like in DNG.
Gabor,
I devised a custom InverseGamma curve using calculated raw pixel values from a Stouffer wedge and using the measured valued in an sRGB rendering.
Calculated values:
[attachment=18949:Excel.gif]
Scatter Plot:
[attachment=18950:InverseGammaExcel.png]
Resulting Picture Control
[attachment=18951:PictureC...rseGamma.png]
Imatest Plot of Inverse Gamma conversion. The results are approximately linear with a gamma of 1. The curve needs more work. The Nikon utility is rather difficult to work with and I need more practice. The exercise disproves Tony Beach's theory of a linear tone curve and demonstrates that the resulting picture control tone curve is on top of the tone curve from which it was derived.
[attachment=18952:D3_0008_Step_3.png]
-
Imatest Plot of Inverse Gamma conversion. The results are approximately linear with a gamma of 1. The curve needs more work. The Nikon utility is rather difficult to work with and I need more practice. The exercise disproves Tony Beach's theory of a linear tone curve and demonstrates that the resulting picture control tone curve is on top of the tone curve from which it was derived.
Totally boring and frankly irrelevant. The primary issue for the purposes of ETTR is not where the other values appear, but where the highlights are. Besides that, who cares? I don't. Can you show that the non-linearity you are fixated on is caused by the Picture Controls, or that it is actually related to how the sensor captures the data? [Never mind, I don't care about that either. After all is said and done, we do not create photographs with perfect linearity -- unless all we care about is accurately recording a Stouffer wedge -- so I for one have better things to ponder than this digression from the OP.
-
Bill,
regarding non-linearity: Iliah, who knows of Nikon cameras' behavior much more than I do, wrote somewhere, that the raw data requires some channel specific offset to make it linear. You can easily check this in Rawnalyze on the Stouffer wedges: start on a very bright strip, record the intensity values (the AI values, as the distance from clipping in EV), and now go lower and lower by three strips. The decrease in intensity is quote consistent down to the dark strips, but when it gets close to zero, the differences become quite wild, even in 1/3 steps. Unfortunately, I don't know where/how the required offsets are recorded in the NEF MakerNote.
-
Bill,
regarding non-linearity: Iliah, who knows of Nikon cameras' behavior much more than I do, wrote somewhere, that the raw data requires some channel specific offset to make it linear. You can easily check this in Rawnalyze on the Stouffer wedges: start on a very bright strip, record the intensity values (the AI values, as the distance from clipping in EV), and now go lower and lower by three strips. The decrease in intensity is quote consistent down to the dark strips, but when it gets close to zero, the differences become quite wild, even in 1/3 steps. Unfortunately, I don't know where/how the required offsets are recorded in the NEF MakerNote.
Gabor,
I followed your advice and did use Rawnalize to read the raw pixel values of the green channel of the the raw file of a selected image of the step chart and the results are shown. Step one is clipped in the raw file and step 3 is at clipping in the aRGB JPEG. I set the camera to record both raw and JPEG.
[attachment=18959:Excel2.gif]
[attachment=18960:RawVsStep.gif]
[attachment=18962:Raw14_vs_aRGB.gif]
The clipping in the raw is confirmed by Rawnalize:
[attachment=18961:RawnalizeScrCap.png]
I went as far as possible in the shadows until I could no longer see the steps on screen. As you can see, I did not recover the shadow portion of the S curve.
I constructed an inverse gamma curve in the Picture Control utility with slightly better results than before. Since the JPEG is at clipping in step 3 and the Picture Control curve appears to be applied on top of the base picture control, I can devise no method to show step 2 of the wedge at 255, because the data are already clipped. Any suggestions?
-
I followed your advice and did use Rawnalize to read the raw pixel values of the green channel of the the raw file of a selected image of the step chart and the results are shown
What is the Raw_255 value in the chart?
I went as far as possible in the shadows until I could no longer see the steps on screen. As you can see, I did not recover the shadow portion of the S curve
Why don't you increase the intensity (add 3-4 EV)? The raw clipping shows, how far it is reasonable to go: when there are many clipped pixels, the result can not (or should not) be in line with the other strips, because the reduction between such steps is less than 1/3 EV.
Since the JPEG is at clipping in step 3 and the Picture Control curve appears to be applied on top of the base picture control, I can devise no method to show step 2 of the wedge at 255, because the data are already clipped. Any suggestions?
Simply reducing the intensity by negative "exposure" adjustment of ACR?
-
What is the Raw_255 value in the chart
Raw_255 is merely the raw value expressed in 8 bit (0..255) notation. Raw_255 = Raw value/16383 * 255.
-
The irony here is that if the shot was exposed a full stop more to the right (which would have disastrous results throughout the image), you would need to set even more negative EC in the RAW converters than what I showed. This proves my point that if you are not using negative EC for relatively low contrast scenes then you are underexposing them.
That is often the case, but whether or not negative exposure is required for proper ETTR exposure of short scale subjects depends on the distribution of tones within the subject. For the purpose of demonstration, I will use a Kodak Q14 chart. With a full scale image, exposing for mid tones gives proper exposure with the highlights just short of clipping. No negative exposure correction is needed; the -0.5 EV in ACR is to correct for the BaselineExposure offset that ACR uses for this camera.
[attachment=19029:FullScale.png]
With a short scale subject centered on the midtones and using normal exposure, there would be room on either end of the histogram. Optimum ETTR exposure would require an increase of exposure to place the highlights just short of clipping; one would then have to use negative exposure compensation in the raw converter to bring the tones down to where they should be.
[attachment=19031:MidKey.png]
With a high key image proper ETTR exposure would again place the highlights just short of clipping. However, in this case no negative exposure compensation would be necessary since the highlights are already where they should be.
[attachment=19033:HighKey.png]
-
That is often the case,
In the example I offered it was the case. I was already over a stop past what the meter was indicating as a "proper" exposure for that scene. That was for Center-weighted metering; and while it may well have been closer using Matrix metering, I don't really care because I simply avoid Matrix metering for several reasons. What is scary about this is that if a photographer relies on Nikon's metering and then goes to pull the shadow detail up, they will bring up an extra stop of noise and hit the practical limits of the sensor's DR that much sooner.
but whether or not negative exposure is required for proper ETTR exposure of short scale subjects depends on the distribution of tones within the subject. For the purpose of demonstration, I will use a Kodak Q14 chart. With a full scale image, exposing for mid tones gives proper exposure with the highlights just short of clipping. No negative exposure correction is needed; the -0.5 EV in ACR is to correct for the BaselineExposure offset that ACR uses for this camera.
I still don't get your "BaselineExposure offset" argument, since you are essentially saying that you are applying negative EC to ACR's default -- this strikes me as being a semantical disagreement. Also (as I already said), there are other RAW converters, and the use of negative EC is close to universal to all of them for RAW conversions of ETTR files (again, assuming a lower contrast scene). Of course for high key photographs you will probably skip negative EC in RAW conversion.
For higher contrast scenes it is a struggle to simply keep the highlights from blowing while retaining shadow detail -- so the histogram will stretch from one end to the other even starting with ETTR. Often metering for those scenes requires no adjustment or even negative adjustment, and how you deal with it depends on whether loosing highlight detail and color accuracy is more or less important than noise in the subject (depending of course on how many stops down from the highlights that subject is).
-
I still don't get your "BaselineExposure offset" argument, since you are essentially saying that you are applying negative EC to ACR's default -- this strikes me as being a semantical disagreement. Also (as I already said), there are other RAW converters, and the use of negative EC is close to universal to all of them for RAW conversions of ETTR files (again, assuming a lower contrast scene). Of course for high key photographs you will probably skip negative EC in RAW conversion.
I would venture that most scenes are not low contrast. Also, negative exposure is also not need for images where the dynamic range of the scene is equal to that of the camera. Thus, negative EC is hardly close to universal.
BaselineExposure is defined in the Adobe DNG Specification (http://www.adobe.com/products/dng/pdfs/dng_spec.pdf) on page 32, which is quoted below:
BaselineExposure
Camera models vary in the trade-off they make between highlight headroom and shadow noise. Some leave a significant amount of highlight headroom during a normal exposure. This allows significant negative exposure compensation to be applied during raw conversion, but also means normal exposures will contain more shadow noise. Other models leave less headroom during normal exposures. This allows for less negative exposure compensation, but results in lower shadow noise for normal exposures.
Because of these differences, a raw converter needs to vary the zero point of its exposure compensation control from model to model. BaselineExposure specifies by how much (in EV units) to move the zero point. Positive values result in brighter default results, while negative values result in darker default results.
When you take a picture, you probably want mid gray (L* = 50) in the scene to be mid gray in the image. In a 2.2 gamma space, L* of 50 has a pixel value of 118. If you take a picture of a gray card (or any other uniformly reflecting surface) you would want the raw file to have a saturation of 18% (mid gray), which would represent a pixel value of 2949 in the 14 bit raw file or 739 in a 12 bit file. When you take such a picture according to the light meter reading, the resulting pixel value will reflect the calibration of the system, which includes the meter calibration and the tone curve. Nikon (and the ISO standard) specify that light meters are calibrated for 12% reflectance (see Thom Hogan (http://www.bythom.com/graycards.htm)). This allows 0.5 EV for highlight headroom. However, for ETTR exposure, one does not want much if any headroom.
To test my own D3, I took 3 shots of an 18% gray card and looked at the raw pixel values with Rawnalize.
[attachment=19043:Rawnalize.png]
The 12 bit green pixel value was 497; the saturation is 497/4095 = 12.1%, corresponding to a pixel value of 99.8 in a gamma 2.2 space, as shown by Rawnalize. The system is at spec for exposure. The sRGB pixel value of the file rendered with Capture NX and the Standard Picture Control was 149. Now that is a hot tone curve and mid gray is lighter than it should be. This is why many photographers complain that the camera "overexposes". With ACR and the Camera Standard calibration, the pixel value was 149. Using an ACR exposure of -0.5 EV gave a pixel value of 126, which is close to the desired value of 118.
[attachment=19045:ACR_scrCap.png]
Using the Adobe Standard calibration and a linear tone curve and compensating for the baseline offset with -0.5 EV, the rendered value in Adobe RGB was 99, just where it should be with a gamma 2.2 space. With a full ETTR exposure with the raw value just short of clipping, the rendered value in sRGB would also be just short of clipping with the exposure offset of -0.5 EV, but the highlights would be clipped in the rendered file without the compensation. If you didn't use the offset, the highlights would falsely appear to be blown and you would think that you overexposed and might cut back exposure in the future. If you judge exposure with ACR (or any other raw converter, including the camera JPEG) you have to know the BaselineExposure offset. Many inexperienced photographers naively assume that the raw converter histogram at default setting represents the raw histogram.
According to tests done by Bill Claff (http://home.comcast.net/~nikond70/Investigations/Sensor_Characteristics.htm), the D300 behaves in the same way as the D3. If you want 18% saturation for the metered value, you have to use an ISO of 138 rather than the nominal 200. In other words, you have to increase the exposure by 0.5 EV just as Thom described. If you use the real ISO for your Nikon camera (recent digital models), you wouldn't need the -0.5 EV compensation. Of course, I am applying a negative exposure value to the ACR default to bring the rendered values down to where they should be. The raw values were correct, so this is not correction.
You may consider the difference to be one of semantics, but a failure to understand a positive BaselineExposure may result in underexposure. Other cameras may have a negative BaselineExposure. For example, the ISO of the D70 is near the nominal value; however, I don't know what value Adobe uses for that camera. The important thing is to know the behavior of ones camera and raw converter and not be hung up on semantics.
-
I would venture that most scenes are not low contrast. Also, negative exposure is also not need for images where the dynamic range of the scene is equal to that of the camera. Thus, negative EC is hardly close to universal.
A.) Most of the scenes I shoot are, the primary exception being midday under harsh light.
B.) Redefining what I said does not change what I said. I said, "Generally with moderate contrast scenes, if you are not applying negative EC in the RAW converter and/or reducing the Brightness slider, you are working with an underexposed NEF file."
Many inexperienced photographers naively assume that the raw converter histogram at default setting represents the raw histogram.
I do not, which is why I recommend that most of the time you should set negative EC in the RAW converter to get middle gray to the middle, and I would add (with NX for example) that is in addition to lowered brightness or (with Capture One for example) no added brightness.
If you want 18% saturation for the metered value, you have to use an ISO of 138 rather than the nominal 200. In other words, you have to increase the exposure by 0.5 EV just as Thom described. If you use the real ISO for your Nikon camera (recent digital models), you wouldn't need the -0.5 EV compensation.
I use positive EC in the camera, followed by negative EC in the RAW converter; usually more of the latter than the former -- YMMV.
Of course, I am applying a negative exposure value to the ACR default to bring the rendered values down to where they should be. The raw values were correct, so this is not correction.
Compensation is not "correction" according to you whereas I see them as interchangeable in this context (you have to compensate the ACR default, which is the same as "correcting" it; the same as what happens with nearly all the other RAW converters). Spare me from this semantical debate and stop obfuscating this.
You may consider the difference to be one of semantics, but a failure to understand a positive BaselineExposure may result in underexposure.
At some point it would be useful to get back to why it is important to adjust the settings in your camera so that its histogram gives you a reasonable representation of the exposure based on the RAW data. For this, we can go way back in this thread and disregard everything that has been written since then.
Other cameras may have a negative BaselineExposure. For example, the ISO of the D70 is near the nominal value;
Right, less headroom in a D70 file, and in a D200 file too.
however, I don't know what value Adobe uses for that camera. The important thing is to know the behavior of ones camera and raw converter and not be hung up on semantics.
Ignore the RAW converter for the moment, and (as I said above) get back to the camera's settings. Once you have a well exposed file, then configure the RAW converter to take advantage of that. Something I found interesting that Iliah Borg has stated more than once is to expose for the RAW converter. For example, ACR does better than others at actual highlight recovery (which is where highlights are actually overexposed), so in some high contrast scenes where highlights need to be sacrificed know what ACR can or can't do and expose with that in mind.
-
For example, ACR does better than others at actual highlight recovery (which is where highlights are actually overexposed), so in some high contrast scenes where highlights need to be sacrificed know what ACR can or can't do and expose with that in mind.
Unless things have changed in the last versions, the only thing ACR does for highlight recovery is to produce monochrome detail in the areas where some RAW channel got clipped, by copying the information from the non-clipped channels onto the clipped ones in a neutral (R=G=B ) fashion. That is the standard behaviour expected from any decent RAW developer. ACR does not even have an option to try to recover colour, which DCRAW (a simple command line RAW developer written by a single guy) has:
Left ACR, right DCRAW:
(http://www.guillermoluijk.com/tutorial/dcraw/recuperacion.jpg)