Do we really need bracketed exposures and HDR programs anymore?

[Guillermo Luijk]

The RAW zone system reveals that the cat was not the darkest area: some parts of the furniture are even darker than the feline:

(each gray colour represents one stop, two highest and lowest stops are highlighted in other colours)


Regards

[fdisilvestro]

But there is a statistical trick to have a much more obvious (softer) histogram: averaging RAW data fills all levels in the histogram with the amount of light they received (or a very precise approximation of it thanks to statistics). The histogram I showed previously was a 8x8 pixels averaging. This allows to calculate scene's DR with cameras having less DR that the contrast of the scene. Actually the limit is only the RAW bitdepth.

Regards

Guillermo,

How did you made the averaging? Was that by using software you developed?

thanks

[Guillermo Luijk]

Guillermo,

How did you made the averaging? Was that by using software you developed?

I wouldn't call it software, it's just a suboptimum loop written in R:

# dcraw -v -r 1 1 1 1 -o 0 -S 16382 -4 -T blackcat.dng
img=readTIFF("blackcat.tiff", native=F, convert=F)

REDUCTION=8
DIMY=as.integer(dim(img)[1]/REDUCTION)
DIMX=as.integer(dim(img)[2]/REDUCTION)
imgresize=array(1,c(DIMY,DIMX,3))

for (i in 1:DIMY) {
    for (j in 1:DIMX) {
        for (k in 1:3) {
            imgresize[i,j,k]=mean(img[((i-1)*REDUCTION+1):(i*REDUCTION), ((j-1)*REDUCTION+1):(j*REDUCTION), k])
        }
    }
}

writeTIFF(imgresize^(1/Gamma), paste0("imgresize_",REDUCTION,".tif"),
          bits.per.sample=16, compression="LZW")

[fdisilvestro]

Thank you very much!

[kirkt]

Attached is a rendering from the ETTR'ed dng (image 8 of the series).  Raw conversion performed in RPP, with various adjustments done in PS, including some chroma noise reduction (Neat Image) and sharpening, color balance and local contrast.  You can extract a serviceable image for the web, but if you were to print this you would probably want to blend/combine/merge another shot at a longer exposure for the shadow data.  The camera focused on the curtain and the DOF was not sufficient to keep the cat in focus - you can see some small bokeh balls from the specular hits around the ears and whiskers.  Those are easily removed with spot healing or similar touch up.

If you were going for a brighter rendering with a lot of boost to the shadow tones, like the cat, then it would be imperative to add another exposure.

kirk

[Dinarius]

Some Phase backs provide such raw Histogram as can some Canon's with:
https://www.magiclantern.fm/
However it's done under the hood, it's doable and sadly rare. Because for some strange reason, most manufacturers don't want to (not can't). 😩

As an aside...

When it comes to improvements thro' fireware updates, Sony and Hasselblad are way better than Canon. In fact, with Canon, apart from updates for new lenses and bug fixes, you pretty much have to wait for the next camera.

Thanks for the headsup on Magic Lantern. New to me.

D.

Edited. I see 5D MkIV not covered by Magic Lantern. Pity.

[Dinarius]

Interesting discussion...

Unless I missed it, no one here has discussed processing files using Linear (as Capture One call it) or Reproduction/Reproduction Low Gain (as Hasselblad call it).

Basically, you remove the baked-in curve, and whatever else the RAW processing software manufacturer considers a file from your camera model should look like straight out of the camera, and you start from scratch.

I don't think that ACR or LR have this option, but I'm open to correction.

I use it all the time for Sony, Canon and Hasselblad.

If you haven't tried it, you might be pleasantly surprised.

It is not for speedy processing, but then good processing is never fast.

D.

[digitaldog]

I don't think that ACR or LR have this option, but I'm open to correction.

http://www.color.org/scene-referred.xalter

[bjanes]

http://www.color.org/scene-referred.xalter

I've used that method before and it seems to work. However, the linked article refers to Photoshop CS3, which is quite old.

I think it is necessary to use Process Ver 2.0 rather than the current PV 5.0. I presume it would be best to use the Flat camera profile if it is available with the camera used to capture the raw file.

Bill

[David Eichler]

http://www.color.org/scene-referred.xalter

I find Adobe Standard to be an adequate, relatively neutral starting point for my purposes. Sort of my digital "work" print.

[fdisilvestro]

Interesting discussion...

Unless I missed it, no one here has discussed processing files using Linear (as Capture One call it) or Reproduction/Reproduction Low Gain (as Hasselblad call it).

Basically, you remove the baked-in curve, and whatever else the RAW processing software manufacturer considers a file from your camera model should look like straight out of the camera, and you start from scratch.

Linear curve will not bring you a raw histogram. First, a curve does not affect the end points (0 & 255) and even more important, the image & histogram you see in Capture 1 is already color space encoded, which makes the histogram to look different.

There are many occasions where clipped channels are a result of out of gamut issues related to the specific color space, where the raw data is not clipped. Moreover, usually when one thinks out of gamut only considers the maximum values (255) , but there are cases when out of gamut is 0 because the mathematical value would be negative (i.e. the cyan patch in the colorchecker, third row to the right when using sRGB)

Another misconception: there is no "baked" curve in a raw file.

[MauriceRR]

Linear curve will not bring you a raw histogram.

Somhow, yes, on a log scale. Even quite accurate with a repro profile and a linear scientific curve in capture One.

Another misconception: there is no "baked" curve in a raw file.

yes, not in the raw file, just to the default opening in most rawconverters. (ADobe, CaptureOne, Photolab, etc). After a fidelity base, a tone curve is most often applied on top. There is also some baked color processing (color-twist, memorycolors, etc) for more pleasing multi-purpose rendering -> avoiding mangenta skies, preserve skin tones from pink casts, green foliage, etc.
But ilf you are not interested in, for exemple packshoots photography, etc. Sometimes it's easyer to work on a base of color reproduction profiles. Moreover, working with linear profiles and a linear curve can be more suited for very precise works, especially in white shades, with a better control of the roll-off in highlights.

To Dinarius :
A linear curve with a roll-off can be a bit more suited for high dynamic range scenes (I think), but this kind of scene will also benefit from a local tonemapping, not only a bent on the tonecurve (wich does more or less in this case the same thing than pushing shadow/highlights recovery sliders.)
You can have linear values in ACR easily : process in v2, with everything at 0 including sliders and curves.
If you want the benefits of latest process, (highlights color recovery especially), you will need to make linear profile (lumariver profile designer, same dcamprof, basiccolor input), or remove a tone curve from an existing profile (you actually can do that easily with adobe dng profile editor). Then you will need to adjust the exposure with the white point from curves.

[fdisilvestro]

Somhow, yes, on a log scale. Even quite accurate with a repro profile and a linear scientific curve in capture One.

If this were true you would not need a profile at all. You would just use the raw data for repro. I bet it would not work.

[MauriceRR]

If this were true you would not need a profile at all. You would just use the raw data for repro. I bet it would not work.

Somehow, it's a raw histogramm with WB adjustment. The repro profile (or a generic one) doesn't change much the histogram.

[DP]

I've used that method before and it seems to work. However, the linked article refers to Photoshop CS3, which is quite old.

I think it is necessary to use Process Ver 2.0 rather than the current PV 5.0. I presume it would be best to use the Flat camera profile if it is available with the camera used to capture the raw file.

Bill

Adobe supplied DCP profiles use non linear color transforms, hence using them and calling the result (output after ACR/LR) "scene-referred" is plain stupid

[digitaldog]

Adobe supplied DCP profiles use non linear color transforms, hence using them and calling the result (output after ACR/LR) "scene-referred" is plain stupid

Right on, what does the ICC know about color when we have a color expert like you telling us the colorimetric 'facts'. 
http://www.color.org/scene-referred.xalter

[JaapD]

Not so fast Andrew  . DP mentions a "scene-referred" profile. To me this sounds like a kind of an ‘adaptive profile’, a profile that adjusts itself based on the actual content of the scene on hand. We all know this does not exist and as such I fully agree with DP on Adobe’s “scene-referred” statement.

Care to share your expert opinion on this?

Cheers,
Jaap

[digitaldog]

Not so fast Andrew  . DP mentions a "scene-referred" profile. To me this sounds like a kind of an ‘adaptive profile’, a profile that adjusts itself based on the actual content of the scene on hand. We all know this does not exist and as such I fully agree with DP on Adobe’s “scene-referred” statement.

Care to share your expert opinion on this?

Cheers,
Jaap

Yes: REREAD his wrong statement.

hence using them and calling the result (output after ACR/LR) "scene-referred" is plain stupid

Now read what those ”fools” at the ICC describe as the results. 😂

“There are two ways to be fooled. One is to believe what isn't true; the other is to refuse to believe what is true.” -Søren Kierkegaard

[Guillermo Luijk]

If this were true you would not need a profile at all. You would just use the raw data for repro. I bet it would not work.

You can try it, and you get "something" visible. This is what RAW data assigned to sRGB looks like prior to WB:



Composing every pixel colour from 4 RAW pixels: {R, (G1+G2)/2, B}, you get:



and here after WB (performed through a linear scaling on R and B), but with no colour profile conversion applied yet (RAW white balanced RAW values are directly assigned to sRGB):



The image looks pretty decent, and its histogram is a good proxy for the RAW histogram. The bad news is that no commercial RAW developer nor camera provides something like any of those two images. All of them apply white balance and colour profile conversion (which mixes up the three RGB components), so having a "Linear" option in your camera or RAW developer is not very useful if you are interested in the genuine RAW histogram.


Regards


PS: processed image with proper colour profile conversion:

[bjanes]

Linear curve will not bring you a raw histogram. First, a curve does not affect the end points (0 & 255) and even more important, the image & histogram you see in Capture 1 is already color space encoded, which makes the histogram to look different.

There are many occasions where clipped channels are a result of out of gamut issues related to the specific color space, where the raw data is not clipped. Moreover, usually when one thinks out of gamut only considers the maximum values (255) , but there are cases when out of gamut is 0 because the mathematical value would be negative (i.e. the cyan patch in the colorchecker, third row to the right when using sRGB)

If you want a histogram of the image in a linear space, you could first render into ProphotoRGB and then convert to LinearRIMM Ver 4 as per the ICC article that Andrew cited.

Bill