Luminous Landscape Forum

Raw & Post Processing, Printing => Digital Image Processing => Topic started by: Iliah on July 19, 2015, 04:05:22 pm

Title: Dynamic range limitations due to glare and flare
Post by: Iliah on July 19, 2015, 04:05:22 pm
The article discusses a simple experiment allowing to estimate how the camera dynamic range is affected by the glare and flare, and demonstrates that dynamic range figures of 11+ stops may be a bit too optimistic.

http://www.rawdigger.com/Dynamic-range-fair-share-of-flare-and-glare
Title: Re: Dynamic range limitations due to glare and flare
Post by: Tim Lookingbill on July 19, 2015, 04:13:18 pm
Are there any real world photographs that show this limited dynamic range due to flare and glare?

That test on that linked article is not how photographers make images especially when editing in a Raw converter to squeeze every bit of DR possible out of a capture.

IOW that article only helps those that take pictures of stars and heavenly bodies at night with a high powered telescope and there isn't a lot of detail in that to begin with unless you're using the Hubble space telescope.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 19, 2015, 05:10:34 pm
> Are there any real world photographs that show this limited dynamic range due to flare and glare?

All of them do.
I know some will be disappointed, but truth is the effect is there on any photo. If the scene has 11 stops dynamic range - it is quite visible.
Title: Re: Dynamic range limitations due to glare and flare
Post by: ErikKaffehr on July 19, 2015, 05:12:22 pm
+1

Erik


> Are there any real world photographs that show this limited dynamic range due to flare and glare?

All of them do.
I know some will be disappointed, but truth is the effect is there on any photo. If the scene has 11 stops dynamic range - it is quite visible.
Title: Re: Dynamic range limitations due to glare and flare
Post by: bjanes on July 19, 2015, 05:33:25 pm
Are there any real world photographs that show this limited dynamic range due to flare and glare?

That test on that linked article is not how photographers make images especially when editing in a Raw converter to squeeze every bit of DR possible out of a capture.

IOW that article only helps those that take pictures of stars and heavenly bodies at night with a high powered telescope and there isn't a lot of detail in that to begin with unless you're using the Hubble space telescope.

The veiling glare and flare light will be overlaid over the luminances of the scene, and contrast will be reduced. The effect will be most marked in the shadows where the flare light will be a significant proportion of the total luminance, but scene luminances will not be destroyed. The effect is similar to viewing a movie with the lights on. Is that correct, Iliah?

I have a well regarded book on photographic exposure dating back to the film era, where the authors stated that flare is not entirely deleterious since it lifts the shadows to a level that can be printed.

Regards,

Bill
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 19, 2015, 05:38:38 pm
> The veiling glare and flare light will be overlaid over the luminances of the scene, and contrast will be reduced. The effect will be most marked in the shadows where the flare light will be a significant proportion of the total luminance, but scene luminances will not be destroyed. The effect is similar to viewing a movie with the lights on.

Yes, or like projecting the slides with lights on; but that is a part of the story. The other part depends on what we agree to call "scene luminances".

> I have a well regarded book on photographic exposure dating back to the film era, where the authors stated that flare is not entirely deleterious since it lifts the shadows to a level that can be printed.

If it is veiling glare only, and if it is more or less uniform, it is the case sometimes. But even a uniform glare destroys details in shadows.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Tim Lookingbill on July 19, 2015, 06:38:33 pm
I agree with the premise of glare and flare limiting DR. I have old film legacy lenses that do this depending on the angle of light to subject and lens.

It's just the article was difficult for me to comprehend as to how they prove this. I thought this article was showing new information I wasn't aware before on this subject and I just didn't see it.

What I've found in practice editing Raw images that have this glare or veil, is it's best to retain but gradually taper off this veil close to black point in order to make out detail like in tree foliage or else wind up with blacks that look like charcoal blotches or holes in the image. Some of these blacks are around 10,10,10RGB and there's nothing else below to apply a curve adjust to taper off into. It's either all black or there's a veil of flare.
Title: Re: Dynamic range limitations due to glare and flare
Post by: ErikKaffehr on July 19, 2015, 10:07:16 pm
Hi,

This may be a typical image with a lot of flare, great highlight areas within picture, causing a lot of flare:

(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/CanoeStadium/overview.jpg) (http://echophoto.dnsalias.net/ekr/Articles/DRArticle/CanoeStadium/overview.jpg)

This is perhaps an example of a subject with little flare, small windows large dark area. HDR exposure using LumaRiver HDR.

(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/LumariverOverview.jpg) (http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/LumariverOverview.jpg)

Best regards
Erik
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 20, 2015, 09:54:45 am
Hi Erik,

Indeed, if the light source is present and shadows are empty on the RawDigger histogram, it is due to veiling glare that raises all the levels so that no "true black" is present in raw data.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Guillermo Luijk on July 20, 2015, 12:43:02 pm
I am not so sure glare is going to be such a general problem Iliah. The following (tone mapped) scene has more than 16 stops of DR:

(http://www.guillermoluijk.com/article/superhdr/resultado_lite.jpg)

Veiling glare is clearly visible in some areas due to lens flare, but in the very dark areas not affected the deep shadows remained deep. I needed a 4 shots / 3 stops apart HDR composite to capture the entire DR. This means my 8 stops Canon 350D could face up to: 8 + (4 - 1) * 3 = 17 stops of real dynamic range. And it did because this was the DR of this scene. This is the linear composite histogram:

(http://www.guillermoluijk.com/article/superhdr/hist_tot.gif)


And the deep shadows:

(http://www.guillermoluijk.com/article/superhdr/suelo.jpg)


The resulting superhdr.tif (http://www.guillermoluijk.com/download/superhdr.tif) can be lifted up to 15 stops holding detail:

(http://www.guillermoluijk.com/article/superhdr/progev.jpg)


Other not so radical examples of 12 stops and 11 stops scenes were no problem as well with exposure bracketing:

12 stops: indoor scene (the virtual RAW for download (http://www.guillermoluijk.com/download/rawvirtual.dng)):
(http://www.guillermoluijk.com/article/virtualraw/resultadolite6.jpg)

RAW histograms:
(http://www.guillermoluijk.com/article/virtualraw/histos.gif)

11 stops: candle flame
(http://www.guillermoluijk.com/article/candledr/zoneszoom.jpg)

Regards






Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 20, 2015, 12:48:06 pm
I am not so sure glare is going to be such a general problem Iliah. The following (tone mapped) scene has more than 16 stops of DR:

(http://www.guillermoluijk.com/article/superhdr/resultado_lite.jpg)

Sorry, I can't possibly see this as a generally usable image. On a side note, how did you determine the original scene had 16 EV of range?
Title: Re: Dynamic range limitations due to glare and flare
Post by: Guillermo Luijk on July 20, 2015, 01:15:04 pm
How valid the image is results irrelevant, it was just an experiment. The point is that it contained a difference in luminance between its darkest and brightest part of >16 stops. I determined this by looking at the histogram of the linear HDR composite. If it throwed 16 stops of DR it means the real scene was at least of 16 stops, far from the 11 stops limitation you are suggesting. Basically it's a counterexample.

Regards
Title: Re: Dynamic range limitations due to glare and flare
Post by: ErikKaffehr on July 20, 2015, 02:13:29 pm
Hi,

My take is that something like 0.5% of the light bounces around in the lens and gets on the sensor. A large bright area will have a different effect from a small bright area.

The "concert hall" image I posted is a typical example where flare from windows may play a lesser role.

Best regards
Erik


How valid the image is results irrelevant, it was just an experiment. The point is that it contained a difference in luminance between its darkest and brightest part of >16 stops. I determined this by looking at the histogram of the linear HDR composite. If it throwed 16 stops of DR it means the real scene was at least of 16 stops, far from the 11 stops limitation you are suggesting. Basically it's a counterexample.

Regards

Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 20, 2015, 02:32:24 pm
>The point is that it contained a difference in luminance between its darkest and brightest part of >16 stops

I asked you, "how did you determine the original scene had 16 EV of range?" - that is, what is the map of the spotmeter readings?

Saying the image is not usable I'm not discussing the subject of the image, but the subject of the thread. Not only flare and glare directly, but white balance skews all over "the place".
Title: Re: Dynamic range limitations due to glare and flare
Post by: Guillermo Luijk on July 20, 2015, 04:43:14 pm
>The point is that it contained a difference in luminance between its darkest and brightest part of >16 stops

I asked you, "how did you determine the original scene had 16 EV of range?" - that is, what is the map of the spotmeter readings?

Saying the image is not usable I'm not discussing the subject of the image, but the subject of the thread. Not only flare and glare directly, but white balance skews all over "the place".

I didn't care to determine the exact DR of the scene, I just captured it (for which I needed 4 shots 3 stops apart) and the resulting captured data spreaded along 16 stops. That means the original scene had at least 16 stops of DR, well beyond 11.

I didn't use any spotmeter. In fact I didn't meter anything since it is not necessary for HDR capturing. Just a couple of tests are enough and more accurate.

Regards
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 20, 2015, 05:48:32 pm
> the resulting captured data spreaded along 16 stops. That means the original scene had at least 16 stops of DR

Sorry, but no. Nothing substitutes controlled experiments. Those are especially valuable when the image looks wrong.
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 20, 2015, 07:08:49 pm
> the resulting captured data spreaded along 16 stops. That means the original scene had at least 16 stops of DR

Sorry, but no. Nothing substitutes controlled experiments. Those are especially valuable when the image looks wrong.

well, what do you think (subjectively) is the DR of the scene in question and what one can find out from the said 4 raw files (if they are available)... hopefully not something like 8EV vs 16EV claimed ?
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 20, 2015, 07:34:28 pm
The way it is rendered (without visible details in the window) IMHO it is about 11 stops, give or take.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Guillermo Luijk on July 21, 2015, 02:29:09 am
The window has details, even if I could have added some less exposed shot to avoid the little RAW clipping. Don't focus on the particular tone mapping applied (tone mapping captured information that spreads over a range of +16 stops is not an easy task):

(http://www.guillermoluijk.com/article/superhdr/ventanasat.jpg)

I just wanted to show that 11 stops is far from being a limit as imposed by veiling glare.

Regards
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 10:35:46 am
The window has details, even if I could have added some less exposed shot to avoid the little RAW clipping. Don't focus on the particular tone mapping applied (tone mapping captured information that spreads over a range of +16 stops is not an easy task):

(http://www.guillermoluijk.com/article/superhdr/ventanasat.jpg)

I just wanted to show that 11 stops is far from being a limit as imposed by veiling glare.

Regards

No, it does not contain details. Those are some crude changes in brightness, nothing natural, and the overall contrast is not of a photographic image. Even on your "12 stops: indoor scene" the view from the window in not even close to a natural view. Such HDR is in the forensic processing domain. And again, I would be very interested to see the scene measurements because I think only such measurements may really put a number onto the scenes you are shooting.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Guillermo Luijk on July 21, 2015, 11:20:33 am
No, it does not contain details. Those are some crude changes in brightness, nothing natural, and the overall contrast is not of a photographic image. Even on your "12 stops: indoor scene" the view from the window in not even close to a natural view. Such HDR is in the forensic processing domain. And again, I would be very interested to see the scene measurements because I think only such measurements may really put a number onto the scenes you are shooting.

You again focus on the applied tone mapping Iliah, but my lack of skills in HDR processing is irrelevant for what I'm demonstrating: capturing more than 11 stops is nothing veiling glare will prevent as a general rule. Tons of real world scenes with higher DR can be properly captured with just a couple of shots, and glare is not an issue on them.

You can download the RAW file (http://www.guillermoluijk.com/download/rawvirtual.dng) and see for yourself. There is nothing unexpected or unnatural when you process it linearly with 0 adjustments, just the usual dull image one gets before saturation and tone curve are applied:

(http://www.guillermoluijk.com/misc/campanario.jpg)


Regards


Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 11:37:31 am
Sorry, but it is the wrong contrast on the crop. It is washed out.
Now, the things start with metering. You suggest the scene is 12 stops wide, based on the raw data. My question is how wide it is based on direct measurements. Clear sky through a window is 14 EV, office is about 8 EV with artificial light only. Question is - how wide is the given scene?
Title: Re: Dynamic range limitations due to glare and flare
Post by: Guillermo Luijk on July 21, 2015, 12:07:50 pm
At least 12 stops, i.e. larger than your 11 stops figure. Veiling glare is not an issue here, full stop. Photographers take pictures, they are not forensic analyzers.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 12:18:40 pm
At least 12 stops, i.e. larger than your 11 stops figure. Veiling glare is not an issue here, full stop. Photographers take pictures, they are not forensic analyzers.

I do not see 12 stops here, and I see an issue with haze and lost contrast. This is going nowhere unless you have an accurate set of measurements, but the raw histogram shows nothing in the green channel past -8EV, and pretty much nothing past -7 2/3 EV where the noise bell dominates.
Title: Re: Dynamic range limitations due to glare and flare
Post by: EricV on July 21, 2015, 12:31:19 pm
I asked you, "how did you determine the original scene had 16 EV of range?" - that is, what is the map of the spotmeter readings?
The raw histogram from the actual photograph(s) is absolute proof that the original scene had at least as much dynamic range as claimed.  A meter reading could show that the true dynamic range was even greater.  But the dynamic range was certainly not less than what was actually recorded by the camera.
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 12:39:44 pm
Such HDR is in the forensic processing domain.
even so it still means that Guillermo managed to have greater DR captured w/ whatever glare/flare happened... just like a difference between "PDR" and "engineering DR" from DxO... DxO still makes their numbers from some raw data, even in some artificial lab conditions... not from some math model of the sensor, no ?
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 12:41:17 pm
The raw histogram from the actual photograph(s)
the only posted raw was a synthetic one, was it not ? I am not saying that it was made from blended original raws in some creative manner though.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 12:46:26 pm
The raw histogram from the actual photograph(s) is absolute proof that the original scene had at least as much dynamic range as claimed.  A meter reading could show that the true dynamic range was even greater.  But the dynamic range was certainly not less than what was actually recorded by the camera.

> the dynamic range was certainly not less than what was actually recorded by the camera.
There are several reasons why one can't decide on the dynamic range of the scene based on raw files, and one of those is noise, the other is veiling glare. In a word, no measurements from scene taken and mapped over the shot - no way to know the dynamic range of the scene.

If  the HDR raw is a linear superimpose the total DR in HDR file it is an accurate measure of the sum of the dynamic ranges of the source raw files. That means the recording contains less than 12 stops; and if you insist the recording is the accurate representation of the luminocities in the scene, so is the scene - less than 12 stops.

Put a Q13 in the scene, and we will see how the contrast is changed.

Kodak Q13 target, semi-gloss, shot with Df at ISO 100 with -1 2/3 EV, which means the black is 8 stops down from clipping, normal diffused studio light setup, even light from high-grade sources.

First column is the data for Y on Q13, measured with Spectrolino, range 0..100, and normalized to exposure difference (before normalization the brightest patch was 91.26); second column indicates the number of stops down from clipping; third column is shot data normalized to 100 (so it essentially also contains percent of gray); and the last column contains the deviation from linearity, also in stops.

Spectrolino_Y EV_down_from_clipping AsShot_Y Non-linearity(EV)

29.961 -1.739 29.990 0.00

22.903 -2.126 23.712 -0.05

18.090 -2.467 18.778 -0.05

14.331 -2.803 14.978 -0.06

11.307 -3.145 12.035 -0.09

8.969 -3.479 9.322 -0.06

7.180 -3.800 7.591 -0.08

5.647 -4.146 6.050 -0.10

4.495 -4.476 4.848 -0.11

3.641 -4.780 3.868 -0.09

2.886 -5.115 3.130 -0.12

2.311 -5.435 2.525 -0.13

1.839 -5.765 2.042 -0.15

1.468 -6.090 1.673 -0.19

1.202 -6.379 1.349 -0.17

0.965 -6.695 1.166 -0.27

0.798 -6.970 0.986 -0.31

0.607 -7.363 0.841 -0.47

0.545 -7.520 0.793 -0.54

0.417 -7.906 0.712 -0.77
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 12:46:47 pm
but the raw histogram shows nothing in the green channel past -8EV, and pretty much nothing past -7 2/3 EV where the noise bell dominates.

I used FRV and added 8 EV simulated exposure

(http://s7.postimg.org/yqnf1bntn/frv.jpg)

Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 12:50:15 pm
> Guillermo managed to have greater DR captured w/ whatever glare/flare happened

Well, when one starts to look at an image with the eyes of a photographer (or a demanding customer not interested in "modern art") he may want to reconsider "managed". The indoors scene is on the edge, while the toilet is well past usable.
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 12:51:30 pm
> Guillermo managed to have greater DR captured w/ whatever glare/flare happened
Well, when one starts to look at an image with the eyes of a photographer

that's not fair ! again - PhDR vs EnDR ... so fine print shall be added to any statements

Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 12:51:46 pm
I used FRV and added 8 EV simulated exposure

(http://s7.postimg.org/yqnf1bntn/frv.jpg)



And is it a photographic image? Does it contain something that can't be simply clipped?
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 12:53:45 pm
And is it a photographic image? Does it contain something that can't be simply clipped?
in the eye(s) of beholder... forensic/pixelpeeping folks are as good clientelle as anything else !
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 12:58:34 pm
that's not fair ! again - PhDR vs EnDR ... so fine print shall be added to any statements



2008 GoogleTech video of a talk by John McCann

https://youtu.be/ALfiTDYLtAQ

1 log unit is 3.3 EV.

and

http://mccannimaging.com/HDR/HDR_Papers.html
Title: Dynamic range limitations due to glare and flare -- in a single exposure?
Post by: BJL on July 21, 2015, 01:10:37 pm
Guillermo and Illiah,

    if I undersigned the glare/flare/DR issue correctly, this DR limitation arises in a single capture, but can be overcome by taking several shots at different exposure levels and blending them.  Have I got that right?


P. S. No, I didn't!   I see now that once flare/glare from the brightest part of a scene is at a level of, say, 11 or 12 stops below, it will be so in all of the multiple frames used in HDR, all al will have equally veiled deep shadows.  Unless one does something extra like blocking out the brightest part of the scene (ND grad?) in some exposures used to blend.
Title: Re: Dynamic range limitations due to glare and flare -- in a single exposure?
Post by: Iliah on July 21, 2015, 01:13:48 pm
Guillermo and Illiah,

    if I undersigned the glare/flare/DR issue correctly, this DR limitation arises in a single capture, but can be overcome by taking several shots at different exposure levels and blending them.  Have I got that right?

Absolutely not. The limitation is with the system, before the image is captured, and the nature of that limitation comes from the dynamic range of the scene and the performance of the optical system. Increasing exposure, one gets to optical artifacts being more visible, that's all.
Title: Re: Dynamic range limitations due to glare and flare -- in a single exposure?
Post by: AlterEgo on July 21, 2015, 01:14:39 pm
    if I undersigned the glare/flare/DR issue correctly, this DR limitation arises in a single capture,
define single capture... what if I have a sensor with sensels individually controlled for exposure time and/or how translucent is the topping  on top of them ?
Title: Re: Dynamic range limitations due to glare and flare
Post by: EricV on July 21, 2015, 03:07:34 pm
> the dynamic range was certainly not less than what was actually recorded by the camera.
There are several reasons why one can't decide on the dynamic range of the scene based on raw files, and one of those is noise, the other is veiling glare. In a word, no measurements from scene taken and mapped over the shot - no way to know the dynamic range of the scene.
Are you agreeing or disagreeing with the quoted statement?  I agree with you that one cannot determine the dynamic range of the scene from the raw files.  I assert that one can however set a lower limit on the dynamic range of the scene from the raw files.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 03:33:49 pm
Are you agreeing or disagreeing with the quoted statement?  I agree with you that one cannot determine the dynamic range of the scene from the raw files.  I assert that one can however set a lower limit on the dynamic range of the scene from the raw files.
I disagree with your "certainly", because where there is noise, there is uncertainty.
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 03:42:07 pm
I disagree with your "certainly", because where there is noise, there is uncertainty.
the DR can be either greater or equal or smaller... noise can only reduce DR as recorded in raw data vs the original scene, no ? is there a situation when noise can increase the DR ?
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 03:46:01 pm
the DR can be either greater or equal or smaller... noise can only reduce DR as recorded in raw data vs the original scene, no ? is there a situation when noise can increase the DR ?
No, if DR is determined by the presence of the levels in the histogram, noise actualy creates an illusion of wider dynamic range.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Bart_van_der_Wolf on July 21, 2015, 05:06:30 pm
2008 GoogleTech video of a talk by John McCann

https://youtu.be/ALfiTDYLtAQ

1 log unit is 3.3 EV.

and

http://mccannimaging.com/HDR/HDR_Papers.html

Part of the discussion looks like a Déjà vu (http://forum.luminous-landscape.com/index.php?topic=56103.msg456648#msg456648) to me.

Cheers,
Bart

P.S. the moved links in my original reaction are now: here (http://www.mccannimaging.com/Retinex/Publications_files/07%20CIC%20Rizzi.pdf), and here (http://mccannimaging.com/Retinex/Publications_files/07HDR1Hist.pdf), and are also in the link to McCan's homepage that Iliah provided.
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 05:16:33 pm
> Part of the discussion looks like a Déjà vu

True, but there is hope that bis repetita placent.
Title: Re: Dynamic range limitations due to glare and flare
Post by: AlterEgo on July 21, 2015, 06:04:52 pm
No, if DR is determined by the presence of the levels in the histogram, noise actualy creates an illusion of wider dynamic range.
but if we can see the details (S/N > "1") then it is not an illusion though, no matter how unuseful those details are...  so with FRV screenshot, I added +8 EV of simulated exposure and I still see the details, so it is not an illusion, no ?
Title: Re: Dynamic range limitations due to glare and flare
Post by: Iliah on July 21, 2015, 06:09:03 pm
but if we can see the details (S/N > "1") then it is not an illusion though, no matter how unuseful those details are...  so with FRV screenshot, I added +8 EV of simulated exposure and I still see the details, so it is not an illusion, no ?
Illusions can be useful, but the topic is the scene reproduction, in a fairly accurate manner. You can add +12 to a single shot from a noisy sensor and there will be "details".