Luminous Landscape Forum

Site & Board Matters => About This Site => Topic started by: fdisilvestro on April 15, 2015, 06:02:04 pm

Title: Precise Digital Exposure
Post by: fdisilvestro on April 15, 2015, 06:02:04 pm
Hi,

I'm not sure if I understood correctly the article in the home page, but "Precise Digital Exposure" using the rendered RGB values in Capture one? seriously?
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 15, 2015, 06:15:43 pm
In terms of the rendering, can we equate that with ‘development’? If so, can we separate exposure from development?
That said, I glanced at the piece and it’s very, very possible I didn’t understand correctly either.
Title: Re: Precise Digital Exposure
Post by: Jim Metzger on April 15, 2015, 07:07:04 pm
 Lee Varis has had a book out since June 2010 called "Mastering Exposure and the Zone System for Digital Photographers". It is still available. There was a companion webcast at some point.

I found the application of traditional (film) photo technique to digital capture to be very enlightening.
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 15, 2015, 07:36:34 pm
"My publication [2011] is, as far as I am aware, first to apply the Zone System principles and the spot metering correct for digital, simply because I did so according to the basis Adams wrote of"

I am sorry to say that article is nothing new. Two years before that, in 2009, I wrote a whole article about ETTR and spot metering, using real RAW data and plotting log RAW histograms to find out how much highlight headroom spot metering provided in several real world situations with Canon DSLR cameras:

ETTR WITH SPOT METERING (http://www.guillermoluijk.com/article/spot/index.htm)

(http://www.guillermoluijk.com/article/spot/nube.jpg) (http://www.guillermoluijk.com/article/spot/nube.gif)


And two years before that, in 2007, I developed a piece of software to calculate and plot the Zone System of a scene from a linear image of it (it can be applied to RAW data if used properly):

HISTOGRAMMAR (http://www.guillermoluijk.com/tutorial/histogrammar/index_en.htm)

(http://www.guillermoluijk.com/article/nonoise/resultgamma.jpg) (http://www.guillermoluijk.com/tutorial/histogrammar/zonasluma.gif)

Regards



Title: Re: Precise Digital Exposure
Post by: bjanes on April 15, 2015, 09:11:40 pm
Hi,

I'm not sure if I understood correctly the article in the home page, but "Precise Digital Exposure" using the rendered RGB values in Capture one? seriously?

One can apply a tone curve in C1 or ACR to simulate the shoulder of a film exposure, but the fact is that the digital sensor is linear. The author talks about the shoulder of his sensor, but in my experience with the Nikon D800e, the sensor is almost perfectly linear right up to clipping, which occurs abruptly. I think this is typical for most other digital sensors. In the linear scale, clipping is 2.5 stops from mid-gray (rather than the 4 EV values from Zone V to Zone IX, as the author correctly states). Many years ago, Bruce Fraser suggested that one could meter the highlights where it is necessary to have some image detail (which would expose as mid-gray in the raw file) and increase exposure by 2.5 stops above the metered value. This might be extended to 3 or 3.5 stops if you want to chance it with highlight recovery in the raw developer. This is nothing new.

I agree that one should not base exposure on rendered values, but rather on raw values as determined by RawDigger or a similar tool. I do have the Pentax digital meter that the author (and Adams) prefer, but find its use in the field too cumbersome. Exactly which highlight should be metered? In most cases, I rely on the camera histogram and blinking highlights, and allow slight highlight clipping since the histogram is conservative. With the extended dynamic range of the D800 and other modern sensors, precise ETTR is not necessary in most cases. One can get good results with less than optimal ETTR. In challenging situations, it is easier to bracket rather than fiddle around with "precise" exposure values. This gives possibility of HDR, if necessary.

This would be a good opportunity for others to state how they handle exposure in various situations.

Bill



Title: Re: Precise Digital Exposure
Post by: fdisilvestro on April 15, 2015, 09:31:11 pm
I agree that one should not base exposure on rendered values, but rather on raw values as determined by RawDigger or a similar tool.

Exactly!
Title: Re: Precise Digital Exposure
Post by: hjulenissen on April 16, 2015, 01:22:25 am
I agree that one should not base exposure on rendered values, but rather on raw values as determined by RawDigger or a similar tool.
Or (to beat a dead horse):
(http://www.katsurashareware.com/images/plmain.png)

Audio people get to read out levels right after the A/D converter, presented in several ways optimized for (among other things) tuning input sensitivity. How incredibly hard can it be for the camera manufacturers to do something similar? Surely, they must have a few people with basic signal processing skills and the ability to implement a simple gui component?

I am not sure what to make of the article. It seems "stuck" on the notion of mid-grey that may have served film well for 100+ years, but I don't see how relevant it is today.

-h
Title: Re: Precise Digital Exposure
Post by: bernie west on April 16, 2015, 01:24:42 am
I'm waiting for the article to be translated into English ( ;) ) before I comment too much on it.  Basically I've got no idea what he is saying, but I'm interested to see how he thinks using the zone system can apply to digital.  I used to be heavily involved back in the day in the film vs digital war, and there was a thread on photo.net with some guy trying to apply the zone system to digital.  His theory was kooky, but that's all I can remember, and I can't seem to find the thread any more.  One thread I can't find, and one article I can't parse.   ???  But I have had some fun rereading some of the ETTR threads.  It is amazing how so few people understood how the digital sensor worked (and I mean in principle, let alone specifics), and SO many people were resistant to the ideas of digital sensitometry.  Thank god most of that film vs digital bollocks is over.  Rereading the threads again, I am amazed that I and people like Andrew Rodney had the will power to keep explaining it over and over and over again... :)
Title: Re: Precise Digital Exposure
Post by: MarkL on April 16, 2015, 08:04:54 am
Film and digital are similar in that they record light but that is about where the similarities end. I don't really understand tortured theories to fit a system designed around film into digital. The process should match the technology.
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 16, 2015, 08:06:25 am
Apologies if what I explain seem confusing. :P After learning this method I have found it simple and useful as a tool for precise spot metering exposure as per the exact capture latitude of my digital back. In principle it is same as spot metering for different film was (which likewise were different media/technology). Thus it is a specialised exposure tool for when you need to spot meter for preciseness and for and based on the digital technology.

---

The attached is a characteristic curve for my Leaf AFi-II 12, made by shooting test shots of an 18% grey card in constant light, which was made same way as Ansel Adams explained in his book The Negative of making test images one step apart. I have also included curves for extreme adjusted highlight and shadow compensations and together with extreme negative shadow exposure.

What do the curves bring? These tell me the changes in rendering of tonal values when I develop from defaults to extreme adjustments in post. This enables, at time for spot metering of a scene, to visualise precise the exposure latitude compared to the scene. Also for making a choice for a purpose made transfer zone into shoulder.

Linear response? Please note there is actually a slight shoulder in highlight end at defaults for my digital back (I assume due settings/processing in Capture One).
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 16, 2015, 08:06:52 am
I'm not sure if I understood correctly the article in the home page, but "Precise Digital Exposure" using the rendered RGB values in Capture one? seriously?
In terms of the rendering, can we equate that with ‘development’? If so, can we separate exposure from development?
That said, I glanced at the piece and it’s very, very possible I didn’t understand correctly either.

I suggest to think in terms of, that a very basis of the zone system is as a tool for visualising by spot metering a scene before snapping a shot, thus for pre visualising what our captured information will be (to me at defaults in Capture One thru extreme adjustments in post). Same as for film, spot metering is for scenes that require such preciseness, or when we desire to work by the method.


"For correctness this all dates back to what Ansel Adams wrote relating to the Zone System and how to determine spot metering for black and white negative film, Polaroid Land Print and more. Many have misunderstood the Zone System and believe it to merely consist of how Adams applied it for black and white negative film. My publication is, as far as I am aware, first to apply the Zone System principles and the spot metering correct for digital, simply because I did so according to the basis Adams wrote of. I thus tested the spectral response (sensitivity to light) for the particular sensor in my digital back and in detail evaluated the RGB values in my RAW converter of choice, Capture One."

As far as I know the above is correct, because - as stated - I went from the basis of what Adams wrote of for negative film, slide film, polaroid land prints, and applied it on basis of the precise response of my digital sensor. Please refer to my paper and Adams' book The Negative.

All photos in the article were spot metered. An example is in the tea house, which I spot metered from outside for sky and through a window for skin, thereafter went inside and within seconds made my exposure in a sudden appearance in front of the subjects.

Thank you for reading.

Anders
Title: Re: Precise Digital Exposure
Post by: bernie west on April 16, 2015, 08:48:53 am
I'm still not able to parse what you are saying very well, but I don't understand the point of spot-metering to visualise your exposure before you expose it.  You can pretty easily create a unitary white balance via Guillermo's work years ago and then your jpg histogram on the back of the camera will display a very close approximation of what is clipping in the raw data.  It's been years since I was involved in all that stuff, so I can't actually remember the specifics of it, particularly regarding whether the uni white balance takes account for the wb multipliers of greater than one in the green and red channels.  From memory, it did, I think, as the results it gave were really very close to the actual sensor clipping point.  So, using a uni white balance, you can just adjust your exposure with EC in camera till you get the highlights just about clipping.
Title: Re: Precise Digital Exposure
Post by: bjanes on April 16, 2015, 09:15:43 am
Apologies if what I explain seem confusing. :P After learning this method I have found it simple and useful as a tool for precise spot metering exposure as per the exact capture latitude of my digital back. In principle it is same as spot metering for different film was (which likewise were different media/technology). Thus it is a specialised exposure tool for when you need to spot meter for preciseness and for and based on the digital technology.

---

The attached is a characteristic curve for my Leaf AFi-II 12, made by shooting test shots of an 18% grey card in constant light, which was made same way as Ansel Adams explained in his book The Negative of making test images one step apart. I have also included curves for extreme adjusted highlight and shadow compensations and together with extreme negative shadow exposure.

What do the curves bring? These tell me the changes in rendering of tonal values when I develop from defaults to extreme adjustments in post. This enables, at time for spot metering of a scene, to visualise precise the exposure latitude compared to the scene. Also for making a choice for a purpose made transfer zone into shoulder.

Linear response? Please note there is actually a slight shoulder in highlight end at defaults for my digital back (I assume due settings/processing in Capture One).

You are examining the rendered file, not the raw values. The shoulder and toe are added from the rendering and are not in the raw file. In terms of Ansel Adams, the raw file is your negative and the rendered file represents the manipulations added in printing. Here are results from the Nikon D800e showing a linear curve.

Bill
Title: Re: Precise Digital Exposure
Post by: fdisilvestro on April 16, 2015, 09:24:56 am
In terms of Ansel Adams, the raw file is your negative and the rendered file represents the manipulations added in printing.

+1

Rawdigger (http://www.rawdigger.com/) is an excellent tool to analyse raw files and build the response curve of a sensor
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 16, 2015, 10:10:55 am
In terms of Ansel Adams, the raw file is your negative and the rendered file represents the manipulations added in printing.
Development of neg? I asked earlier but haven’t heard back about that.
I’d agree, using a tool to examine the raw itself reduces any other process and tells us more truthful information about just exposure. I wish we had such tool on the damn camera. But in terms of the article and the rendered values that were not defined in highlights, isn’t that fair game since rendering is (I believe) akin to the development of the neg/raw?
Title: Re: Precise Digital Exposure
Post by: fdisilvestro on April 16, 2015, 10:55:50 am
Development of neg? I asked earlier but haven’t heard back about that.
I’d agree, using a tool to examine the raw itself reduces any other process and tells us more truthful information about just exposure. I wish we had such tool on the damn camera. But in terms of the article and the rendered values that were not defined in highlights, isn’t that fair game since rendering is (I believe) akin to the development of the neg/raw?

You could look at it in that way, but how do you know that your rendering is optimal? It would look as if you don't have control on the rendering, which is not the case.

With a negative (film) once you develop it, there is not much you can do. With Raw you can go back and re-render.
Title: Re: Precise Digital Exposure
Post by: MatthewCromer on April 16, 2015, 11:02:42 am
I've found that ETTR techniques usually result in subtle and undesirable color shifts in the finished result. I'd rather have better colors and tonalities and a bit more noise.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 16, 2015, 11:11:59 am
Development of neg? I asked earlier but haven’t heard back about that.
I’d agree, using a tool to examine the raw itself reduces any other process and tells us more truthful information about just exposure. I wish we had such tool on the damn camera.

Do any camera makers make an SDK available to Joe Public to fiddle around with?  Even if you couldn't update the firmware because they want to keep it proprietary, you could perhaps create a phone app that could wirelessly tether with your camera and give you a raw histogram on your phone. 
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 16, 2015, 11:52:48 am
You could look at it in that way, but how do you know that your rendering is optimal?
I’d consider that somewhat (or highly) subjective. I’d prefer not to blow out highlights but if that’s the intent of the image creator, so be it. Blocking up shadows is fine.
Quote
With a negative (film) once you develop it, there is not much you can do.
To a degree yes, but you now have to make a print. So perhaps it’s fair to say rendering is part development (normalizing ETTR) and part subjective (making ‘the print’).
Title: Re: Precise Digital Exposure
Post by: Rhossydd on April 16, 2015, 06:03:39 pm
since rendering is (I believe) akin to the development of the neg/raw?
I don't think this is correct or a useful analogy.
With film 'development' was a process that could be manipulated to change the characteristics of the latent image, but on a 'once only' basis after exposure(capture). In a digital system, there's little other than choosing an ISO setting that can change the characteristics of the medium once you've chosen the camera(sensor) to use.
With digital, once that RAW image is saved, that's it, it's complete and invariable.
People refer to converting raw files as 'development' or rendering, but really the process is more akin to printing in analogue terms. You have a constant master raw file that can be output in various ways dependant on skill and the technology to render the data.

I just fail to understand why people still bother with such out dated concepts as the zone system. It was designed in the days when taking multi different exposures was impossible/time consuming/expensive, when there were post exposure possibilities that needed consideration and there was no immediate feedback of results. That just isn't the case now, you can cover all exposure variables in a swift bracketed burst that leaves far more options at negligible/no cost.
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 16, 2015, 06:25:24 pm
I don't think this is correct or a useful analogy.
With film 'development' was a process that could be manipulated to change the characteristics of the latent image, but on a 'once only' basis after exposure(capture).
Isn’t that the same in the raw converter? If I use ETTR, the initial (default) rendering/development looks too light. Using Michael’s term, I ‘normalize’ the using various sliders.
Quote
In a digital system, there's little other than choosing an ISO setting that can change the characteristics of the medium once you've chosen the camera(sensor) to use.
Agreed.
Rendering is still part of the process to ‘develop’ the image as we desire.
Quote
People refer to converting raw files as 'development' or rendering, but really the process is more akin to printing in analogue terms
I suppose one could look at it that way.
Quote
I just fail to understand why people still bother with such out dated concepts as the zone system.
We are in violent agreement!
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 16, 2015, 08:01:25 pm
I just fail to understand why people still bother with such out dated concepts as the zone system. It was designed in the days when taking multi different exposures was impossible/time consuming/expensive, when there were post exposure possibilities that needed consideration and there was no immediate feedback of results. That just isn't the case now, you can cover all exposure variables in a swift bracketed burst that leaves far more options at negligible/no cost.

And this can be extended to any metering system or exposure trick such as the bracketing you suggest. Modern cameras display a pre-visualization (wasn't that concept so important to Ansel Adams?) of the scene on the EVF that makes metering or bracketing a waste of time and resources. Since I entered the CSC world I got rid of all those things with which I never managed to sympathise when using Canon DSLR's. Now I don't even look at the histogram since the highlight clipping and shadow underexposure warnings are far more powerful tools, providing accurate spatial information. Just need to adjust the exposure like in a RAW developer until I get an image to my liking, and take the picture (only one picture).

I wouldn't use this...
(http://www.guillermoluijk.com/misc/expometer.png)

if I can choose this...
(http://www.guillermoluijk.com/misc/evf.jpg)

Regards!
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 16, 2015, 10:07:34 pm
I've found that ETTR techniques usually result in subtle and undesirable color shifts in the finished result. I'd rather have better colors and tonalities and a bit more noise.
ETTR or ETTR and your raw converter + whatever "camera profile" you are using... granted you can ETTR things to where raw data will not be linear, but I don't think you are talking about such highlights
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 16, 2015, 10:15:54 pm
And two years before that, in 2007, I developed a piece of software to calculate and plot the Zone System of a scene from a linear image of it (it can be applied to RAW data if used properly):

LightZone was doing that in 2006 in raw converter (well it was not a raw converter technically speaking - see further about dcraw) if I am not mistaken... and as it was using dcraw as a program (calling compiled executable) you could modify the input as you wish... no ?
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 16, 2015, 10:19:43 pm
I assume due settings/processing in Capture One.
try linear scientific curve from C1 CH edition.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 16, 2015, 11:40:22 pm
@Guillermo... "CSC world" - what's that?

ETA: Also, can anyone recommend any good online resource regarding camera profiles with Lightroom?  Every now and then I revisit some of my old canon 5D raws and I'm nowhere near as happy with the LR 'default' camera profile settings for the 5D as I am for my Nikon.  Notwithstanding the less than desirable colour relationships when modifying images, I also had trouble with extreme ETTR 'normalisation' in LR - particularly in the highlights.  I also have a fuji x100, and the highlight recovery on 'default' settings in LR with it are not very good.  You can get large grey sections of rebuilt highlights with abrupt transitions to normally rendered colour.  I'm not so worried about that with the fuji, as I only temporarily bought it for street work in India and will be selling it soon anyway.  Also, colour accuracy in extreme highlights wasn't really my primary concern while shooting street.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 17, 2015, 12:01:51 am
"CSC world" - what's that?

it is the improper name for dSLMs (because they don't have to be compact)
Title: Re: Precise Digital Exposure
Post by: bernie west on April 17, 2015, 12:05:51 am
Oh, he's one of them...  :P
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 17, 2015, 12:06:38 am
ETA: Also, can anyone recommend any good online resource regarding camera profiles with Lightroom?

google is by far the best one - you can search for things like : madmanchan, eric chan, sandymc, vitnovak... then you get an idea

Title: Re: Precise Digital Exposure
Post by: bernie west on April 17, 2015, 12:33:30 am
Cheers for that!  I'm just checking his site out now.  I remember that name (madmanchan), I think from a lot of the technical threads here back in the old days.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 17, 2015, 12:59:49 am
Cheers for that!  I'm just checking his site out now.  
actually you need to read his postings in various forums, his site is of a lesser value for your purpose of reading about dcp related technicalities - if you are talking abour mr. Chan.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 17, 2015, 01:09:39 am
Yeah, his site doesn't actually have anything about LR camera profiles.

I'm more after a condensed resource on the topic.  I don't really have time to chase down individual posts by people all over the internet.  I'm hoping someone has put together a resource on this.
Title: Re: Precise Digital Exposure
Post by: Rhossydd on April 17, 2015, 03:44:35 am
Isn’t that the same in the raw converter?
No. RAW converters work non destructively, the original data never changes.
Film development was a variable process that could alter what was captured after exposure on a once only basis (OK, in theory for the pedants, you could use reducers and intensifiers to further modify developed film, but that only had a limited range of effect and degraded the image to an extent). The end result, a negative or positive, was a final locked set of information to print from.
Title: Re: Precise Digital Exposure
Post by: Rhossydd on April 17, 2015, 03:50:09 am
Modern cameras display a pre-visualization (wasn't that concept so important to Ansel Adams?) of the scene on the EVF that makes metering or bracketing a waste of time and resources.
Not completely. You know well that the image shown on the EVF won't fully represent the possibilities of the image saved to file. A degree of understanding of how a raw converter will handle bringing out shadow detail or highlight recovery remains an important skill, akin to 'pre-visualisation'.
Title: Re: Precise Digital Exposure
Post by: MarkL on April 17, 2015, 08:06:47 am
I've found that ETTR techniques usually result in subtle and undesirable color shifts in the finished result. I'd rather have better colors and tonalities and a bit more noise.

As soon as any big adjustment are done in post this seems to happen. The file may have a better SNR but the raw converter algorithms are not perfect. It is a big like making big EQ changes in audio.
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 17, 2015, 09:35:35 am
ETA: Also, can anyone recommend any good online resource regarding camera profiles with Lightroom?  Every now and then I revisit some of my old canon 5D raws and I'm nowhere near as happy with the LR 'default' camera profile settings for the 5D as I am for my Nikon. 


In this 30 minute video, we’ll look into the creation and use of DNG camera profiles in three raw converters. The video covers:

What are DNG camera profiles, how do they differ from ICC camera profiles.
Misconceptions about DNG camera profiles.
Just when, and why do you need to build custom DNG camera profiles?
How to build custom DNG camera profiles using the X-rite Passport software.
The role of various illuminants on camera sensors and DNG camera profiles.
Dual Illuminant DNG camera profiles.
Examples of usage of DNG camera profiles in Lightroom, ACR, and Iridient Developer.

Low Rez (YouTube):
http://youtu.be/_fikTm8XIt4

High Rez (download):
http://www.digitaldog.net/files/DNG%20Camera%20profile%20video.mov
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 17, 2015, 09:41:24 am
Yeah, his site doesn't actually have anything about LR camera profiles.

I'm more after a condensed resource on the topic.  I don't really have time to chase down individual posts by people all over the internet.  I'm hoping someone has put together a resource on this.

you have a choice - read the original or read the hearsay...
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 17, 2015, 10:19:01 am
Not completely. You know well that the image shown on the EVF won't fully represent the possibilities of the image saved to file. A degree of understanding of how a raw converter will handle bringing out shadow detail or highlight recovery remains an important skill, akin to 'pre-visualisation'.

Ok ok, let's be more precise:

- If you shoot JPEG an EVF is a high precision tool that turns your camera into a WYSYWYG device

- If you shoot RAW an EVF is a far more powerful and user friendly tool to decide exposure than an OVF
Title: Re: Precise Digital Exposure
Post by: bernie west on April 17, 2015, 10:30:18 am
you have a choice - read the original or read the hearsay...

Or read a good summary...

Thanks dd for the link.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 17, 2015, 10:35:28 am
Or read a good summary...
did he still states that ICC profiles are not scene-referred ?
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 17, 2015, 11:02:48 am
As soon as any big adjustment are done in post this seems to happen. The file may have a better SNR but the raw converter algorithms are not perfect. It is a big like making big EQ changes in audio.

Correcting exposure needs no algorithm.

If 1000 photons produce RAW level=500, then 2000 photons (+1EV in exposure) will produce RAW level=1000

So correcting RAW exposure is simply scaling all its RAW numbers by a constant factor. e.g. 1000/2 recovers 500. When color shifts take place is surely because some data got clipped in the RAW file or wrong post processing technique was applied.

This is a HDR mix of a 3-shot bracketing obtained at 2EV intervals (each gray color indicates the source RAW file). The resulting composite is color seamless. Exposure correction by -2EV and -4EV was just a matter of multiplying numbers by a constant value.

(http://www.guillermoluijk.com/misc/seamless.jpg)

(http://www.guillermoluijk.com/misc/mazinger.jpg)

Regards!

Title: Re: Precise Digital Exposure
Post by: digitaldog on April 17, 2015, 02:03:37 pm
did he still states that ICC profiles are not scene-referred ?
You should take this up with the ICC (you know about them right?). From their white paper #17, Using ICC profiles with digital camera images:
Quote
Also, ICC color management workflows generally assume that the colorimetry expressed in the PCS is of a [color-rendered] picture, and not of a scene. There is currently no mechanism to indicate that the colorimetry represented in the PCS by a camera profile is relative scene colorimetry. Even if there were, use of the PCS to contain relative scene colorimetry is not fully compatible with current ICC workflows, which assume color rendering has been performed. This distinction is especially important with respect to highlight reproduction. Many scenes contain highlights that are brighter than the tone in the scene that is reproduced as white in a picture. An important part of the color rendering process is selection of the tone in the scene that is considered "edge of white", and graceful compression of brighter tones to fit on the reproduction medium (between the "edge of white" tone and the medium white).

Maybe they will invite you to be a member of the ICC photography work group committee* and you can write your own piece on the subject for their site.
Quote
An ICC working group* has been formed to attempt to address issues with the use of ICC profiles in digital photography applications, but at present progress is difficult. Even if improved characterization targets (such as narrow-band emissive targets) and profiling tools are introduced, colorimetric intents will still be illumination specific, and perceptual intents will optimally be scene-specific. Some argue that scene-to-picture color rendering should be restricted to in- camera processing and camera raw processing applications, and correction of color rendering deficiencies limited to image editing applications.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 18, 2015, 08:19:57 am
Correcting exposure needs no algorithm.

If 1000 photons produce RAW level=500, then 2000 photons (+1EV in exposure) will produce RAW level=1000

So correcting RAW exposure is simply scaling all its RAW numbers by a constant factor. e.g. 1000/2 recovers 500. When color shifts take place is surely because some data got clipped in the RAW file or wrong post processing technique was applied.

I think what MarkL was referring to (and I've wondered the same thing in the past) is pulling back exposure in LR, which is dependent on what demosaicing algorithm is used and what camera colour profile is used.  And then, as you say, you can get their attempt at rebuilt highlights if one or more channels has clipped.   Essentially, it seems as if the "exposure" slider in LR isn't a simple linear function.  Or perhaps it's linear over most of the range, but non-linear near the highlights.  I pretty much gave up on bothering with ETTR years ago due to the crappy renderings I was getting out of LR for images that were pushed right to the ETTR limit.  I got much better renderings detail and gradient wise with dcraw, but it was too much effort getting good colour due to the need to go through two different bits of software just for a half-decent initial rendering.  Then to photoshop if necessary after that.  Since pulling back just a tad from true ETTR, I've had much better results.  But if there is a fool proof way of normalising true ETTR images, then I'm all for learning it.
Title: Re: Precise Digital Exposure
Post by: BobD on April 18, 2015, 08:28:01 am
I've found that ETTR techniques usually result in subtle and undesirable color shifts in the finished result. I'd rather have better colors and tonalities and a bit more noise.

Matthew,

Do you use a custom "camera profile" in your processing to assure that these "subtle and undesirable color shifts" are accurate to the scene's color? If not accurate then precise?

Bob 
Title: Re: Precise Digital Exposure
Post by: Internaut on April 18, 2015, 08:41:24 am
I've found that ETTR techniques usually result in subtle and undesirable color shifts in the finished result. I'd rather have better colors and tonalities and a bit more noise.

This usually happens if one colour channel is blown.  I must admit, I don't fully understand the article, but the gist of it is, I think, being able to predict precisely how far you can push the exposure, from behind the camera, having metered for a given highlight (i.e. avoiding a problem we both encounter).  When I get time, I'll have a further read.  At the moment, my ETTR technique, with my smaller format cameras, is bracketing with matrix/pattern metering.  Modern sensors are good, so it's not unusual to find a +1 or +1.3 exposure with fully recoverable highlights, depending on the scene. 

Of course, the flip side of the debate is whether ETTR applies at all, with modern sensors, for most photographers and shooting situations.
Title: Re: Precise Digital Exposure
Post by: Torbjörn Tapani on April 18, 2015, 09:09:10 am
Of course, the flip side of the debate is whether ETTR applies at all, with modern sensors, for most photographers and shooting situations.

The way I see it, ETTR will always apply. Sure I know what you mean, we can recover shadows just fine on newer chips. But shot noise is a property of light itself. More exposure gives cleaner shadows, simple as what.
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 18, 2015, 10:34:23 am
I think what MarkL was referring to (and I've wondered the same thing in the past) is pulling back exposure in LR, which is dependent on what demosaicing algorithm is used and what camera colour profile is used.  And then, as you say, you can get their attempt at rebuilt highlights if one or more channels has clipped.   Essentially, it seems as if the "exposure" slider in LR isn't a simple linear function.  Or perhaps it's linear over most of the range, but non-linear near the highlights.

It's possible but that would be a LR issue. In the example I posted the dark areas were pulled down by 4 stops (i.e. RAW linear values were divided by 16.0), and colour is perfect.
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 18, 2015, 11:20:57 am
The way I see it, ETTR will always apply.
Yes, I agree, of course it will. ETTR is just optimal exposure. The exposure is either such it produces optimal data or it isn’t and there are degrees in which sub optimal data affects our work. Now what the comment might imply is that less than optimal exposure (ETTR) will produce results no one can see and that I suppose is possible. This is much like the use of editing in high bit (16-bit) because we know rounding errors could, possibly result in data loss that is visible at some point on some output devices. It might not. But why take the chance?
Title: Re: Precise Digital Exposure
Post by: BobD on April 18, 2015, 12:01:44 pm
...perhaps it’s fair to say rendering is part development (normalizing ETTR) and part subjective (making ‘the print’).

Andrew,

I think there are parallels to B&W film processing and digital image processing. A while back, I posted on my blog what I think are 4-Phases of digital image processing and it’s parallel to black and white film. I think it fits this discussion well.

Here’s the link http://bobd.tv/lightroom/4-phases-of-processing-the-digital-image/
I’d like you to take a look at it and give me your thoughts.

BTW, I'm the guy that wrote the article on "The Optimum Digital Exposure" on LuLa. The one that Anders says “presents essentially nothing new".  Have you read my LuLa article? A lot of the concepts are from my book which, if your interested, I can send to you.

Bob DiNatale
Title: Re: Precise Digital Exposure
Post by: Rhossydd on April 18, 2015, 12:11:28 pm
I posted on my blog what I think are 4-Phases of digital image processing and it’s parallel to black and white film. I think it fits this discussion well.
But it's wrong. See #reply 32 http://forum.luminous-landscape.com/index.php?topic=99565.msg815359#msg815359
Your idea of stage 1 'import' where you apply "Apply Camera Defaults" is NOT like film development because at any stage in the future those defaults can be modified and changed.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 18, 2015, 01:30:41 pm
Also, ICC color management workflows generally assume that the colorimetry expressed in the PCS is of a [color-rendered] picture, and not of a scene.

but we are not talking about PCS in camera (input) profiles in icc containers because dcp profiles are too directing color transform into some [color-rendered] picture, so you have absolutely the same situation... we are talking that the input when building such profiles is a scene-referred data
Title: Re: Precise Digital Exposure
Post by: MarkL on April 18, 2015, 02:20:18 pm
I think what MarkL was referring to (and I've wondered the same thing in the past) is pulling back exposure in LR, which is dependent on what demosaicing algorithm is used and what camera colour profile is used.  And then, as you say, you can get their attempt at rebuilt highlights if one or more channels has clipped.   Essentially, it seems as if the "exposure" slider in LR isn't a simple linear function.  Or perhaps it's linear over most of the range, but non-linear near the highlights.  I pretty much gave up on bothering with ETTR years ago due to the crappy renderings I was getting out of LR for images that were pushed right to the ETTR limit.  I got much better renderings detail and gradient wise with dcraw, but it was too much effort getting good colour due to the need to go through two different bits of software just for a half-decent initial rendering.  Then to photoshop if necessary after that.  Since pulling back just a tad from true ETTR, I've had much better results.  But if there is a fool proof way of normalising true ETTR images, then I'm all for learning it.

Yes that's right. It might not need them but certainly in LR if you pull the exposure around you can see non-linear changes to the channels in the histogram.
Title: Re: Precise Digital Exposure
Post by: BobD on April 18, 2015, 02:39:44 pm
Anders,

How did you arrive at your +2½ to 3 ½ stops for highlights? Trial and error? Experience?
“I was able to establish… key values to spot meter, where… the location of extreme highlight and shadow values with RGB values maintained or with recovery”

It doesn't need to be this complicated! Simply expose a gray card to +5 stops in 1/3 stop increments. This will produce 16 exposures. View these 16 exposures in your digital Raw Processing software and see which exposure reads 99% brightness - the Optimum White Point [OWP] for your system (meter/camera/software) combination.

My experience with all my cameras shows the [OWP] falls between +3_2/3 stops to +4_1/3 stop (arrived at by the exposure just before I reach LR's red “Highlights Clipping” warning). This is the Exposure Bias [EB] that needs to be applied to your spot meter reading of the brightest area in your scene. This will produce 99% brightness in your raw software- the [OWP].

I can appreciate you liking your Pentax spot meter… I had one also. However, I now use a Sekonic L-758. This is a wonderful tool because you can place the Exposer Bias [EB] you found in your testing into the L-758's ISO2 button. Now, you simply read the brightest part of the scene with your spot meter and press the ISO2 button… the L-758 displays the exposure settings needed to place the brightest part of the scene at the Optimum White Point [OWP] - 99% brightness in your software.

As far as applying the Zone System Process/Development routine to the digital process – this gets a little tricky. Digital images are so much different than photographic film images. A digital image is comprised of monochromatic mosaic linear B&W digital data. A photographic film image is a panchromatic emulsion with continuous tone densities that responds to non-linear (geometric progression) of light.
Title: Re: Precise Digital Exposure
Post by: BobD on April 18, 2015, 03:09:04 pm
But it's wrong... Your idea o f stage 1 'import' where you apply "Apply Camera Defaults" is NOT like film development because at any stage in the future those defaults can be modified and changed.
Rhossydd

Yes. After Import there are more modification and changes with raw processing software. That is Phase2 “Global” and Phase3 “Selective”.

Phase2 “Global” in film/print processing is your base exposure set on your Gray Lab timer… “Minimum exposure for Maximum black”.
Phase3 “Selective” in film/print processing parallels “dodging & burning” done under the enlarger.
This is not even taking in to consideration that the digital tools use adaptive algorithms … neither linear nor geometric but adapting as needed based on the data it sees!

All these unique aspects of digital image processing mentioned above is the flaw in Anders' idea of trying to parallel digital processing to the Zone System.
Title: Re: Precise Digital Exposure
Post by: bjanes on April 18, 2015, 04:05:52 pm
I think what MarkL was referring to (and I've wondered the same thing in the past) is pulling back exposure in LR, which is dependent on what demosaicing algorithm is used and what camera colour profile is used.  And then, as you say, you can get their attempt at rebuilt highlights if one or more channels has clipped.   Essentially, it seems as if the "exposure" slider in LR isn't a simple linear function.  Or perhaps it's linear over most of the range, but non-linear near the highlights.  I pretty much gave up on bothering with ETTR years ago due to the crappy renderings I was getting out of LR for images that were pushed right to the ETTR limit.  I got much better renderings detail and gradient wise with dcraw, but it was too much effort getting good colour due to the need to go through two different bits of software just for a half-decent initial rendering.  Then to photoshop if necessary after that.  Since pulling back just a tad from true ETTR, I've had much better results.  But if there is a fool proof way of normalising true ETTR images, then I'm all for learning it.

The behavior of the exposure slider in ACR/LR depends on which process version is in use. With the current process, PV2012, the exposure slider as well as all the other sliders in the basic panel are image adaptive. See the post (https://forums.adobe.com/message/4253400) by Eric Chan on the Adobe forums. Auto highlight recovery is always in use, which can hide overexposure in the raw file. With the earlier process version, PV2010, I think that the exposure slider was not image adaptive (that is was linear) unless highlight recovery is taking place.

The newer Adobe profiles have hue twists, which can cause problems when exposure and recovery are in use. See SandyMc's post here (http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html). These are introduced with profiles that have lookup tables in addition to the matrix math. Sandy's DCPtool an address this problem. The whole topic is complicated and beyond the scope of my expertise. However, in my experience the hue shifts are not problematic if there are no blown highlights.

Hopefully, some of forum heavies will enter into discussion.

Bill
Title: Re: Precise Digital Exposure
Post by: fdisilvestro on April 18, 2015, 06:33:18 pm
in LR if you pull the exposure around you can see non-linear changes to the channels in the histogram.

That's expected because the scaling is performed on the raw channels, not the rendered RGB channels. Additionally the histogram representation in LR is not linear and the vertical axis is autoscaled.

You need a more reliable way to determine if there are color shifts or not
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 18, 2015, 07:46:11 pm
That's expected because the scaling is performed on the raw channels, not the rendered RGB channels.
Adode code does not do exposure correction on raw channels, the data are already after demosaick and then after color transform...
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 18, 2015, 10:46:06 pm
BTW, I'm the guy that wrote the article on "The Optimum Digital Exposure" on LuLa. The one that Anders says “presents essentially nothing new".  Have you read my LuLa article? A lot of the concepts are from my book which, if your interested, I can send to you.

My article - "A publication titled Perfect Exposure on my website dates from 2011 and was reviewed by Michael Reichman at the time. It points out both the simplified concept to upon one single spot meter reading determine exposure for precise location of the latitude of a sensor compared to a scene, and much more. For correctness this all dates back to what Ansel Adams wrote relating to the Zone System and how to determine spot metering for black and white negative film, Polaroid Land Print and more."
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 18, 2015, 10:46:26 pm
How did you arrive at your +2½ to 3 ½ stops for highlights? Trial and error? Experience?
“I was able to establish… key values to spot meter, where… the location of extreme highlight and shadow values with RGB values maintained or with recovery”
It doesn't need to be this complicated! Simply expose a gray card to +5 stops in 1/3 stop increments. This will produce 16 exposures. View these 16 exposures in your digital Raw Processing software and see which exposure reads 99% brightness - the Optimum White Point [OWP] for your system (meter/camera/software) combination.

Bob,

Precise Digital Exposure is not complicated, it is simple  :). Yes, I followed the recommendation by Ansel Adams (his book The Negative) to expose at stop increments, and I applied by 1/3 increments to an 18% grey card for the full exposure range of my digital back. Included and next to the grey card I also had red, green, and blue paper cards. This is described in my paper.

Photographically speaking a digital sensor can be viewed no different than other photographic media such as was negative B&W film, slide film, negative colour film, Polaroid Land prints and more. We use aperture, shutter, film speed etc to control the exposure. Thereby we can test the spectral response (sensitivity to light) by same means as Ansel, which is what you also point out with the increments of stops in above. Therefore the principles of the Zone System apply to digital perfect well as they did to other media, however as already pointed out by Ansel they need an adaptation in how we apply them to a new media based on the characters of that media, which Ansel did to several media and which I have pointed out for digital in my article and paper. - A very essence, contrary to the belief of some posters in above, old and traditional methods of photography apply and they can be more precise for situations when we need or want so precise.

In my post above I attached the characteristic curve which I arrived at per increment photos of test cards and which is in my paper. Based on this I decided for which number of stops above mid tone that I was comfortable to spot meter before clipping of first channel and before clipping of last channel (which was based on what I viewed as viewable texture information, again similar to Adams did). I also do not want to risk clipping beyond these for readings with my spot meter, since I view important to spot meter just before these clippings occur. Because we have RGB sensors, if my understanding is correct the clipping may vary pending on sensitivity of the coloured filters over different sensors, and pending on any colour cast of the light. If you have looked into this please advise.

What I find advantage with the Pentax Spot Meter is the fact that I can with great simplicity view and visualise with a zone scale for the above highlight points and also for ease of inspecting/visualise also the shadow end. I am able to precise expose based upon one of, or two, highlight points, just before clipping of first and last channels, and can pending on if it is high dynamic scene also inspect at the shadow end. The disadvantage with the Pentax Spot Meter is that it is discontinued...

Anders
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 18, 2015, 11:27:22 pm
AN ADDED EXPLANATION

In an attempt to explain more simple the effectiveness of the method of Precise Digital Exposure...

Having determined the following extreme points per 1/3 stops increment testing (textural maintained):
(A) Stops above 18% grey card just prior to clipping of first channel
(B) Stops above 18% grey card just prior to clipping of last channel
(C) Stops below 18% grey card just prior black point
(D) Same as C) with maximum negative exposure in post. My assumption is that deliberate negative exposure is only of interest to extend shadow end due benefits of ETTR to other data.

This brings a tool to determine a Precise Digital Exposure based upon the exposure range of my digital back compared to that of a scene. Hence my metering is simply a decision for determining the placing of the exposure range for a specific scene, and which in most simple application can be done upon one spot meter reading.

I can do so per:
1) Spot metering based on (A) for a brightest point in a scene - For simple situations this is all I need.
2) Spot metering based upon (A) and (B) - With sky and high dynamic range of sky this is what I can apply. Obvious, beyond clipping of one channel is not optimum data but what I find acceptable for sky etc. Per choice I can also decide exposure less than (B) for intention to later apply an extended shoulder in post to be able to create a more extended shoulder transition into highlights similar to film.
3) For a high dynamic range scene, inspect for (C) and (D). Further, pending on the scene at hand, I can check for what falls on -2 ~ -3 stops and upwards which is what for my digital back per my trial and error appear to maintain quality data when some require to be brought upwards in post.

Please refer to my paper for download, see link in article. Pointed out in it is more information and parallels referred to Ansel Adams principles.

Anders
Title: Re: Precise Digital Exposure
Post by: ErikKaffehr on April 19, 2015, 01:54:41 am
Hi,

Most raw converters have some hue twists that may depend on exposure. ETTR may affect those "hue twists".

http://dcptool.sourceforge.net/Hue%20Twists.html

http://chromasoft.blogspot.se/2009/02/visualizing-dng-camera-profiles-part-1.html

Best regards
Erik
I've found that ETTR techniques usually result in subtle and undesirable color shifts in the finished result. I'd rather have better colors and tonalities and a bit more noise.
Title: Re: Precise Digital Exposure
Post by: ErikKaffehr on April 19, 2015, 02:15:26 am
Hi,

It is very simple, optimal capture is where as many photons as possible are collected, without clipping any channel.

The reason is simply that shot noise is minimised at maximum exposure, or more correctly Signal/Noise ratio is maximised with maximum possible exposures.

The issue is a bit muddled when we leave base ISO, as readout noise may be lower on some sensors at higher ISO. Other muddling factors are that raw converters cheat a lot with histograms. Lightroom always seems to do some highlight compression  and can be quite cavalier about highlight recovery. Capture One's "film curve" makes an ETTR image look horribly overexposed and the histogram goes with it. :-) We may ask Adobe and Capture one for correct raw histograms before asking camera vendors. :-)

Personally, I used to use a spot meter in film times. With slide film I exposed for the highlights and with B&W film I exposed for the shadows. The idea with the zone system was that exposure and development were interrelated. So, processing was taken into account at exposure and different processing was applied to individual exposures.

With digital all this makes little sense. Expose ETTR, or for mid tones if concerned about hue twists. Once capture has been made, we can apply any kind of processing.

Using a spot meter can be useful for finding optimal exposure. Personally I stopped spot metering a couple of weeks after going into digital. With film I still use the spot meter, as the film back has no histogram.

Best regards
Erik

Yes, I agree, of course it will. ETTR is just optimal exposure. The exposure is either such it produces optimal data or it isn’t and there are degrees in which sub optimal data affects our work. Now what the comment might imply is that less than optimal exposure (ETTR) will produce results no one can see and that I suppose is possible. This is much like the use of editing in high bit (16-bit) because we know rounding errors could, possibly result in data loss that is visible at some point on some output devices. It might not. But why take the chance?
Title: Re: Precise Digital Exposure
Post by: bernie west on April 19, 2015, 05:56:42 am
Yes that's right. It might not need them but certainly in LR if you pull the exposure around you can see non-linear changes to the channels in the histogram.

It will definitely be non-linear in demosaiced white-balanced images.  I'm more wondering if it's non-linear on the underlying raw data.  As Guillermo says, if you 'normalise' your image with linear raw adjustments, then it will by definition (more or less) turn out the same as a 'normal' jpg exposure.  That you and I and others are finding that they sometimes aren't the same would suggest that the Exposure slider in LR isn't working in a fully linear way with the underlying raw data.  I haven't played with raw data since back in 2008 when we where all trying to work out the optimal approach to ETTR, so I might shoot some test shots and see exactly what is happening with my current setup (Nikon with PV2012 vs Canon with some other earlier PV), when I get some time.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 19, 2015, 06:01:42 am
The behavior of the exposure slider in ACR/LR depends on which process version is in use. With the current process, PV2012, the exposure slider as well as all the other sliders in the basic panel are image adaptive. See the post (https://forums.adobe.com/message/4253400) by Eric Chan on the Adobe forums. Auto highlight recovery is always in use, which can hide overexposure in the raw file. With the earlier process version, PV2010, I think that the exposure slider was not image adaptive (that is was linear) unless highlight recovery is taking place.

The newer Adobe profiles have hue twists, which can cause problems when exposure and recovery are in use. See SandyMc's post here (http://chromasoft.blogspot.com/2009/02/adobe-hue-twist.html). These are introduced with profiles that have lookup tables in addition to the matrix math. Sandy's DCPtool an address this problem. The whole topic is complicated and beyond the scope of my expertise. However, in my experience the hue shifts are not problematic if there are no blown highlights.

Hopefully, some of forum heavies will enter into discussion.

Bill

Interesting stuff!  If it's beyond your understanding, it will be beyond mine as well.  I've only just started to bother changing from the default Adobe profile to the camera specific profiles, so I'm hoping I might see some changes now.  I think I'll shoot some test shots and take a look at the raw data and do conversions in dcraw vs LR and see if I can get a picture of what's going on for my specific Nikon model.  Either way, I'm much more happy with adobe default renderings of my Nikon raws than I ever was with the raws off my Canon.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 19, 2015, 06:02:54 am
Adode code does not do exposure correction on raw channels, the data are already after demosaick and then after color transform...

It has to be doing something with the raw data, or else it could never rescue blown highlights.
Title: Re: Precise Digital Exposure
Post by: bjanes on April 19, 2015, 07:36:23 am

How did you arrive at your +2½ to 3 ½ stops for highlights? Trial and error? Experience?
“I was able to establish… key values to spot meter, where… the location of extreme highlight and shadow values with RGB values maintained or with recovery”

It doesn't need to be this complicated! Simply expose a gray card to +5 stops in 1/3 stop increments. This will produce 16 exposures. View these 16 exposures in your digital Raw Processing software and see which exposure reads 99% brightness - the Optimum White Point [OWP] for your system (meter/camera/software) combination.

My experience with all my cameras shows the [OWP] falls between +3_2/3 stops to +4_1/3 stop (arrived at by the exposure just before I reach LR's red “Highlights Clipping” warning). This is the Exposure Bias [EB] that needs to be applied to your spot meter reading of the brightest area in your scene. This will produce 99% brightness in your raw software- the [OWP].

Bob,

I would not advise using LR/ACR to determine where the sensor saturates, especially with the current process version (PV2012) which applies automatic highlight recovery. In addition, LR/ACR have a baseline exposure adjustment. For example, the BaselineExposure for the Nikon D3 is +0.5 EV and the values for the D70 and D200 are -0.75 and -0.5 respectively. These adjustments increase or decrease the rendered values by the amount of the baseline adjustment.

Most light meters (including those built into our cameras) are calibrated to yield 12% saturation rather than the 18% expected for mid-gray (see Thom Hogan (http://www.bythom.com/graycards.htm)). This allows 0.5 EV of highlight headroom. The difference between 18% and 100% expressed in stops is log base 2(1.0/0.18) or 2.5 stops. For 12% the difference is 3 stops. To place a metered tone at saturation with a camera calibrated for 12%, one must increase the metered exposure by 3 stops. If you are willing to use highlight recovery of 0.5 to 1.0 EV in the raw converter, this could be increased to 3.5 or 4.0 stops, which is consistent with your empirically observed results. The rendered values with ACR/LR would be affected by the baseline offset.

Regards,

Bill
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 19, 2015, 07:42:18 am
Adode code does not do exposure correction on raw channels, the data are already after demosaick and then after color transform...

If data is demosaiced and linearly (matrix) colour transformed, and even if it has been gamma transformed (this only applies to pure gamma lifting, not sRGB-like gammas), it should be possible to change exposure without any hue shift through scaling the RGB values by a constant value.

I think the point is that commercial RAW developers could apply more sophisticated non-linear processing (colour transform, gamma, highlight/shadow recovery strategies) either prior to the exposure adjustment, or after it but exposure dependent, that surely have other benefits but ruin the linear properties of the colorimetrics in RAW data. The differences are not big though, so only colour peepers could be interested in these hue shifts, not the regular user.

Regards!
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 19, 2015, 11:03:53 am
It has to be doing something with the raw data, or else it could never rescue blown highlights.
no, inventing the data in "blown" highlights does not need (it is not about what is better - but about what is possible) "raw" data... you have for example 2 channels in their internal RGB color space after the color transform w/o "clipping" - so you can use that information to invent the data in the 3rd channel...
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 19, 2015, 11:14:40 am
If data is demosaiced and linearly (matrix) colour transformed, and even if it has been gamma transformed (this only applies to pure gamma lifting, not sRGB-like gammas), it should be possible to change exposure without any hue shift through scaling the RGB values by a constant value.

which is exactly why it is no longer the raw data - it is already 1) demosaicked and 2) color transformed into some proper color space and 3) with new Adobe profiles a LUT (=non linear transform) can be applied before exposure slider operation (Adobe supplies 2.5D LUTs, but you can get a profile that does full 3D transform if you want, that is totally allowed - so there you go with hue shifts)...

Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 19, 2015, 11:29:18 am
In addition, LR/ACR have a baseline exposure adjustment.
Adobe code has one hidden expocorrection inside it (hardcoded, might be zero) and one picked (if present) from DCP profiles (BaselineExposureOffset)... so the real hidden expocorrection is a sum of two components... convert a raw to DNG using Adobe's code (ACR, LR, DNG converter) and check "BaselineExposure" tag for what is in the code (vs what is in profiles)... that tag is "ISO" dependent... it can be different for raws shot with different nominal "ISO"s... for example Sony A7  = ISO 50 : BaselineExposure = -0.65 and ISO 100+ : BaselineExposure = +0.35 ... you can also check Fuji's x-trans cameras for high ISOs  ;D
Title: Re: Precise Digital Exposure
Post by: bernie west on April 19, 2015, 10:27:23 pm
no, inventing the data in "blown" highlights does not need (it is not about what is better - but about what is possible) "raw" data... you have for example 2 channels in their internal RGB color space after the color transform w/o "clipping" - so you can use that information to invent the data in the 3rd channel...

I'm talking about highlights blown in the jpg rgb, but not in the raw data.  LR absolutely works on the raw data.  The only question is: does the exposure slider work linearly on the underlying raw data?  And from what others have said, it appears it doesn't.  This verifies my feeling over the years from playing around with extreme ETTR (i.e. exposed just to the point of raw clipping).
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 19, 2015, 11:34:06 pm
LR absolutely works on the raw data. 

no, it does not... as it was noted - demosaick and color transform (linear /matrix/ or non linear /matrix + LUT or dummy matrix + LUT/) before exposure correction in UI... that makes it a non-raw data... demosaick alone makes it a non-raw data... think about this - Adobe code works identically with both original raw files and with lossless linear DNG files converted from original raw files using that code, because it ( the code) is written to do this by design - that is the Adobe's strategy - no difference between the original raw files and lossless (linear or not) DNG when using LR/ACR...

do you have the original raw channels in linear DNG files ? but you still can "recover" (what a wrong word, as if the data is actually present !) clipped highlights there...

still think that "or read a good summary" is a good idea ?
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 12:08:28 am
You're kind of missing the point.  The distinction under discussion was between raw and fully rendered rgb values (i.e. the histogram in LR - that is, post everything, including white balance and gamma correction).  Not between raw and post demosaic/color-transform.  I accept your point that it's not working directly on the raw data itself, but in the context of this debate, it's still working very close to the raw data itself, as opposed to after white balancing and gamma correction, as is the case when you work on an image in photoshop.  If there was no benefit to be gained in this distinction we wouldn't bother doing any image manipulation in LR.  We'd just do it all in photoshop after rendering with dcraw or something equally similar and fast.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 01:06:44 am
I accept your point that it's not working directly on the raw data itself... as opposed to after white balancing and gamma correction

consider this - exposure correction is done there (ACR/LR) after WB operations and after color transform which might be as well applying some "gamma" (or any other curve - whatever is in the 1st stage LUTs by their design)

Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 01:12:14 am
We'd just do it all in photoshop after rendering with dcraw or something equally similar and fast.
consider this - such raw converter as lightzone ( http://www.lightzoneproject.org/ ) was doing everything after running dcraw executable ;)
Title: Re: Precise Digital Exposure
Post by: hjulenissen on April 20, 2015, 01:42:12 am
no, it does not... as it was noted - demosaick and color transform (linear /matrix/ or non linear /matrix + LUT or dummy matrix + LUT/) before exposure correction in UI... that makes it a non-raw data... demosaick alone makes it a non-raw data...
I'd be surprised if Adobe chose to do highlight recovery on data that has been processed by demosaic, and even more if it has been processed by (potentially nonlinear) color processing. After such processing, the saturation point is hard to define and a single blown sensel in a single channel can affect all 3 channels in a spatial neighborhood.

Not saying that it can't be done, only that it (from my POV) seems like making the task harder than it needs to be.

-h
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 02:25:40 am
consider this - exposure correction is done there (ACR/LR) after WB operations and after color transform which might be as well applying some "gamma" (or any other curve - whatever is in the 1st stage LUTs by their design)



So now it's after demosaic, colour transform, AND white balance??  What's it going to change to next comment?

I find it hard to believe it would be operating on the data after WB, as it would recover stuff-all highlights that were blown as indicated on the LR histogram.  As I said, what would be the point of using LR if it only worked on essentially rendered data? 
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 02:46:12 am
So now it's after demosaic, colour transform, AND white balance??  What's it going to change to next comment?

that was always the case - where did you see the changes ?

Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 02:48:54 am
I'd be surprised if Adobe chose to do highlight recovery on data that has been processed by demosaic

did you see the note about linear DNG files ? that is a fact - those have demosaicked data, yet there are no difference between them and original raw files if you use Adobe software and conversion to linear DNG was done by Adobe software (the same version for DNG conversion and raw conversion, lossless DNG, just to make sure)

Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 02:55:39 am
that was always the case - where did you see the changes ?



You originally said 'demosaic and colour transform'.  Twice.  Unless "colour transform" includes white balance.  I assumed colour transform means conversion to a colour space.  

But once again, this is all dodging around the point of the discussion.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 03:06:01 am
You originally said 'demosaic and colour transform'.  Twice.  Unless "colour transform" includes white balance.  I assumed colour transform means conversion to a colour space.  

But once again, this is all dodging around the point of the discussion.
sorry... I forgot to insert the WB between demosaick and color transform... WB operation is closely related to a color transform as ColorMatrix tags are part of dcp profile (in fact if your profile has only CM tags - those are guiding both WB and color transform operations) and potential interpolation of various matrices/luts from dcp profile also depends on your selection of white balance in ACR/LR UI (when you have a dual illuminant profile).
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 03:19:10 am
So how is it possible for the exposure slider to 'normalise' blown highlights in the rendered data then?  And no, I'm not talking about 'highlight reconstruction' from 1 or 2 channels.  I'm talking about when all three channels are blown in the LR histrogram (and in the jpg preview).  If the slider worked on the post rendered data, then it shouldn't be able to pull blown highlights back into normal ranges.
Title: Re: Precise Digital Exposure
Post by: Guillermo Luijk on April 20, 2015, 03:27:20 am
For those interested I opened a specific thread not to deviate the Precise Digital Exposure discussion with the hue shift off topic, here:

http://forum.luminous-landscape.com/index.php?topic=99659.0

Regards
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 08:51:13 am
So how is it possible for the exposure slider to 'normalise' blown highlights in the rendered data then?  And no, I'm not talking about 'highlight reconstruction' from 1 or 2 channels.  I'm talking about when all three channels are blown in the LR histrogram (and in the jpg preview). 

then Adobe invents the data for all 3 RGB channels to make it look nice visually - for example you can a try to produce nice gradual transition to "white"... I am not sure why to you think that it is necessary to have "raw" data to engage in inventing things that looks nice ? the mere fact that some software (PS ?) does not invent things does not mean anything except that it was not designed to do so.

If the slider worked on the post rendered data, then it shouldn't be able to pull blown highlights back into normal ranges.

why :D ? they are all numbers and you do whatever you want with them...
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 10:03:21 am
You really don't make much sense.


Adobe doesn't invent data for all three channels, as you could confirm for yourself if you did a comparison between a dcraw conversion and a LR conversion.

Recovering detail from a nominally blown raw file doesn't appear to be a case of making a "nice gradual transition to "white".  It's about bringing actual detail back into the image.  
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 10:14:43 am
Top one - jpg out of camera.  Bottom one - highlights recovered in Lightroom.

(http://static.photo.net/attachments/bboard/00Q/00QTjS-63617584.jpg)

(http://static.photo.net/attachments/bboard/00Q/00QTjW-63617684.jpg)
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 10:35:18 am
You really don't make much sense.

 :D

Adobe doesn't invent data for all three channels, as you could confirm for yourself if you did a comparison between a dcraw conversion and a LR conversion.

if the data in all 3 channel is "clipped" what do you think is happening ?

You appear to be trolling.  Have you ever recovered detail from a nominally blown raw file?  I have, many times, and no, it's not a case of making a "nice gradual transition to "white".  It's about putting actual detail back into the image.  

Bernie, that's why I suggested to read the originals, not watch 30 minutes hearsay  ;)

PS: if the details "appear" that simply means "nominally blown raw file" was not "nominally blown" in all channels
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 20, 2015, 10:37:53 am
if the data in all 3 channel is "clipped" what do you think is happening ?
ASAIK, based on text from Thomas and Eric at Adobe, if one (perhaps two?) channels are clipped, they can reconstruct them from the remaining data. I don't believe they've ever indicated they can do this if all three are actually clipped although PV2012 *may* be different.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 10:45:15 am
Top one - jpg out of camera.  Bottom one - highlights recovered in Lightroom.
that does not illustrate anything, Bernie... what you see on screen (and what raw converter in camera saves as "OOC JPG") is not what is inside the raw converter after demosaick and all those operations before exposure adjustment for its code to work with, that's it... when you play with exposure adjustment in LR/ACR the code does not work with some fixed result of some previous exposure adjustment (like when you load that "OOC JPG")...
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 10:46:11 am
:D

if the data in all 3 channel is "clipped" what do you think is happening ?

Exactly (in a broad sense) what is happening in DCRAW when all three channels are clipped in the jpeg and LR histogram.  As I said, you can simply test this yourself by doing a side by side raw conversion with DCRAW vs LR.  Have you ever tested this yourself?  It doesn't sound like it.  I and a lot of others have, and the only differences are small differences.  BOTH raw converters recover more or less the same highlight detail.  LR, because it isn't truly linear as described by a number of posters in this thread, can get some funky stuff happening in the colours sometimes.

Quote
PS: if the details "appear" that simply means "nominally blown raw file" was not "nominally blown" in all channels

I don't think you've really followed what this debate  is actually about.  No one is suggesting that data blown in the actual raw data itself can be saved (other than through rebuilding via other channels where possible).  What we are talking about is where the LR histogram indicates that all three channels are blown.  According to your description, that data is lost.  According to reality, it's not necessarily blown.  THAT'S the whole point about ETTR.  
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 10:51:21 am
that does not illustrate anything, Bernie... what you see on screen (and what raw converter in camera saves as "OOC JPG") is not what is inside the raw converter after demosaick and all those operations before exposure adjustment for its code to work with, that's it... when you play with exposure adjustment in LR/ACR the code does not work with some fixed result of some previous exposure adjustment (like when you load that "OOC JPG")...

Ok, well if that's the case, then the LR environment would have to be greater than 16bit to hold all this extra headroom that's outside the 16bit limit after white balance and gamma etc.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 10:54:57 am
ASAIK, based on text from Thomas and Eric at Adobe, if one (perhaps two?) channels are clipped, they can reconstruct them from the remaining data.

we were talking about the case when everything is clipped - then no details can be invented ("reconstructed"... one can understand how data is reconstructed in case of ECC codes - because it was not actually lost) but raw converter can keep the area nicely white (invent the color to fill - no details) for a pleasant visual look

for Bernie's benefit some quotes from EChan

"the DNG processing model performs a linearization of the original raw image values followed by demosaicing, then white balance. All of the other image stages follow. So to answer your question, all of the image ops except for linearization (which isn't under user control anyways) happens after demosaicing. "

"In the minus direction, the only difference is that ACR tries to keep clipped whites white (i.e., we did not feel it was photographically useful to let speculars turn into gray blobs, though we still let users accomplish that with the point curve).  Remember, when reducing (software) Exposure, there's nothing "above" the sensor saturation point (very unlike at capture time, where reducing the capture exposure can indeed record additional information), so there is a question of how to treat the whites."




Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 10:57:18 am
Exactly (in a broad sense) what is happening in DCRAW when all three channels are clipped in the jpeg and LR histogram.

we are not talking about dcraw or LR histogram - we are talking about when exposure correction is done inside LR/ACR... that is after demosaick, wb/color transform = not with raw data...

Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 11:08:29 am
According to your description, that data is lost.  
yes, data is lost (forever that is) - some details and color can be invented/guessed (plus Adobe tries to make it visually nice while inventing/guessing) if 1 or 2 channels still not clipped... if 3 are clipped - see the quote from EChan above... this is not a reconstruction like with ECC, this is purely guesswork - but it works quite close to the reality (or simply logical/pleasant from visual perspective) in a lot of cases

Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 11:15:45 am
we are not talking about dcraw or LR histogram - we are talking about when exposure correction is done inside LR/ACR... that is after demosaick, wb/color transform = not with raw data...



We actually were, which is why I made the point earlier that you don't seem to be following the debate very well.  Follow the quote trails back and you'll see.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 11:22:33 am
yes, data is lost (forever that is) - some details and color can be invented/guessed (plus Adobe tries to make it visually nice while inventing/guessing) if 1 or 2 channels still not clipped... if 3 are clipped - see the quote from EChan above... this is not a reconstruction like with ECC, this is purely guesswork - but it works quite close to the reality (or simply logical/pleasant from visual perspective) in a lot of cases



We're talking past each other.  Once again, no one is talking about the situation where channels are clipped in the linear off sensor raw data.  We are talking about where the data is clipped in the LR histogram and what would be a rendered jpg.  I.e. we are talking about ETTR.

And you haven't addressed a fundamental point from my previous post:  That is, for your description of how the exposure slider works to be accurate, it would have to be working in a data space greater than 16 bits.  So the question is:  Does it do this?  I can imagine that it could be doing this, but as some other poster said, it doesn't make a lot of sense (unless we are missing something).  
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 11:36:56 am
We actually were, which is why I made the point earlier that you don't seem to be following the debate very well.  Follow the quote trails back and you'll see.
may be you did, myself just interested in a code flow /stages/ inside ACR/LR...
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 11:59:25 am
We are talking about where the data is clipped in the LR histogram

then I am sorry - I have no interest in considering LR (or ACR) histogram at all, when someone needs to see the situation one needs to use tools like rawdigger or fastrawviewer or may be using UniWB raw files with special profile that compensates WB, hidden expocorrections, etc and process 2010 in LR/ACR - too much hassle then... as I am not using LR, but rather using ACR I have no issues to open raw in FRV from XnView/PhotoMechanic/whatever, understand what is (and where, and how) actually clipped and then call PS/ACR from FRV to work w/o paying attention to the histogram to guess what is really clipped...

again - if you want to discuss how other rawconverters do that or how/what histogram in ACR/LR works to indicate something please do not ask me (or argue with me or point me to) in this thread... my interest here is very narrow - that is data processing stages inside ACR/LR


That is, for your description of how the exposure slider works to be accurate

you fail to consider that it is not my description - it is how processing was explained (more then once) by developers, that is exposure correction stage (pull/push/leave as is) in ACR/LR is after demosaick, wb/color transform (dcp has several parts, matrices /CM, FM/ and HueSatMap LUTs are applied before exposure correction)

 
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 20, 2015, 03:05:55 pm
Lets get back on subject…

It is very simple, optimal capture is where as many photons as possible are collected, without clipping any channel.
...
Expose ETTR, or for mid tones if concerned about hue twists. Once capture has been made, we can apply any kind of processing.

Where to locate the maximum of photons collected, and how ?? ?
Ensure shadows will be adequate collected ?


EXAMPLES:

How to expose the attached scenes precisely; Ziczac Bridge, View from Resturant, Bierstadt (painting by)?

Consider exposing for allowing in processing for an extended shoulder transition into highlights of around 1/3 stops more than available highlight recovery (Velvia slide film, page 19-20 in my paper), and for maintaining as much quality pixel info as possible in the "important" parts of image.

Having determined the following extreme points per 1/3 stops increment testing (textural maintained):
(A) Stops above 18% grey card just prior to clipping of first channel
(B) Stops above 18% grey card just prior to clipping of last channel
(C) Stops below 18% grey card just prior black point
(D) Same as C) with maximum negative exposure in post. My assumption is that deliberate negative exposure is only of interest to extend shadow end due benefits of ETTR to other data.

I can check for what falls on -2 ~ -3 stops and upwards which is what for my digital back per my trial and error appear to maintain quality data when some require to be brought upwards in post.

I can spot meter for (A) or (B) above, or if I want to expose per knowing more for any of remaining in above. After, all I need is one shot. To add, although my Leaf back has an excellent RAW histogram, due 80MP it is slower to use than a DSLR and more battery hungry.

Anders
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 20, 2015, 03:14:40 pm
Quote
Lets get back on subject…
Where to locate the maximum of photons collected, and how ?? ?
What I did was akin to a film test. Bracket a controlled studio setup with a very white tile (BableColor). View where in my raw converter of choice I really clipped all three channels compared to no clipping the exposure below that.
Quote
Ensure shadows will be adequate collected ?
That's rather the point of ETTR (optimal exposure).
Once I know the limits of what the sensor can take based on sound exposure of the scene, I know how much more to Expose To The Right of the recommendation of the meter (in my test, incident).
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 20, 2015, 03:48:41 pm
painting
good one
Title: Re: Precise Digital Exposure
Post by: bernie west on April 20, 2015, 10:09:13 pm

you fail to consider that it is not my description - it is how processing was explained (more then once) by developers, that is exposure correction stage (pull/push/leave as is) in ACR/LR is after demosaick, wb/color transform (dcp has several parts, matrices /CM, FM/ and HueSatMap LUTs are applied before exposure correction)
 

Well it is your description here in this thread.  Do you understand it, or are you just parroting something without understanding it? 

If acr/lr is working in 16bit then when white balance is applied to a nominally over-exposed image a whole bunch of pixels will max out at 65535.  Pulling back the exposure slider will be unable to create detail out of that blown data.  It will only be able to render those areas as some shade of grey.  So either you are wrong, or acr/lr are working in a larger bit-depth than 16bit.  If that's the case, then you've got to explain how it is that both DCRAW and LR render nominally blown highlights the same way.  DCRAW I'm pretty sure only works in 16 bits (well, it did when I last played with this stuff in 2009 or so).  Can you explain all this, or are you uninterested in the details of your beliefs?











Title: Re: Precise Digital Exposure
Post by: BobD on April 21, 2015, 09:21:24 am
...I would not advise using LR/ACR... especially with the current process version (PV2012)...
...the BaselineExposure... increase or decrease the rendered values by the amount of the baseline adjustment.
...Most light meters (including those built into our cameras) are calibrated to yield 12% saturation rather than the 18%

I understand what you are saying about LR's adaptive technology and I try to debunk the 18% myth by referencing Thom Hogan’s article “Meters Don’t See 18% Gray” in my book.  However, what I learned from the Zone System was to understand AA’s procedure then empirically apply it to my personal “zone system”… my Exposure/Development; my +/-N develop times; my cold-light enlarger; etc. 

The keyword here is “system”… an empirically defined personal system. Not in a theoretical scientific realm… but real world photography… “your” real world photography! (Your camera, Your raw software, Your workflow.)

Photographers are supposed to take pictures not run tests and be scientists.

The goal in the analytical labs at Polaroid was for scientist to refine processes THEN develop procedures (systems) to be used by technicians. Many technicians might not understand the “why” of the procedures but were good observers and craftsmen. The real good craftsmen grew to become associate scientists.  Not unlike a “picture-taker” becoming a “photographer.

My method of exposure is intended to empirically test “our personal system” then get us photographers out shooting and not running endless test.
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 21, 2015, 10:02:56 am
Photographers are supposed to take pictures not run tests and be scientists.
Sorry but I don't see how the two are mutually exclusive.
When I was shooting film as a pro, I always ran tests. Film tests for each emulsion, working with my E6 lab for ideal CC filtration to further balance. Testing stobes, differing ways of printing. The statement is one useful to those who want to suggest photography is just about pointing a camera and capturing a moment. That's a large part of photography but a large part, perhaps not equally so IS running tests, and understanding the science behind the process. I don't know it's much different from a painter. Painters are supposed to paint and not mix let alone create their own pigments. I'm not a painter, but I think not.

In terms of taking pictures, a photographer I respect once told me: Great pictures are made, not taken.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 21, 2015, 10:21:23 am
Well it is your description here in this thread.  Do you understand it, or are you just parroting something without understanding it? 

If acr/lr is working in 16bit then when white balance is applied to a nominally over-exposed image a whole bunch of pixels will max out at 65535.  Pulling back the exposure slider will be unable to create detail out of that blown data.  It will only be able to render those areas as some shade of grey.  So either you are wrong, or acr/lr are working in a larger bit-depth than 16bit.  If that's the case, then you've got to explain how it is that both DCRAW and LR render nominally blown highlights the same way.  DCRAW I'm pretty sure only works in 16 bits (well, it did when I last played with this stuff in 2009 or so).  Can you explain all this, or are you uninterested in the details of your beliefs?


again - you fail to comprehend that those are not my "beliefs" - I am merely repeating you (as you can't find out yourself) what more than once was said by Adobe developers (E.Chan) about where the exposure happens in ACR/LR pipeline in terms of operations with the data  :)


if you are interested in the data types that Adobe are using internally during various operations then you probably need to see not dcraw.c code, but rather Adobe's DNG SDK code (available publicly) - that might give you an idea (because they use that code in ACR/LR).


if you are interested to find out how Adobe achieves the sequence of operations they disclosed (demosaick -> ... -> wb/color transform till/including HueSatMap tables -> ... -> exposure ->... ) w/o losing any necessary data please feel free go and dig there, but that's not something I am interested in here, sorry, so don't drag me into "bits" discussion... or you can chase E.Chan and challenge his description about the sequence of operations with the data in ACR/LR (that is, if I understand you correctly, that exposure operations /relevant UI sliders/ in ACR/LR code must be done before WB as you, apparently, want to say, not after, as he says)... here I want to include again one his many quotes = ""the DNG processing model performs a linearization of the original raw image values followed by demosaicing, then white balance. All of the other image stages follow. So to answer your question, all of the image ops except for linearization (which isn't under user control anyways) happens after demosaicing. "

Title: Re: Precise Digital Exposure
Post by: BobD on April 21, 2015, 10:26:18 am
Andrew,

"but a large part, perhaps not equally so IS running tests, and understanding the science behind the process."
My comment was never intended to mean that becoming a photographer meant you no longer have to practice and keep learning your craft!

"Great pictures are made, not taken."
As is the learning process throughout our life, we tried to simplify our lives.
I feel this is the same for us photographers… To simplify our photographs

When I do my photography talks, I try to emphasize that…
..."Our goal is to not take snapshots... but, to make snapshots.
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 21, 2015, 11:23:39 am
"but a large part, perhaps not equally so IS running tests, and understanding the science behind the process."
My comment was never intended to mean that becoming a photographer meant you no longer have to practice and keep learning your craft!
And IMHO that involves testing, some science and of course, 'taking' pictures.
Title: Re: Precise Digital Exposure
Post by: bernie west on April 21, 2015, 12:02:37 pm

again - you fail to comprehend that those are not my "beliefs" - I am merely repeating you (as you can't find out yourself) what more than once was said by Adobe developers (E.Chan) about where the exposure happens in ACR/LR pipeline in terms of operations with the data  :)


if you are interested in the data types that Adobe are using internally during various operations then you probably need to see not dcraw.c code, but rather Adobe's DNG SDK code (available publicly) - that might give you an idea (because they use that code in ACR/LR).


if you are interested to find out how Adobe achieves the sequence of operations they disclosed (demosaick -> ... -> wb/color transform till/including HueSatMap tables -> ... -> exposure ->... ) w/o losing any necessary data please feel free go and dig there, but that's not something I am interested in here, sorry, so don't drag me into "bits" discussion... or you can chase E.Chan and challenge his description about the sequence of operations with the data in ACR/LR (that is, if I understand you correctly, that exposure operations /relevant UI sliders/ in ACR/LR code must be done before WB as you, apparently, want to say, not after, as he says)... here I want to include again one his many quotes = ""the DNG processing model performs a linearization of the original raw image values followed by demosaicing, then white balance. All of the other image stages follow. So to answer your question, all of the image ops except for linearization (which isn't under user control anyways) happens after demosaicing. "



I'm not questioning what Chan says, all I'm saying is that there's a chance that you might not have understood what he was saying.  In the end, it doesn't really matter, as it is clear that LR can pull back nominally clipped channels in the rendered data, just like all raw converters can.  Whether it does this before the WB or after with a larger bit space, doesn't really make much difference. 
Title: Re: Precise Digital Exposure
Post by: BobD on April 21, 2015, 12:26:03 pm
Andrew,

We are on the same "wavelength" on this - trust me.
We all need to be a little more empirical by testing what we read - not just doing what we read. (see my examples below).
Bob

1. I plotted many bracketed exposures many time testing the top 2 stop occur in the top 10% of our exposure. (Yes, I read George Jardine's" article about revisiting the Zone System a while back explaining this... but, I'm an empirical kind of guy... I need to test it on "my" system)
(http://onezone.photos/wp-content/uploads/OWP_XLS.png)

2. When testing the dynamic range of my camera sensor using Sekonics DTS software and the L758, I determined that
> the dynamic range of my camera's sensor was different with a camera raw captured saved in AdobeRGB1988 space
> rather than shooting a camera "jpg" shot in AdobeRGB1988 space (as recommended).
(http://onezone.photos/wp-content/uploads/DTS_CRtoA1998.png)
Title: Re: Precise Digital Exposure
Post by: digitaldog on April 21, 2015, 12:31:36 pm
Andrew,
We are on the same "wavelength" on this - trust me.
I know we are. I'm just not as comfortable as you are with the statement: Photographers are supposed to take pictures not run tests and be scientists.
It runs counter to everything you've written prior to that text, that's all.
Title: Re: Precise Digital Exposure
Post by: AlterEgo on April 21, 2015, 01:28:52 pm
I'm not questioning what Chan says, all I'm saying is that there's a chance that you might not have understood what he was saying.

well, suggest an alternative explanation to a very clear statement (and not only one, but just one out of many) that WB follows right after demosaick and all other things later...  E.Chan is a technical person, not from marketing department, so I have no reason to question that direct statement... now, again, if you curious, how they manage to do this - DNG SDK code has C/C++ data types for you to study what they do as 16 bit signed/unsigned integers, what they do in 32/64 and float... or it seems that with ACR9/LR6 being released he is out here (I saw him replying today on Adobe forums) - so ask him
Title: Re: Precise Digital Exposure
Post by: Anders_HK on April 22, 2015, 07:16:15 am
1. I plotted many bracketed exposures many time testing the top 2 stop occur in the top 10% of our exposure. (Yes, I read George Jardine's" article about revisiting the Zone System a while back explaining this... but, I'm an empirical kind of guy... I need to test it on "my" system)
(http://onezone.photos/wp-content/uploads/OWP_XLS.png)

Bob,

I politely view your adoption of 99% per the above basis as incorrect, for the following three reasons. If I am missing something with your justification, please advise.

1) I do not see basis for why to choose 99%.

2) A picture is not mere based upon optimum white, but is important based on textures and tonal values, which with digital colour photography for the latter relates to R, G, and B.

3) Please refer to the attached to what Ansel referred to as Zone System Ranges. Contrary to there, today we typically call Dynamic Range in digital what he referred to as Full Black to Pure White. What he referred to as Dynamic Range is in fact Tonal Range. Additionally there is Texture Range. I will argue that what is correct to base the exposure upon towards the ends of your sensors exposure range, are actually the last texture or tonal values respectively before clipping of channels (before and after recovery settings). Firstly lost is the texture, later after are the tonal values in each channel R, G, B which does not necessarily occur at same time). If I recall correct, I determined my 2 1/3 upon upper point without recovery and without clipping any channel where I could still view texture, while 3 1/2 the last channel before clipping with max highlight recovery that I could still clearly view tonal value.



Anders
Title: Re: Precise Digital Exposure
Post by: ErikKaffehr on April 25, 2015, 01:01:39 pm
Hi,

Regarding shadows, a proper ETTR exposure is always optimal. It is simply the maximum exposure that avoid clipping non specular highlights.

I would suggest to check the raw files with a tool showing a proper histogram, without manipulation in the raw processor. What I have found on my P45+ and also on the Sony Alpha 99 I have is that blinkies and pretty accuarate indications of clipping.

Here is an example, taken from my latest outdoor shoot: http://echophoto.dnsalias.net/ekr/Articles/ETTR2/20100721-CF046421.iiq

This exposure is slightly overexposed, which is quite obvious in RawDigger:
(http://echophoto.dnsalias.net/ekr/Articles/ETTR2/SomeClipping.png)

Lightroom silently applies some highlight recovery:
(http://echophoto.dnsalias.net/ekr/Articles/ETTR2/LIghtroomView.png)

Capture One correctly indicates clipping:
(http://echophoto.dnsalias.net/ekr/Articles/ETTR2/C1_film_curve_view.png)

Reducing exposure in C1 fixes clipping (by highlight recovery):
(http://echophoto.dnsalias.net/ekr/Articles/ETTR2/C1_exposure_reduced.png)

Now, in this case I looked at the blinkies and histogram on my P45+, and decided that correct exposure was 0.5 stops down. So the final exposure had a raw histogram like this:
(http://echophoto.dnsalias.net/ekr/Articles/ETTR2/ProperETTR.png)

Now, I could have used my Minolta Spotmeter F to adjust highlight exposure to +3 EV and gotten the same results. Using the histogram, I need to make an additional exposure, of course. Also, the spot meter can give indications how dark the darks are.

More modern cameras often have "zebras" which give adjustable limits on exposure and can be used with live view.

So to sum it up:

ETTR is always right, but it needs to be decided what is OK to clip. Blinkies are helpful in that regard.

Histograms shown in the raw converter always show manipulated data. It is therefore better to analyse raw histograms like those produced by RawDigger.

This is a decent example of an image with relatively high dynamic range:

(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/CanoeStadium/20141109-_DSC6262_photographic1.jpg)

Note that the disc of the sun is clearly visible:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/CanoeStadium/20141109-_DSC6262_crop2_small.jpg)

But the near silhouette of the rowers can be made quite clean:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/CanoeStadium/20141109-_DSC6262_crop1.jpg)

The raw histogram of this image indicates around ten stops of luminance range:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/CanoeStadium/overview.jpg)



Here is a subject with larger luminance range, the Alpha 99 SLT reproduces highlights decently, but also gives good details in the darks:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/20140617-_DSC4758_small.jpg)
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/20140617-_DSC4758_piano.jpg)

In this case the P45+ had somewhat less exposure, based on camera histogram:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/20140617-CF045290_small.jpg)

And the P45+ straggled in the shadows:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/20140617-CF045290_piano.jpg)

With the P45+, a spotmetered exposure may have resulted in a better exposure. Around one stop difference between SLT 99 and P45+.

A HDR merge using Lumariver HDR fixed he shadows issue on the P45+:
(http://echophoto.dnsalias.net/ekr/Articles/DRArticle/Lockenhous/20140617_lumariver_piano.jpg)

Best regards
Erik

Lets get back on subject…

Where to locate the maximum of photons collected, and how ?? ?
Ensure shadows will be adequate collected ?

EXAMPLES:

How to expose the attached scenes precisely; Ziczac Bridge, View from Resturant, Bierstadt (painting by)?

Consider exposing for allowing in processing for an extended shoulder transition into highlights of around 1/3 stops more than available highlight recovery (Velvia slide film, page 19-20 in my paper), and for maintaining as much quality pixel info as possible in the "important" parts of image.

I can spot meter for (A) or (B) above, or if I want to expose per knowing more for any of remaining in above. After, all I need is one shot. To add, although my Leaf back has an excellent RAW histogram, due 80MP it is slower to use than a DSLR and more battery hungry.

Anders
Title: Anders's examples are very good
Post by: ErikKaffehr on April 26, 2015, 03:18:56 am
Hi,

Apart from the discussion about exposure determination, the images that Anders HK has posted are both really good and quite illustrative.

Best regards
Erik
Title: Re: Anders's examples are very good
Post by: bjanes on April 26, 2015, 10:02:18 am
Apart from the discussion about exposure determination, the images that Anders HK has posted are both really good and quite illustrative.

I agree that Anders' images are lovely, but I think that attempts to adapt zone principles to digital are misdirected since film and digital sensors have a very different response to light. Digital is linear whereas film responds proportionally to the logarithm of exposure. The linear part of the H&D is linear only when plotted on a log-log scale (log exposure vs log(1/transmission). Figure 10.6 in this link (https://books.google.com/books?id=jTAwGTYYiusC&pg=PA260&lpg=PA260&dq=film+logarithmic+response&source=bl&ots=grhypbwHwZ&sig=yE9MTCfUgmx48GuYWi2K5_9dCak&hl=en&sa=X&ei=MtE8VcXIEIvUoATv5oGoAw&ved=0CF0Q6AEwCQ#v=onepage&q=film%20logarithmic%20response&f=false) demonstrates the differences between film (in this case radiographic film) and digital.

The s-curve of the raw converter response does not correspond to the toe and shoulder of the H&D plot of film exposure.

Bill
Title: Re: Anders's examples are very good
Post by: digitaldog on April 26, 2015, 12:01:55 pm
I agree that Anders' images are lovely, but I think that attempts to adapt zone principles to digital are misdirected since film and digital sensors have a very different response to light. Digital is linear whereas film responds proportionally to the logarithm of exposure.
That's always been my understanding and belief too so it's good to hear confirmation on this.
Title: Re: Precise Digital Exposure
Post by: hjulenissen on April 27, 2015, 06:10:10 am
I just found the article that this image seems to stem from:
(http://cnet1.cbsistatic.com/hub/i/2012/04/18/2d7fb674-fdc3-11e2-8c7c-d4ae52e62bcc/751dadeda7198d4ab1d5f1f66dba0322/DxO-film-vs-digital-dynamic-range.png)
"An objective protocol for comparing the noise performance of silver halide film and digital sensor", Frédéric Cao, Frédéric Guichard, Hervé Hornung, Régis Tessière
https://www.dxo.com/sites/dump.dxo.com/files/dxoimages/ei/sci-publications/2012%20Film_vs_Digital_final_copyright.pdf
Title: Re: Precise Digital Exposure
Post by: bjanes on April 27, 2015, 11:12:19 am
I just found the article that this image seems to stem from:
(http://cnet1.cbsistatic.com/hub/i/2012/04/18/2d7fb674-fdc3-11e2-8c7c-d4ae52e62bcc/751dadeda7198d4ab1d5f1f66dba0322/DxO-film-vs-digital-dynamic-range.png)
"An objective protocol for comparing the noise performance of silver halide film and digital sensor", Frédéric Cao, Frédéric Guichard, Hervé Hornung, Régis Tessière
https://www.dxo.com/sites/dump.dxo.com/files/dxoimages/ei/sci-publications/2012%20Film_vs_Digital_final_copyright.pdf

Thanks for the link to a very good article by the scientists at DXO. The plot is for SNR, not the characteristic curve, but is nonetheless interesting.

I downloaded the graph for the characteristic curve of a hypothetical film and used Photoshop to measure the values on the axes and interpolate the Y-axis values for X-axis values and converted from density to opacity in Excel (opacity = 10^density) and plotted the values in Excel. The plot shows a logarithmic response as expected. Digital is linear as the article states.

Bill

Title: Re: Precise Digital Exposure
Post by: Anders_HK on May 16, 2015, 08:37:04 pm
... a proper ETTR exposure is always optimal. It is simply the maximum exposure that avoid clipping non specular highlights.

I would suggest to check the raw files with a tool showing a proper histogram, without manipulation in the raw processor.

Hi Erik,

Apologies for a belated reply. I agree that what you state is the theory. However, in practicing photography we can be faced with situations where similar as the example with the sun you posted, there is less haze (stronger light from sun), whereby we need decide to not ETTR on basis of the strongest light within the sun. This is because we else will not get sufficient data for mid tonal range of a picture, and because the collection of optimum high values within the dynamic range within sun is not of most important.

"a tool showing a proper histogram" - My Leaf back show me a RAW histogram, including extended ends for what in principle can be recovered. I also do rely on the histogram in Capture One (including at my choice for defaults. However, the important to my image is what results I can (have capability to) achieve in a processed image (no recovery and/or towards full recovery). That will be my picture. Therefore is why in my spot metering I consider both the RAW values at extremes, and at what spot metered values are beyond these to the maximum I can recover in Capture One.

Further, when studying test images of a grey card per viewing them in Capture One, at both extremes firstly lost will be the texture, second the tone, and third clipping. For my image, I wish to control on marginally safe side to where in image that texture and tone extends at defaults in Capture One (default RAW values) and at maximum recovery. That will help me define exposure for how I wish optimum basis RAW for my later processing towards a picture. This is how I precise can visualise for assuring of captured dynamic range versus that of a scene.

So why do I not simply rely on my Leaf histogram fully? For situations where I demand full control for precise allocation of capable dynamic range of my sensor (with and without recovery in post) per pin pointing spot metered values in a scene.

Best regards,
Anders