Luminous Landscape Forum

Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: walter.sk on November 25, 2012, 04:15:58 pm

Title: No way to balance color with R,G,B points?
Post by: walter.sk on November 25, 2012, 04:15:58 pm
Sorry if my Subject Line above doesn't describe what I'm trying to say, but here it is:

In Photoshop, one of my favorite techniques, even if seldom used, is to color-balance in Levels, using each color channel in conjunction with the Alt key to show me the darkest and lightest pixels in each channel.  So far, I have found no way to do that in LR.  I tried Curves, using the R, G and B curves but the Alt key has no effect.  Dragging the endpoints of the curve in each channel seems too crude to use without seeing the brightest or darkest pixels.

Dragging the ends of the histogram works with the Alt key, but of course you can't adjust individual channels there.

Is there any way to accomplish this in LR?  And if not, is there anything about LR structure that would prevent this technique from being written into the program?  And if not, maybe some kind of simulation of it?
Title: Re: No way to balance color with R,G,B points?
Post by: Tim Lookingbill on November 26, 2012, 12:04:58 am
Have you tried holding down the Alt key with the Black Point or Exposure slider?

It works for me in ACR with preview indicators similar to Levels.
Title: Re: No way to balance color with R,G,B points?
Post by: walter.sk on November 26, 2012, 03:10:52 pm
Have you tried holding down the Alt key with the Black Point or Exposure slider?

That works well, but not for color balancing, which works by adjusting the end-points of the individual R, G and B channels separately.  The closest I can come in LR or ACR is to go to Curves, select the individual color curves, and try by eye to move the endpoints of the curve to just meet where the data seems to start.  However, it is no as accurate as it would be if holding the Alt key would let me see the darkest and brightest pixels in each channel.
Title: Re: No way to balance color with R,G,B points?
Post by: Scott Hargis on November 27, 2012, 12:32:38 am
Walter, I'm right there with you. The ability to set a true black point in LR would be WONDERFUL.
You use Levels; I normally accomplish this in Photoshop using a Threshold adjustment layer to identify the black/white points, and then a Curves adjustment layer to eyedropper them - colors and contrast across all tones "snap" into place.
I have a lot of images that would never need to leave LR if I could get this done on a RAW file.
Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 27, 2012, 01:00:15 am
Walter, I'm right there with you. The ability to set a true black point in LR would be WONDERFUL.

No, actually, what you think you want you really don't want. Two entirely different tool sets and workflows.

It would do you guys well to forget and ignore what you do in Photoshop VS ACR/LR. Why? Because in Photoshop you have already committed to a gamma encoded color space. In ACR/LR you are dealing with raw images in a linear gamma and in the camera's native, undefined color space. That's a really big friggin' difference in tool sets.

You may wish you could use techniques you've found that may work in Photoshop, but you are spitting into the wind...it ain't gonna happen and your only option is to learn how to use ACR/LR...then do the stuff that can't be done there in Photoshop. The more you learn to do in ACR/LR, the better your images will be (and the less time you spend diddling images in Photoshop).

Seriously, break the bond between Photoshop tools and ACR/LR tools...it'll only hold you back.
Title: Re: No way to balance color with R,G,B points?
Post by: Bart_van_der_Wolf on November 27, 2012, 05:01:58 am
No, actually, what you think you want you really don't want. Two entirely different tool sets and workflows.

It would do you guys well to forget and ignore what you do in Photoshop VS ACR/LR. Why? Because in Photoshop you have already committed to a gamma encoded color space. In ACR/LR you are dealing with raw images in a linear gamma and in the camera's native, undefined color space. That's a really big friggin' difference in tool sets.

You may wish you could use techniques you've found that may work in Photoshop, but you are spitting into the wind...it ain't gonna happen and your only option is to learn how to use ACR/LR...then do the stuff that can't be done there in Photoshop. The more you learn to do in ACR/LR, the better your images will be (and the less time you spend diddling images in Photoshop).

Seriously, break the bond between Photoshop tools and ACR/LR tools...it'll only hold you back.

Hi Jeff,

While I agree that one should be careful and not try to mimick Photoshop controls in Lightroom, the comparison between ACR and LR functionality where both are using e.g. Process 2012 is tempting. Sure, LR offers some controls that are not found in ACR, but that's because ACR can in principle relay those activities (and more) to Photoshop itself. However, the ACR Tonecurve panel has a Point tab next to the Parametric tab, and that Point tab does allow to do per channel Black point adjustments.

Obviously, that ACR Point tab functionality will only kick in after the actual Raw conversion because it needs RGB rendered image data to work on, but that doesn't mean that LR could not do something similar, like in its very useful Softproofing mode which works with RGB data, when creating the exported output.

Cheers,
Bart
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 27, 2012, 10:08:15 am
In ACR/LR you are dealing with raw images in a linear gamma and in the camera's native, undefined color space.
no, you are not... when you work with UI (except probably changing things like "process" and "camera profile") you are not dealing with raw data in ACR/LR, you are dealing with something that was demosaicked and color transformed into a defined color space just w/ linear gamma.
Title: Re: No way to balance color with R,G,B points?
Post by: Scott Hargis on November 27, 2012, 10:26:59 am
Jeff, thanks for chiming in. I'm reading The Digital Negative now, with Real-World Sharpening waiting just behind it.

I would say that what I'm really after is the RESULT, more than the technique for getting there. But I don't know how to describe it except as it relates to the Photoshop Curves method I described.

Is there a recommended way to identify a black point and then set the R,B,G curves to it? Nothing I know how to do in LR gets me the same result.
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 27, 2012, 10:35:19 am
no, you are not... when you work with UI (except probably changing things like "process" and "camera profile") you are not dealing with raw data in ACR/LR, you are dealing with something that was demosaicked and color transformed into a defined color space just w/ linear gamma.

No, Jeff's correct. You are presented with a "gamma" corrected preview (actually not a gamma curve but the sRGB TRC), the data is always processed as Jeff describes.
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 27, 2012, 11:09:23 am
No, Jeff's correct. You are presented with a "gamma" corrected preview (actually not a gamma curve but the sRGB TRC), the data is always processed as Jeff describes.


ACR/LR do not operate on raw data (be it RGBG or CMYG or Foveon or whatever) data UI wise... all sliders that you are moving in UI are instructions of what shall be done with the data that was already 1) demosaicked and 2) transformed to a color space with prophoto coordinates and linear gamma... why do you want to argue w/ that ?

the only time when ACR/LR does redemosaicking and color transform of the data from "camera's native, undefined color space." to a defined, internal working spaces (where the raw conversion actually ends) is when you change the process and/or dcp camera profile... that is the core of what Adobe calls "dng processing model" that makes its UI close to real time speedwise, exactly because you do not operate with raw data.

PS: I hope Jeff explains this in his book and not misrepresents the situation.
Title: Re: No way to balance color with R,G,B points?
Post by: RFPhotography on November 27, 2012, 11:10:04 am
Hi Jeff,

While I agree that one should be careful and not try to mimick Photoshop controls in Lightroom, the comparison between ACR and LR functionality where both are using e.g. Process 2012 is tempting. Sure, LR offers some controls that are not found in ACR, but that's because ACR can in principle relay those activities (and more) to Photoshop itself. However, the ACR Tonecurve panel has a Point tab next to the Parametric tab, and that Point tab does allow to do per channel Black point adjustments.

Obviously, that ACR Point tab functionality will only kick in after the actual Raw conversion because it needs RGB rendered image data to work on, but that doesn't mean that LR could not do something similar, like in its very useful Softproofing mode which works with RGB data, when creating the exported output.

Cheers,
Bart

You have this in LR 4.x with PV2012 as well though, right?  In PV2012, the Tone Curve panel allows you to, when using the Point Curve, select the composite RGB or any of the individual channels.
Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 27, 2012, 11:48:41 am
ACR/LR do not operate on raw data (be it RGBG or CMYG or Foveon or whatever) data UI wise... all sliders that you are moving in UI are instructions of what shall be done with the data that was already 1) demosaicked and 2) transformed to a color space with prophoto coordinates and linear gamma... why do you want to argue w/ that ?

For the purposes of displaying an image preview, yes...ACR/LR is showing you a "preview" of what the raw data "will" look like for the purposes of adjustment, but nothing has been done to the raw data processing pipeline until you actually process the image. Then ACR/LR (export or open in PS) will demosiac the raw data, transform from camera color, run the image through the raw processing pipeline, then do the final gamma encoded color space transform. So, of course, ACR/LR operate on the raw data...all of the adjustments are done to a preview of the raw data...but sure what your argument is?
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 27, 2012, 12:44:05 pm

ACR/LR do not operate on raw data (be it RGBG or CMYG or Foveon or whatever) data UI wise... all sliders that you are moving in UI are instructions of what shall be done with the data that was already 1) demosaicked and 2) transformed to a color space with prophoto coordinates and linear gamma... why do you want to argue w/ that ?

It's not working on the raw data? What's it working on?

The preview is just that, a preview. The raw is just that, raw. You're simply creating instructions along with the preview and at some point raw data to produce RGB data from an engine that works with linear encoded data (even if you feed it a rendered gamma corrected image). That doesn’t happen until you export the data or send it though a module (make me a web gallery).

Quote
the only time when ACR/LR does redemosaicking and color transform of the data from "camera's native, undefined color space." to a defined, internal working spaces (where the raw conversion actually ends) is when you change the process and/or dcp camera profile...

So what's happening when I move a slider? What happens when I export a web gallery or print the image? It sure isn't using the preview (unless I ask to print in Draft mode).
Title: Re: No way to balance color with R,G,B points?
Post by: Bart_van_der_Wolf on November 27, 2012, 01:34:09 pm
You have this in LR 4.x with PV2012 as well though, right?  In PV2012, the Tone Curve panel allows you to, when using the Point Curve, select the composite RGB or any of the individual channels.

Hi Bob,

You are correct, my bad. I overlooked the small icon at the bottom of the Tonecurve palette which after clicking it hides the region sliders and displays a channel selector when you click on the RGB acronym. Then the individual Black points can be adjusted per channel, just like the OP wants (except for the Alt/Option preview). So then the question becomes; Why no preview?

Cheers,
Bart
Title: Re: No way to balance color with R,G,B points?
Post by: RFPhotography on November 27, 2012, 02:07:18 pm
If you turn on the clip warnings on the histogram, does that do it?
Title: Re: No way to balance color with R,G,B points?
Post by: walter.sk on November 27, 2012, 02:27:46 pm
I overlooked the small icon at the bottom of the Tonecurve palette which after clicking it hides the region sliders and displays a channel selector when you click on the RGB acronym. Then the individual Black points can be adjusted per channel, just like the OP wants (except for the Alt/Option preview). So then the question becomes; Why no preview?
You understand my question.  While I get lost in the heady level of discussion at the technical level, my naive thinking says to me that whether or not the per channel adjustments are actual or just simulated in a preview, it does seem to me that the Alt/Option preview should not be impossible in LR, and I would find it helpful.  I am getting better at adjusting each channel's black point and white point by dragging the end point of each channel's curve, which I find *much* more accurate and efficient than clicking around with the eyedropper looking for something that is supposed to be neutral in the image, in the White Balance part of the basic panel. (I know I should shoot a gray card in the field, but that doesn't always happen.)
Title: Re: No way to balance color with R,G,B points?
Post by: Bart_van_der_Wolf on November 27, 2012, 02:47:48 pm
If you turn on the clip warnings on the histogram, does that do it?

Yes, in a way, but only for the clipped levels (no info on the proximity to the clipping point, so one needs to slide back and forth for an impression). Maybe it's close enough for Walter's use.

Cheers,
Bart
Title: Re: No way to balance color with R,G,B points?
Post by: RFPhotography on November 27, 2012, 04:04:30 pm
I don't find that the Alt method with Levels in PS works much differently.  It overlays a mask and shows clipped pixels.  Unless I'm missing something.  The thing I'm not sure of in LR is whether the clipping warnings pertain to each channel when in that mode of the Tone Curve module or are still overall RGB clipping.
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 27, 2012, 05:17:14 pm
It's not working on the raw data? What's it working on?

ACR/LR are 2 in 1

1) a raw converter hidden from you (almost, again - you can select the process version and dcp profile) that reads the data, does whatever normalization you need, does demosaick, does color transform and the pass the result to next component with which you actually interact, thinking that you work with raw data

and

2) a version of PS designed to work with images originating from that hidden raw converter - that is what exposed to you through UI, where you do "WB", "exposure compensation" and so on... but you do not do this with the raw data... and hence you can actually do this in a proper raster image editor, the mere fact that PS is lacking something that is present in ACR/LR does not change that fact.


The preview is just that, a preview. The raw is just that, raw. You're simply creating instructions along with the preview and at some point raw data to produce RGB data from an engine that works with linear encoded data (even if you feed it a rendered gamma corrected image). That doesn’t happen until you export the data or send it though a module (make me a web gallery).

the point is - all of your sliders in ACR/LR UI do not work operate on raw data... the mere fact that working color space is linear encoded does not make it raw data - you can as well get that to PS and work there (yes, not convenient)

Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 27, 2012, 05:23:18 pm
So, of course, ACR/LR operate on the raw data

operations that you control through UI (except process version and camera profile selected) in ACR/LR do not operate on raw data

...all of the adjustments are done to a preview of the raw data

no - all adjustments are done on demosaicked and color transformed data, you can't call that raw data... if I do something before demosaick then I can say that I operate with raw data, but none of ACR/LR UI sliders or curves do that... not even white balance.

Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 27, 2012, 05:32:35 pm
operations that you control through UI (except process version and camera profile selected) in ACR/LR do not operate on raw data

They operate on previewed data derived from the raw data and in the case of the raw processing pipeline, you can't really separate one part of the pipeline from the rest...the entire processing pipeline is being applied from the start of the pipeline to the end of the pipeline. If you want to hold to the fact that ACR/LR must first demosaic the raw data and as a result, any processing is done on the demosaiced data (which you claim makes it no longer "raw") I think that's a distinction without relevance...ACR/LR process raw files into gamma encoded RGB files.
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 27, 2012, 06:11:40 pm
1) a raw converter hidden from you (almost, again - you can select the process version and dcp profile) that reads the data, does whatever normalization you need, does demosaick, does color transform and the pass the result to next component with which you actually interact, thinking that you work with raw data

I'm sorry, I don't understand that.

Quote
2) a version of PS designed to work with images originating from that hidden raw converter - that is what exposed to you through UI, where you do "WB", "exposure compensation" and so on... but you do not do this with the raw data

If not the raw data, what? Not the current rendered preview. It's updated on the fly as I move sliders and such. I now have to do something with the preview but I need something a lot bigger. If I'm not exporting, printing, building web galleries from the raw data, then what data?

Quote
the point is - all of your sliders in ACR/LR UI do not work operate on raw data...

OK what stuff is raw and what stuff is, whatever else there is when I make a 16x20 print?
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 27, 2012, 07:39:19 pm
I'm sorry, I don't understand that.

you can use dcraw to output .tiff with demosaicked data with color space = prophoto and linear gamma and no WB applied (you can set your own WB multipliers there)... is that a raw data ? now what is the difference between the raw file opened in ACR and that .tiff opened in ACR for you working with ACR's UI ? different demosaick ? that does not change anything... may be different color transform from camera RGB to prophoto coordinates - that does not change anything either in principle... so where is the difference for you playing w/ the sliders in ACR ? no difference... so to say that you are working with raw data in ACR through UI is not true... ACR has a raw conversion component, yes... but sliders and curves are not working with raw data... no matter how you will change a WB or exposure in ACR it will not redo demosaick and it will not apply WB multipliers to raw data channels - but will be working with demosaicked data in a regular color space instead, linear gamma or not - does not make that data a raw data... you can call it scene-referred as much as you want - but it is not a raw data... and you might as well get some image editor and work with the above mentioned tiff from dcraw...  and do you white balancing (ACR style), tone mapping and further color transforms there and you will not call that raw conversion, right ?
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 27, 2012, 08:07:50 pm
you can use dcraw to output .tiff with demosaicked data with color space = prophoto and linear gamma and no WB applied (you can set your own WB multipliers there)... is that a raw data ?

No it isn't. It's rendered (in this case) to a TIFF at some size at some resolution.

Quote
now what is the difference between the raw file opened in ACR and that .tiff opened in ACR for you working with ACR's UI ?

I don't know what you mean by open in ACR. If you 'open' a raw file, ACR provides a preview it generates of some rendering instructions just like Lightroom. When you say open if you mean the command telling ACR to render the data, it's the same as the example you asked about above. It's a rendered TIFF. The difference is that the preview is one thing, a proxy. The rendering instructions are another. If I ask ACR to open the raw data within Photoshop it has to render the data with raw+instructions, from the raw data.

Quote
but sliders and curves are not working with raw data... no matter how you will change a WB or exposure in ACR it will not redo demosaick and it will not apply WB multipliers to raw data channels -

OK then please explain then the processing path. I import a raw into LR. I've got a preview right? It was generated by LR agreed? At some point I have a pile of sliders set such I like the way the preview looks. Now I want a 16-bit RGB full resolution from this 5DMII. You are saying none of the raw is touched at this point? LR renders the data from what? Can you explain the processing path once I've told LR or ACR via instructions what color appearance like based on the preview.
Title: Re: No way to balance color with R,G,B points?
Post by: RFPhotography on November 28, 2012, 08:35:44 am
I think what Vladimirovich is saying is that once the 'raw data' is previewed on screen in a demosaiced form with a colour model applied to it it's no longer raw data and that any changes you make via the application UI aren't working on the actual undemosaiced, non-colour transformed (i.e., no ProPhoto working space with linear gamma, no calibration profile).  He's saying, I believe, that any changes you make don't require or don't force a rerendering of the undemosaiced, non-colour transformed raw data or provide a new preview but that the changes made via the UI sliders are made on top of the already demosaiced, colour transformed preview and that as a result of not generating a new screen preview from the undemosaiced, non-colour transformed raw data it's no longer a raw image that's being worked on.  I think what he's saying is that those first couple steps - demosaic and colour transform - are like hidden steps in the LR history.  I think what he's saying is that at the point of the image preview on screen in demosaiced, colour transformed form that it's no different from working on a fully rendered tiff in PS.
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 28, 2012, 09:42:42 am
I think what Vladimirovich is saying is that once the 'raw data' is previewed on screen in a demosaiced form with a colour model applied to it it's no longer raw data and that any changes you make via the application UI aren't working on the actual undemosaiced, non-colour transformed (i.e., no ProPhoto working space with linear gamma, no calibration profile). 

To which I'd answer: Duh (or, most here area totally aware of that).

I tried to separate the preview (which is clearly not raw but a low rez rendered proxy) from the processing of the data and asked Vladimirovich to more clearly define the processing path since some of what Vladimirovich has written I don't quite understand.

When I ask LR to provide an sRGB 900x1800 image OR I ask for the full rez in ProPhoto 16-bit (or any combo you want to make up), the raw data has to be accessed I'd believe, to produce this new data unless Vladimirovich can explain how the processing path works without the raw data plus instructions.
Title: Re: No way to balance color with R,G,B points?
Post by: Tim Lookingbill on November 28, 2012, 01:06:47 pm
Vladimirovich might as well be saying we don't really see reality. Our brains decipher photon particle wave patterns across 400nm to 700nm light spectrum onto our rods and cones in the back of our eyeballs which excite our optic nerve to form a picture that's still not considered technically as reality.

Reality doesn't exist unless there's an observer to define it. So...

You can't edit what you can't see. Can't see it? It doesn't exist as we know it.

We can't edit RGB filtered mosaic patterns into a picture our brains associate with reality. All Raw data comprises (off the electronic sensor source) is luminance grayscale pixels from black to white captured from light projected through the lens and onto the sensor filtered by Bayer (RGGB) patterns into voltage signals that represent grayscale values which are then converted by the A/D electronics into 1's and 0's that will later be interpreted back into a visual pattern that is recognized by humans called a preview by software.

Further metaphysics discussions will begin momentarily if so desired.
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 28, 2012, 01:30:22 pm
Further metaphysics discussions will begin momentarily if so desired.

Not till I can get out of the Matrix!
Title: Re: No way to balance color with R,G,B points?
Post by: Scott Hargis on November 29, 2012, 12:09:24 am
So......I think what you guys are saying is, there's no way to balance color with R,G,B points.


Right?
Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 29, 2012, 01:14:20 am
So......I think what you guys are saying is, there's no way to balance color with R,G,B points.
Right?

Wrong...there's simply no way to do it the way Photoshop does...LR/ACR is a different animal. Color balancing with RGB curves in ACR/LR is useful but there are no Option/Alt clicking ways of making an auto adjustment. Do it by hand/eye.
Title: Re: No way to balance color with R,G,B points?
Post by: bjanes on November 29, 2012, 08:56:21 am
It would do you guys well to forget and ignore what you do in Photoshop VS ACR/LR. Why? Because in Photoshop you have already committed to a gamma encoded color space. In ACR/LR you are dealing with raw images in a linear gamma and in the camera's native, undefined color space. That's a really big friggin' difference in tool sets.

You may wish you could use techniques you've found that may work in Photoshop, but you are spitting into the wind...it ain't gonna happen and your only option is to learn how to use ACR/LR...then do the stuff that can't be done there in Photoshop. The more you learn to do in ACR/LR, the better your images will be (and the less time you spend diddling images in Photoshop).

I'm not sure if the OP is white balancing or merely adjusting colors to some artistic intent. However, if the former, it is not a good idea to white balance on gamma encoded data. Lightroom and ACR can white balance gamma encoded files, but they do so in a linear space. With a curve adjustment of gamma encoded data, the endpoints (0 and 255 in 8 bit) will be fine, but the midtones will be distorted when one performs a linear adjustment on the nonlinear data in Photoshop. One could convert the PS data to linear using a linear profile space, perform the adjustment, and then convert back to the gamma encoded space. Alternatively, one could merely use preset curves to perform the conversions.

These curves are shown below for sRGB and inverse sRGB. Points for 0 and 255 are the same for linear and sRGB, but the midpoints are quite different.
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 09:30:51 am
No it isn't. It's rendered (in this case) to a TIFF at some size at some resolution.
the same is with ACR/LR - in its processing workflow/path (codewise), raw data is demosaicked as soon as possible and color transformed into a regular colorspace (colorimetric - isn't that the word ?) just with linear gamma... that happens before you see any rendered (from raw data by ACR) image on screen... and then, from that point on, it is not different from working with .TIFF opened in ACR/LR (or linear DNG for that matter), provided that .TIFF will generated the same way (w/ demosaicked data, prophoto coordinates, linear gamma)... data obtained at this stage is the data that will be used by ACR/LR code to generate previews, do all adjustments that you control through UI (including WB and exposure)... so to claim that you adjust raw data is quite incorrect here... you adjust the data that was already converted (by ACR/LR) and not raw data... that is unlike proper "raw converter" like dcraw (WB before demosaicking for example, so WB on raw data, per raw channel)... the only operation available to you in ACR/LR UI that actually works with the raw data is changing "process" version (= new demosaick) and camera profile change (= new color transform, even that does not require new demosaick)... so the codewise there is a point in ACR/LR code after which everything is the same for raw file, for linear DNG file converted from raw, for .TIFF... and that point in code, like it or not, lies before any preview is generated and before UI controlled code that does changes (except process/camera profile controls)...
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 10:08:41 am
the same is with ACR/LR - in its processing workflow/path (codewise), raw data is demosaicked as soon as possible and color transformed into a regular colorspace (colorimetric - isn't that the word ?) just with linear gamma...

Hold on, you are still not clear enough for me to understand. Are you saying that as soon as I import the image into LR (or open in ACR), the full resolution raw data is demosaiced? And from that point on, if I want a 1mb or 30mb file, the ACR engine takes this processed (or particularly processed data) of full resolution data and then provides the size/color space/bit depth I've asked for?

Quote
that happens before you see any rendered (from raw data by ACR) image on screen... and then, from that point on, it is not different from working with .TIFF opened in ACR/LR

Again, I'm having difficulty with perhaps our native languages. When you say open, do you mean the preview seen as soon as I select "Open" on a raw file for ACR OR open when I expect ACR to provide the data size, color space, bit depth etc shown to me IN Photoshop?

Perhaps if you outlined how you think the processing path operates I'd understand better:

1. Select Open in ACR
2. ACR access raw data and builds (you have to fill in the rest)
3. Parametric edits from user are applied
4. User exports 1mb image in sRGB for web (you have to fill in the rest in terms of what happens next).

Here's what I *think* happens and I'm wondering if you feel this is true or not:

I 'open' a raw file in ACR or import into LR.
A small preview is generated (in LR we have preferences to select the size).
I edit the image by building a list of parametric instructions. I suspect there is no reason this has to be applied to anything but this small preview but maybe you are saying the engine has processed data using the full resolution raw data.
The raw data file is accessed and with the edits I've built, an image is now rendered as I desire. Size, bit depth, color space etc.

Where above am I off?

If this sounds correct, then the where we are not communicating clearly is the very last sentence above, where it seems that the raw data plus the instructions have to both be accessed so rendering can take place. If that's the case, then the raw has to be used here no? Or are you saying the raw data was processed the very first time I accessed the data (or build a preview), that data lives somewhere waiting to be rendered? It seems that would make a huge overhead in processing and storage just to view a raw you might not edit.
Title: Re: No way to balance color with R,G,B points?
Post by: RFPhotography on November 29, 2012, 10:22:14 am
the same is with ACR/LR - in its processing workflow/path (codewise), raw data is demosaicked as soon as possible and color transformed into a regular colorspace (colorimetric - isn't that the word ?) just with linear gamma... that happens before you see any rendered (from raw data by ACR) image on screen... and then, from that point on, it is not different from working with .TIFF opened in ACR/LR (or linear DNG for that matter), provided that .TIFF will generated the same way (w/ demosaicked data, prophoto coordinates, linear gamma)... data obtained at this stage is the data that will be used by ACR/LR code to generate previews, do all adjustments that you control through UI (including WB and exposure)... so to claim that you adjust raw data is quite incorrect here... you adjust the data that was already converted (by ACR/LR) and not raw data... that is unlike proper "raw converter" like dcraw (WB before demosaicking for example, so WB on raw data, per raw channel)... the only operation available to you in ACR/LR UI that actually works with the raw data is changing "process" version (= new demosaick) and camera profile change (= new color transform, even that does not require new demosaick)... so the codewise there is a point in ACR/LR code after which everything is the same for raw file, for linear DNG file converted from raw, for .TIFF... and that point in code, like it or not, lies before any preview is generated and before UI controlled code that does changes (except process/camera profile controls)...

I think this is not correct and where you're going off the rails a bit.  A tiff or jpeg file, even when opened in ACR/LR has the pixels fully cooked.  White balance is established and while it can be altered the effect is not the same as working on a raw file.  It's gone from being simply a metadata entry to actually fixing pixel values.  Similarly any changes to colour or contrast, while able to be made, aren't the same as working on a raw file.

While a raw file opened in ACR/LR is still a raw file in that the pixels are not locked.  At this point White Balance is still just a metadata entry.  You're much more free to adjust exposure/white point/black point to recover highlights or open up shadows.  You're not working on fully cooked pixels as you are with a tiff or jpeg.  While the image is demosaiced and has a render curve applied via the Adobe Standard, or whatever dcp is used, dcp and has a gamma adjustment for on screen preview, those settings aren't locked in.  That render curve can be changed by using a different dcp.  That is necessary to get an image on screen that can be worked with and that looks, visually, similar to reality.  But those settings don't lock pixels at that point.  Any adjustments you make in LR/ACR are parametric and don't affect the actual pixels in the file until the image is exported in a fixed file format.  
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:27:15 am
Hold on, you are still not clear enough for me to understand. Are you saying that as soon as I import the image into LR (or open in ACR), the full resolution raw data is demosaiced?

YES, that is a core feature of Adobe's so called "DNG processing model", to get rid of the raw data as soon as possible and work with the data that is demosaicked and color transformed to a proper color space... the mere fact that gamma is 1 does not change much, you can have tiff w/ the data that is linear...
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:31:44 am
I think this is not correct and where you're going off the rails a bit.  A tiff or jpeg file, even when opened in ACR/LR has the pixels fully cooked.

first of all - I did not talk about JPG, I was talking about TIFF (those are just containers, but let us leave JPG out to make things simple)... define fully cooked ? I can get an image data output from a raw converter to tiff that will be exactly the same processing wise (demosaicked and no WB... no WB means WB = UniWB by the way... so absence of raw channel data multiplication is just UniWB WB applied formally) as in linear DNG produced by ACR/LR... do you call linear DNG data fully cooked ?
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:34:01 am
White balance is established

you can have WB = UniWB... which is the same as no WB established... by the way raw data also have WB established always (and no - I am not talking about tags - I am talking about data recorded per sensel)... it is just that WB is UniWB (except some exotic cameras where firmware actually does multiplication of the raw data per channel)  ;)
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 10:34:11 am
YES, that is a core feature of Adobe's so called "DNG processing model", to get rid of the raw data as soon as possible and work with the data that is demosaicked and color transformed to a proper color space... the mere fact that gamma is 1 does not change much, you can have tiff w/ the data that is linear...

I'm very clear understanding that the gamma encoding has nothing to do with whether the data is raw or rendered. But I find it hard to believe that instead of accessing the raw + instructions at rendering time, the Adobe engine builds a full rez, demosaiced document whether I edit the image or not. I do however remember the role of the ACR cache or more recently the option for DNG quick preview data. But my assumption is this isn't the full resolution demosaiced but only partially processed preview.

Why would LR for example, build all this data before I even move into Develop or throw away some of the imported images I don't want?
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 10:36:49 am
do you call linear DNG data fully cooked ?

That's partially cooked data. And I don't really get behind the idea of a linear DNG or for that matter, a TIFF that's been saved out as a DNG.

IOW, a DNG doesn't have to be about raw data.
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:40:03 am
But I find it hard to believe that instead of accessing the raw + instructions at rendering time, the Adobe engine builds a full rez, demosaiced document whether I edit the image or not.

That is why I find it very strange that Schewe did not describe that in his book - he knows that 100%... it is very clear that for performance reasons Adobe can't redemosaick always to have a real time UI feedback... so Adobe demosaick raw data when the file is opened and then just works with that... and the further processing path is the same as for linear DNG files... that is why the work with original raw and linear DNG converted from that raw by Adobe's tools is identical in ACR/LR !!! it is so clear... that is why in dcraw you have WB applied to raw data and in Adobe's tools not to raw data... because users are moving temp/tins sliders and they want real time feedback... and you can't have that with WB applied to raw data before demosaicking.. unless you do some tricks (like binning 4->1 instead of demosaicking, showing only part of the image demosaicked in a 100% loupe, etc)... Schewe likes to call this Knoll's genius - but that was just performance considerations, wasn't it ?
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:42:33 am
That's partially cooked data.

yes, and that is what ACR/LR do internally - they partically cook the data and then all previews, all your UI work is with that partially cooked data... so if you will feed that partically cooked data to ACR/LR there will be no difference for you in preview or UI operations... that is why linear DNG exists... and you can get TIFF prepared the same way...
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 10:43:54 am
That is why I find it very strange that Schewe did not describe that in his book - he knows that 100%...

Not only haven't I seen this in any book (I own a few on these products), I don't recall hearing this from Adobe. I guess it's time for Jeff and/or Eric to reply...
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 10:49:36 am
yes, and that is what ACR/LR do internally - they partically cook the data and then all previews, all your UI work is with that partially cooked data...

That part makes sense. Assuming this is done to a lower resolution proxy. The differences here between a preview and all the data being processed, wherever we edit or not is what I'd like to know more about.

Quote
so if you will feed that partically cooked data to ACR/LR there will be no difference for you in preview or UI operations... that is why linear DNG exists... and you can get TIFF prepared the same way...

I'm not sure bringing DNG, linear or not into the discussion makes the path clearer. Are you saying that if someone imports a proprietary raw into LR, even if they do not convert to DNG, behind the scene there's some conversion going on?

My understanding of linear DNG was for someone who wants partially processed data. Explained like this:

Quote
DNG is both a raw image format and a format that supports "non-raw", or partly processed, images.[2] The latter (non-raw) format is known as "Linear DNG".[31] Linear DNG is still scene-referred and can still benefit from many of the operations typically performed by a raw converter, such as white balance, the application of a camera color profile, HDR compositing, etc.
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:55:47 am
I guess it's time for Jeff and/or Eric to reply...

I guess Eric said that many times, may be not that directly... here is a piece where you can see that happens in DNG processing model (shown in DNG SDK) -

Eric Chan : ( http://www.luminous-landscape.com/forum/index.php?topic=38898.msg321693#msg321693 ) @ November 01, 2009

Quote
   The best place to start in terms of the code would be dng_validate.cpp. The dng_validate () routine in there will read in a negative, parse through it, and build the various image stages. This includes decompressing the raw image data, linearizing it if necessary, and render to a TIFF file if you want. From a developer's perspective, I'd suggest putting a breakpoint somewhere in that routine, feeding a DNG raw file to it, and then stepping through to see how the code flows. In terms of terminology, the code comments use the term "stage 1" image to describe the raw image data, "stage 2" to describe the linearized raw image data (in a canonical space [0, 1], or [0, 65535] if you prefer), and "stage 3" to describe the 3-channel or 4-channel data (i.e., after demosaic). If you want to do processing on linearized mosaic data, you want to grab the stage 2 image; see the Stage2Image () routine in the dng_negative class (dng_negative.h). If you want to do processing on RGB demosaiced data in the native camera color space, prior to white balance, you want the stage 3 image; see the Stage3Image () routine in the dng_negative class.

see - RGB demosaicked prior to WB... WB is not applied to raw data...



Eric Chan : ( http://www.luminous-landscape.com/forum/index.php?topic=56863.msg461536#msg461536 ) @ August 17, 2011

Quote
    HueSatMap1 and HueSatMap2 tables precede the Exposure & Blacks controls.

see again - exposure is not applied to raw (per channel) data...

reading postings of Eric Chan is the best book you can get about LR/ACR
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 10:57:09 am
That part makes sense. Assuming this is done to a lower resolution proxy.

no, you can view what you do at 100% in ACR/LR and you can have a screen that will show you all pixels... what proxy you are talking about ?
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 11:01:48 am
no, you can view what you do at 100% in ACR/LR and you can have a screen that will show you all pixels... what proxy you are talking about ?

The proxy would be that 1:1 preview. IOW, build the 1:1 for the screen view (maybe a bit more), update as the user moves around. An one-the-fly rendering of only what you need to see rather than rendering the entire image. Kind of like Live Picture from the last century. It too would process (render) instructions and data when you asked it to. JIT (Just In Time) processing.

Doesn't it seem odd to render a full 5DMII image, let alone those big medium format captures just to show me a 1024x768 or similar 1:1 preview?

As to Eric's post, well it's over my head. Where in there does it say the entire image is processed at this stage of the editing? And when I ask that question, I'm asking you to decipher what he wrote as I don't understand it.
Title: Re: No way to balance color with R,G,B points?
Post by: RFPhotography on November 29, 2012, 11:35:48 am
you can have WB = UniWB... which is the same as no WB established... by the way raw data also have WB established always (and no - I am not talking about tags - I am talking about data recorded per sensel)... it is just that WB is UniWB (except some exotic cameras where firmware actually does multiplication of the raw data per channel)  ;)

What you're suggesting is contrary to anything I've read and heard about white balance and raw files. 
Title: Re: No way to balance color with R,G,B points?
Post by: walter.sk on November 29, 2012, 12:26:53 pm
I'm not sure if the OP is white balancing or merely adjusting colors to some artistic intent.
It could be either.  Often, when I want to adjust colors to some artistic intent it helps me to see what a neutral starting point would be for comparison.

I also don't think I am looking for an automatic adjustment by using the Alt key in conjunction with the color levels in Photoshop.  What I do is find the first meaningful black and white points in each of the channels ignoring the "noise" to the left or right of where the "real" data starts in that channel.  I can eyeball the white or black overlay on the screen to determine where I should put those black or white points in each channel, and this usually does a creditable job of creating neutral grays/blacks/whites.  Whether that produces what I want artistically is a decision I make at that point.

My original question is based on the fact that many of the adjustments in LR utilize the Alt Key and a temporary change to the displayed image to a white or black screen with threshold-like representations of the brightest or darkest areas of the image appearing as I drag the slider.  That, plus the fact that the Point Curves panel allows individual adjustment of the R, G and B curves, including dragging the endpoints of those curves to meet the parts of the histograms containing real picture data, cause me to think that it should be possible to recreate in LR the process I use in Photoshop.

So, at this point my question is to find which of the following is (are) true:

1) This is not possible in LR.
2) This is possible in LR but just not previously considered.
3) This is possible in LR but would require too much planning, code changes or code writing to make it feasible.
4) Nobody knows.
Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 29, 2012, 04:00:34 pm
That is why I find it very strange that Schewe did not describe that in his book - he knows that 100%...

What I know and what I can say publicly are not always the same. While Eric is free to disclose whatever he wants to (and Eric is very good at walking that line), what I know about the Camera Raw processing pipeline can only be discussed in general terms by me.

Honestly, this whole question is a rabbit hole. Exactly what ACR/LR is doing and in what order isn't really important for the user since the user really has no control over the pipeline. To be accurate, the "first" thing ACR does when opening a proprietary raw files is to actually convert the file to DNG on the fly, load the entire image in memory and demosaic the data. What is does next is apply some normalizations to the linear Pro Photo RGB data and wait for the user to decide what controls, if any, need adjustment. The final stage of processing is to transform to a gamma encoded color space. In between the open and the save, it's really a black box. Some of the information is disclosed in the DNG SDK, but not all of the pipeline...the rest is still proprietary.

If you want to know EXACTLY what ACR/LR is doing when and where and why, the only people who can answer are Thomas and Eric.
Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 29, 2012, 04:14:18 pm
So, at this point my question is to find which of the following is (are) true:

1) This is not possible in LR.
2) This is possible in LR but just not previously considered.
3) This is possible in LR but would require too much planning, code changes or code writing to make it feasible.
4) Nobody knows.

None of the above...

It's a question of usefulness. The point curve editor in LR is first designed for tone mapping. In ACR 7/LR 4, Thomas snuck in per channel color curves because he finally decided it might be useful. The use case you are trying to make is actually questionable because the per color curves were not designed to be used to set end points per color. They were designed to address cross curved color casts. Trying to use the per channel curves to adjust the white balance doesn't make sense. That's what the white balance is designed to do. The curves function in ACR/LR will never match the functionality of curves in Photoshop.
Title: Re: No way to balance color with R,G,B points?
Post by: digitaldog on November 29, 2012, 04:24:38 pm
To be accurate, the "first" thing ACR does when opening a proprietary raw files is to actually convert the file to DNG on the fly, load the entire image in memory and demosaic the data. What is does next is apply some normalizations to the linear Pro Photo RGB data and wait for the user to decide what controls, if any, need adjustment.

Fascinating and thanks Jeff. That's a heck of a lot more about this path than I've seen yet!

If one converts to DNG first, that save any time?

Title: Re: No way to balance color with R,G,B points?
Post by: walter.sk on November 29, 2012, 06:18:19 pm
None of the above...

It's a question of usefulness. The point curve editor in LR is first designed for tone mapping. In ACR 7/LR 4, Thomas snuck in per channel color curves because he finally decided it might be useful. The use case you are trying to make is actually questionable because the per color curves were not designed to be used to set end points per color. They were designed to address cross curved color casts. Trying to use the per channel curves to adjust the white balance doesn't make sense. That's what the white balance is designed to do. The curves function in ACR/LR will never match the functionality of curves in Photoshop.
Thank you.  I can live with that. (Doesn't make me too blue  ::)
Title: Re: No way to balance color with R,G,B points?
Post by: Tim Lookingbill on November 29, 2012, 06:37:17 pm
Quote
Trying to use the per channel curves to adjust the white balance doesn't make sense. That's what the white balance is designed to do.

The WB sliders have one hue missing in its construct which the RGB curves might address and that is the cyan hue. The actual color temp cast appearance in the scene won't change this.

Where cyan is useful in this regard is to perform a hue twist so that warm reddish to yellowish orange tungsten and sunset color casts turns a brownish hue or coffee stain color. This requires cyan and a bit of saturation adjustments without changing luminance which the WB sliders tend to do.

Split Tone offers the solution in ACR but of course it's a bit hard to isolate the effect only in the highlights. I've tried this technique and it works most of the time.

The reason I mention this is because some one posted here a Nikon shot with these brownish tungsten hues and was complaining about this orangish WB appearance with their Canon the poster couldn't remedy.

But if I had the use of separate RGB curves in ACR I wouldn't use them as much except for the above. 

Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 29, 2012, 07:03:09 pm
What you're suggesting is contrary to anything I've read and heard about white balance and raw files. 

the true WB is just multiplying the raw data per channel (like dcraw does) - there are no kelvins, no tint numbers... just the numbers used to multiply the data written per sensel... so if you will multiply each channel by 1.0 you do not change anything and yet the raw data w/o any multiplication (x 1.0) might have the "right" WB as is  (hint - something like magenta filters on your lens or magenta filters on your light)... the to say that the data has no WB is not quite right... it just has WB = 1.0, 1.0, 1.0, 1.0 applied  ;) and with magentish (for classic RGBG bayer) light coming through your lens it might be just what a  doctor prescribed.
Title: Re: No way to balance color with R,G,B points?
Post by: Peter_DL on November 30, 2012, 03:33:26 pm
>> The best place to start in terms of the code would be dng_validate.cpp…  In terms of terminology, the code comments use the term "stage 1" image to describe the raw image data, "stage 2" to describe the linearized raw image data, and "stage 3" to describe the 3-channel or 4-channel data (i.e., after demosaic). If you want to do processing on linearized mosaic data, you want to grab the stage 2 image… If you want to do processing on RGB demosaiced data in the native camera color space, prior to white balance, you want the stage 3 image...<<

the true WB is just multiplying the raw data per channel (like dcraw does)

But this is just parts of the story.

White Balance in ACR/LR by means of Color Temperature and Tint also let you interpolate between, or even extrapolate beyond, a pair of two camera profiles built for illuminants D65 and A.  So the influence of these sliders stretches back to the colorimetric interpretation of the demosaiced data.

Next, the execution of the per-channel-multiplications was reported to be part of the conversion from the camera space to 1.0 ProPhoto RGB, rather than being just applied after arrival in this intermediate working space.  It was explained that Color Temperature, Tint and Exposure, next to the 3 x2 Hue/Sat.-sliders of the Calibrate tab, actually tweak the conversion while corresponding to the 9 degrees of freedom of a 3 x 3 matrix which is still the core to describe the camera space, even though today we may have a HueSat-delta table applied before.  Like with dcraw this allows to steer the highlight clipping behavior.

Or, the correction of chromatic aberration was explicitly stated to be done soon after demosaicing and in the native camera space.  I'd further assume that the repair of single clipped raw channels (by emulating surrounding colors, if done like in dcraw) and also the lens corrections happen at such very early stage as well.

Seems there was more openness on such mechanistic details in early comments and contributions by Bruce Fraser or Thomas Knoll himself – from what I remember.

I'd exempt the above tools and sliders from your initial claim, while counting it among the basic steps of raw conversion, or what is sometimes called scene reconstruction.

Peter

--
... all sliders that you are moving in UI are instructions of what shall be done with the data that was already 1) demosaicked and 2) transformed to a color space with prophoto coordinates and linear gamma... why do you want to argue w/ that ?
Title: Re: No way to balance color with R,G,B points?
Post by: Vladimirovich on November 30, 2012, 04:18:03 pm

I'd exempt the above tools and sliders from your initial claim, while counting it among the basic steps of raw conversion, or what is sometimes called scene reconstruction.


nope - as long as the do not operate on per channel data from sensor sensels they do not operate w/ raw data... but yes, if Eric Chan posted somewhere (I need to find, I did not finish my work studying all of his posts over years) that CA corrections operate on demosaicked data BUT before chains of conversions to cieXYZ to Prophoto/1 then certainly that operates at least in camera's "RGB space" (non colorimetric - is it the right way to say ?)...

PS: is it legal to publish a collection of somebody's posts quoted as a single file (free to download) ?
Title: Re: No way to balance color with R,G,B points?
Post by: Schewe on November 30, 2012, 05:18:43 pm
PS: is it legal to publish a collection of somebody's posts quoted as a single file (free to download) ?

Reposting a collection of what somebody wrote without explicit permission would be questionable. Quoting somebody is acceptable...