Does a raw file have a color space?

[Peter_DL]

I remember that quote well because I was participating in that thread on the Adobe forum. I could never quite figure out what "linear combination" meant. What is being plotted on the x and y axis to give a linear curve?
[{POST_SNAPBACK}][/a]

Bill,

Based on the CIE XYZ weighting functions, their x-axis is stated as Wavelength [nm]. For respective XYZ color filters, the y-axis would have to scale in terms of Transmittance; at least that’s what the logic of the [a href=\"http://www.fho-emden.de/~hoffmann/ciexyz29082000.pdf]Color matching experiment[/url] seems to suggest.

A linear combination is a blend of these curves as follows:
R-filter= a*X(nm) + b*Y(nm) + c*Z(nm)
G-filter= d*X(nm) + e*Y(nm) + f*Z(nm)
B-filter= g*X(nm) + h*Y(nm) + i*Z(nm)

If this is perfectly fulfilled, the camera’s unambiguously interpretable gamut would be one single matrix space – representing a subset of the corresponding CIE XYZ master space.

If this not perfectly fulfilled (metamerism index of the sensor < 100), we are in need of a Lut-type profile or a “best-fit” matrix space per illuminant, in order to come to a somewhat colorimetric interpretation and conversion of the Raw data into a CIE XYZ sphere again.

My 2ct.
& Best regards….

--
P.S.: fyi

[bjanes]

Based on the CIE XYZ weighting functions, their x-axis is stated as Wavelength [nm]. For respective XYZ color filters, the y-axis would have to scale in terms of Transmittance; at least that’s what the logic of the Color matching experiment seems to suggest.
[a href=\"index.php?act=findpost&pid=168645\"][{POST_SNAPBACK}][/a]

Peter,

Many thanks for the explanation of the x and y axis and the Wandell reference. It is now clear to me what linear refers to.

A linear combination is a blend of these curves as follows:
R-filter= a*X(nm) + b*Y(nm) + c*Z(nm)
G-filter= d*X(nm) + e*Y(nm) + f*Z(nm)
B-filter= g*X(nm) + h*Y(nm) + i*Z(nm)

If this is perfectly fulfilled, the camera’s unambiguously interpretable gamut would be one single matrix space – representing a subset of the corresponding CIE XYZ master space.

If this not perfectly fulfilled (metamerism index of the sensor < 100), we are in need of a Lut-type profile or a “best-fit” matrix space per illuminant, in order to come to a somewhat colorimetric interpretation and conversion of the Raw data into a CIE XYZ sphere again.
[a href=\"index.php?act=findpost&pid=168645\"][{POST_SNAPBACK}][/a]

To relate these principles to commonly used raw converters, I surmise that Camera Raw and many other converters use a global linear 3x3 matrix conversion and can not obtain a true colorimetric interpretation. Nonetheless, the results can be quite good, since pleasing rather than perfectly accurate color is sufficient for most general photography.

I'm not that familiar with Capture One, but understand that it can use LUT type profiles for a somewhat colorimetric interpretation, but this only works for a specific illuminant. This might be helpful in the studio with controlled lighting, but not for general field photography where the illuminant varies widely.

Wandell talks about polynomial fits and neural networks. Are these used in common raw converters?

Bill

[Jonathan Wienke]

Wandell talks about polynomial fits

I have them when I spent too much time reading pages of physics equations (research for a project unrelated to photography, at least for now)...

[Eric Myrvaagnes]

"Polynomial fits", indeed!

Thank you Bill and Jonathan for brightening my day.  

[eronald]

"Polynomial fits", indeed!

Thank you Bill and Jonathan for brightening my day.   
[a href=\"index.php?act=findpost&pid=168710\"][{POST_SNAPBACK}][/a]

I think this discussion is regressing toward infantility. Of course that's only a first approximation of my feelings

Edmund

[bjanes]

I think this discussion is regressing toward infantility. Of course that's only a first approximation of my feelings

Edmund
[{POST_SNAPBACK}][/a]

Edmund,

To each his own. Here is another take on this thread by some pretty knowledgeable people over on the [a href=\"http://forums.dpreview.com/forums/read.asp?forum=1021&message=26426262]DPReview Nikon D3 forum[/url]. There are also some interesting comments on the behavior of some contributers to the current thread.

Iiah Borg has written Raw Magick, a highly regarded Nikon raw converter.

Bill

[Schewe]

Here is another take on this thread by some pretty knowledgeable people over on the DPReview Nikon D3 forum.
[a href=\"index.php?act=findpost&pid=168919\"][{POST_SNAPBACK}][/a]

Well,  Iliah doesn't seem to agree with ya when he says: "There is definition of what colour space is (in the sense we use it in image processing, and according to ICC standards), and raw data does not follow it."

So, I guess Iliah doesn't think raw captures (or cameras) have a color space either...

Cool!

:~)

[Tim Lookingbill]

Just found something new and interesting buried in this thead I've been exploring. I'm glad I hitched on to this discussion.

On page 26 of Gernot Hoffman's pdf posted in this thread are plotted coordinates of Optimized Primaries on the CIE locus. I've been wondering if such a thing actually existed and would be of some practicle use.

Entered them into Raw Developers custom matrix based profile builder and got a better rendering of a random test shot containing wide gamut colored objects and fleshtone-(the back of my hand) lit by golden hour sunlight.

I was able to bring out more distinction between subtle colors for example in a limestone rock fence included as a backdrop in the shot where I could tell more of a difference between the beige colored mortar, off white limestone rock and the bluish concrete foundation. The default internal profiles of two raw converters pretty much made them look as one color.

I also didn't have to go fishing around for scene referred color temp constantly adjusting red/green sliders. Just clicked on and applied small tweaks to each color gamut triangle corner points and center color temp dot and the white balance as I saw it fell into place. Better than using Hue/Saturation sliders, neutral eyedropper tool or working in LAB space.

This is like a completely different color correcting tool even though I'm not using it as it was intended. I get to work in native camera color space which clips far less than using RD's other CIELab space based color sliders.

I just set color temp to 5000K, output space and working space to ProPhotoRGB and start tweaking. Probably not the most accurate or efficient way of doing things I admit but I found the Optimized Primaries gave me a better starting point to work from in nailing the color as I saw it. No more reddish or overly yellowish fleshtones which is how both Raw Developer and Pentax's SilkyPix based converters' default settings rendered it in the test shot.

I haven't tested if the saved profile would render other images the same way on a consistant basis. I doubt it, but I thought it was a fun, interesting and new way of editing color I've never tried before.

You might say I "found" or "defined" the color space of my camera which seems to be based loosely on Optimized Primaries and 1.0 gamma.

[Graeme Nattress]

At some point, and I think we're beyond  that here, these discussions just turn into "the definition game".

Graeme

[papa v2.0]

Entered them into Raw Developers custom matrix based profile builder and got a better rendering of a random test shot containing wide gamut colored objects and fleshtone-(the back of my hand) lit by golden hour sunlight.


[a href=\"index.php?act=findpost&pid=168945\"][{POST_SNAPBACK}][/a]

Is that the program by Iridient Digital, and if so where can you enter the primaries? Cant seem to find how

cheers

[Tim Lookingbill]

Is that the program by Iridient Digital, and if so where can you enter the primaries? Cant seem to find how

cheers
[a href=\"index.php?act=findpost&pid=168998\"][{POST_SNAPBACK}][/a]

It's probably been disabled by default. At least that's what RD's Help section indicates. Click on Enable Advanced Settings in Preferences.

At the bottom of the Input Tab will be a button that says Edit/Create Profile. You won't be able to edit RD's internal profile of your particular model of camera. What does get assigned/loaded as a starting point is standard sRGB primaries with a 1.0 gamma which will make the preview appear significantly desaturated and washed out depending on exposure and/or how different RD's internal profile is.

It really pays off to ETTR when messing around like this at least it was with my test image. Another option that can be a bit difficult but can grab quite a bit of dynamic range with very little noise in the shadows is editing RD's internal tone curve also in the Input Tab.

Editing for shadows in a linear space requires careful tweaking in a tiny area off the corner black point or else you'll kick up quite a bit of noise in the shadows. You'll be surprised how many tweak points you can cram into there.

Depending on the colors in your image you may have to increase saturation just as a starting point in the Input Color Matrix. Applying it in the other tab encodes in CIELab which can make the histogram go off the charts quite quickly even with Output and Working Space set to ProPhotoRGB. Though in my case it may be due to my using the standard 1.8 gamma ProPhotoRGB which isn't the same as ACR's internal 1.0 gamma version.

Just remember all adjustments in the Input tab only encodes in the linear camera native color space.

[bjanes]

Well,  Iliah doesn't seem to agree with ya when he says: "There is definition of what colour space is (in the sense we use it in image processing, and according to ICC standards), and raw data does not follow it."

So, I guess Iliah doesn't think raw captures (or cameras) have a color space either...

Cool!

:~)
[a href=\"index.php?act=findpost&pid=168931\"][{POST_SNAPBACK}][/a]

As with all of the experts who have commented, Iliah places qualifications on what he considers a color space. He is apparently talking about a colorimetric ICC type color space.The color space of the camera does not conform the ICC spec. He is apparently talking about a colorimetric ICC type color space.

RikWalowit wrote:

<<Does a raw file have a color space?>>

Fundamentally, absolutely YES, but we may not know what that color space is. The image was recorded through a set of camera spectral sensitivities which defines the intrinsic colorimetric characteristics of the image. An simplistic way to think of this (while not purely accurate) is that the image was recorded through a set of "primaries" and these primaries define the color space of the image.

Practically, it makes little difference unless you are interested in accurate scene-referred data. In the context you described, a simple transform is applied to convert from the camera primaries to a new set of primaries (eg CIE or working space) that have more desirable characteristics than the RAW primaries.

Mathematically, of course, you know the foregoing is true. Matrix algebra informs that if the final color image encoding is a "color space" then so must be all of its infinite number of linear transforms (lets ignore the 1 D non-linear transfer functions), including the original RAW encoding.

Thomas Knoll wrote:

The fact that a mosaic array is “grayscale” is a red herring in this argument.  An early processing step fills in the missing values, and you have a 3 or 4 channel image as a result.  For most cameras, if you just “assign” a working space RGB profile, you get a recognizable color image as a result, so it certainly seems like a color space.

The camera color space differences from a more common working color space in that it does not have a unique one-to-one transform to and from CIE XYZ space.  This is because the camera has different color filters than the human eye, and thus sees colors differently.  Any translation from camera color space to CIE XYZ space is an approximation because of this.

Chris Murphy wrote:

I think we'd all agree that CIE XYZ, ProPhoto RGB, and sRGB are each a color space. We consider the whole of ProPhoto RGB a color space even though it contains imaginary colors, from a human point of view. And we consider the whole of sRGB a color space even though it contains less than the colors that we humans can see.

A camera has colors it really can capture and encode as unique values compared to others, that are imaginary to us. They don't exist. There are colors we can see, but the camera can't capture that are imaginary to it. So what to do with these captured values that humans can't see? Well they typically get mapped to something, possibly something that another rather different SPD is already mapped to. Two reds that appear the same to us, but have different SPDs, yet the camera sees them as different so it encodes them with two different RGB values. That's fugitive colors. It's nothing new in photography.

So yes a camera (and thus a Raw file) has a color space. The challenge is what to do with the real and "unreal" colors it's encoding. And how to deal with luminance and dynamic range which is still a really big weak area of the ICC workflow. This is not a one size fits all campaign, to get good results. Color appearance models can help us deal with this better, as getting a 1:1 correlation to XYZ is an appropriate starting point prior to color appearance and gamut mapping, but it's not a desirable end result either on-screen or in print.

Your own arguments about a raw file being grayscale are specious and have been refuted by Thomas Knoll and others. You completely ignore opinion and evidence contrary to your own inflexible and narrow views and do not respond to explanations by others to your objections. Perhaps better not to respond than to argue the unarguable and hope no one will notice. I agree with the DPReview author that it is useless to argue anything with some resident "experts".

"He who corrects an arrogant man earns insult; and he who reproves a wicked man incurs opprobrium. Reprove not an arrogant man, let he hate you; reprove a wise man, and he will love you. Instruct a wise man, and he becomes still wiser; teach a just man, and he advances in learning."
-- Proverbs 9:7-9"

[digitaldog]

As with all of the experts who have commented, Iliah places qualifications on what he considers a color space. He is apparently talking about a colorimetric ICC type color space.The color space of the camera does not conform the ICC spec. He is apparently talking about a colorimetric ICC type color space.

Yes so in this context, its not a color space. Jack Holms said the same thing:

A purist might argue that a color space not based on colorimetry is not really a color space because it is not an assignment of numerical values to colors, defining colors as a human sensation. In the standards committees we decided it is useful to be able to talk about non-colorimetric color spaces so we allow them and use “colorimetric color spaces” when appropriate.

So Jeff and Iliah are as Jack puts it, purists.

We've gone over this to death and it apparently will not end until we now define if the Raw represents a colorimetric space. Some say yes, some say no.

[bjanes]

Yes so in this context, its not a color space. Jack Holms said the same thing:
So Jeff and Iliah are as Jack puts it, purists.

We've gone over this to death and it apparently will not end until we now define if the Raw represents a colorimetric space. Some say yes, some say no.
[a href=\"index.php?act=findpost&pid=169056\"][{POST_SNAPBACK}][/a]

Quite true. Iliah is a purist. To the best of memory, all experts in this thread have verified that raw is not colorimetric, at least with the current CFA filters. However, it still can be considered a color space according to a broader definition. The matter is more than semantics, since the nature of the color space has broad implications for rendering of the color. Dr. Wandell's article discusses ways to overcome non-colorimetric properties.

These distinctions seem to escape Jeff, who seems more like a bull in a china shop than a purist. His lack of insight and subtlety astonish me.

[Schewe]

These distinctions seem to escape Jeff, who seems more like a bull in a china shop than a purist. His lack of insight and subtlety astonish me.
[a href=\"index.php?act=findpost&pid=169072\"][{POST_SNAPBACK}][/a]

Very little actually "escapes" me bud...and I have no desire nor intent to be "subtle", why should I? But for several years now, you've gone out of your way to be a pain in my arse because of some unknown (and prolly unintended) perceived injustice I committed against you on the Camera Raw User to User fourms...

If you think your petty and childish petulance will have any impact on my judgement or behavior, I suspect you've got other delusional problems as well.

There are things I know I know...there are things I think I know and there are things I would like to know. The sources of my knowledge are pretty varied...it helps to hang around geeks and absorb knowledge (which doesn't always come with understanding) and I'm more than happy to learn things I don't know. I am however, rightly skeptical about the alleged knowledge of many other people I don't know.

Take you for example...I've NEVER seen an actual PHOTOGRAPH from you that gives me any indication you do anything other than sit around with charts and graphs and a calculator. You sometimes seem to grasp some things while being incredibly stubborn about other things. But you couldn't prove it by me that you ARE a photographer...

This whole thread, for an example, is a lightly veiled attempt at disputing things that Andrew and I have said about raw files.  Fine and dandy dooode...no, the actual raw image captured on card ain't grayscale...ok? But, it's more like a grayscale file than a color file at least until some sort of color interpretation or demosaicing is done.

You've tried to prove that cameras or raw captures have a "color space" because you either want to prove Andrew and I wrong and therefore enjoy some sort of superiority over us or try to embarrass us. If your true desire were an honest and noble "search for the truth", you wouldn't be throwing the zingers in the discussion about our supposed (or my supposed) boorish behavior.

If you truly were without fault, you wouldn't be stooping to those wasted attempts at behavior modification. I would suggest you look into the mirror dooode. Ask yourself, how are YOU behaving? You may be more "subtle" (something you seem to think has some perceived value) but I challenge you name yourself blameless...people in glass houses shouldn't be throwing stones, ya know?

So, the opening question and the premise of this whole thread "Does a raw file have a color space?" has been answered? Does it or doesn't it? What do _YOU_ think?

[digitaldog]

Quite true. Iliah is a purist. To the best of memory, all experts in this thread have verified that raw is not colorimetric, at least with the current CFA filters. However, it still can be considered a color space according to a broader definition.
[a href=\"index.php?act=findpost&pid=169072\"][{POST_SNAPBACK}][/a]

Yes, if you decide that's the definition that best makes your argument correct. At this point, that's where we're heading. It seems that unless everyone agrees with you that a Raw expresses a color space and to do that, we'd need to disagree with the experts that say its not colorimetric.

One could say, the broader definition is, Raw is essentially Grayscale but that didn't sit well with you. So at this point, I don't see why this thread needs to go any further. Broader or narrower, this is a debate that isn't going to come to full agreement. Why continue?

[eronald]

Edmund,

To each his own. Here is another take on this thread by some pretty knowledgeable people over on the DPReview Nikon D3 forum. There are also some interesting comments on the behavior of some contributers to the current thread.

Iiah Borg has written Raw Magick, a highly regarded Nikon raw converter.

Bill
[a href=\"index.php?act=findpost&pid=168919\"][{POST_SNAPBACK}][/a]

I'm sorry, Bjanes, somebody else was talking about polynomial fits, which can be used as a form of the approximation method called regression. Regression is the approved doctrine for constructing camera profiles.

Last time I looked puns were still allowed on this forum. We went from throwing fits to regression, maybe what is funny to me isn't funny to you  

From the list of participants here, seems to me we've done the expert bit quite adequately, and it might be time to wrap up and head home.

Andrew - I agree we've come full circle. Time to go have a virtual beer.


Edmund

[John Sheehy]

One could say, the broader definition is, Raw is essentially Grayscale but that didn't sit well with you. So at this point, I don't see why this thread needs to go any further. Broader or narrower, this is a debate that isn't going to come to full agreement. Why continue?
[a href=\"index.php?act=findpost&pid=169092\"][{POST_SNAPBACK}][/a]

RAW is not "grayscale", any more than aRGB or sRGB is grayscale.  The files or data for all contain numbers, numbers that are colorless until they are colored through some understood meaning.

The confusion comes, perhaps, from the fact that the most efficient and convenient way to store a RAW image is to lay it out flat in one plane, as opposed to three (since with CFAs, there is only one color plane used per 2D pixel location), and hence the RAW data "fits" into the greyscale image format.  Its R,G, and B channels are just as distinct in meaning as are those in aRGB and sRGB.

[Graeme Nattress]

Actually, the same can be said of many RGB formats which are often stored interleaved (ie non planar), RGBA, and can be interpreted as a greyscale mosaic.

I can see how people can discus "what do we mean by "colour space"", but to get into disagreements over RAW Bayer pattern data saying it can't be a colour space because it's greyscale are, I'm sorry to say, daft beyond belief. There are valid un-demosaiced interpretations of raw Bayer data that directly produce colour images that are as equally valid as the so called greyscale interpretation, just as I can easily produce a greyscale mosaic interpretation of some RGB data.

Graeme

[digitaldog]

RAW is not "grayscale", any more than aRGB or sRGB is grayscale.  The files or data for all contain numbers, numbers that are colorless until they are colored through some understood meaning.

Ah but if we use the so called wider definition, sRGB is most certainly three Grayscale channels. You see, if you wish to define something widely, you can.

And you say "colored through some understood meaning" but an untagged document is an undefined color space. So what is it? We can define it as we wish, again using wider definition.

I was told I was being silly earlier in the thread by taking like a lawyer. But there's so much fudge factor here, we might as well end this by agreeing to disagree. But the color geeks have said pretty clearly that a non colorimetric Raw file isn't in a color space. But it is in the wider definition....