Content of RAW files?

[r010159]

I apologize if this topic is in the wrong forum. I cannot find one strictly on technology. I am interesting in finding out what makes up a RAW file. Also, I would like some idea about how this information is processed to produce the final result. I have heard of some terms like demosaicing. I understand this involves approximations. Then there is the application of a tonal curve that the application chooses to apply. There may be some noise reduction process considering that there are approximations being made.

My objective is to know the factors that result in one image from a RAW processor to be handled differently from that produced by another RAW processing program other than saturation, sharpening, and other functionality normally considered as a part of a photo editing program.

Bob

[Tony Jay]

Its nonetheless a fair question.

Basically your question revolves mostly around the process of demosaicing.
This is part of the process of raw file conversion done in a raw converter.

In order to explain how demosaicing works a very quick and basic explanation of a digital camera sensor and how it works is needed:
Think of the light-gathering part of the sensor as a grid with a well that collects light in each grid component. Depending on how much light hits the well each sensel (sensor element) converts that physical light hitting the sensel into a charge which is ultimately recorded as a number. This is referred to as analog-to-digital conversion. (accept for now that this is not a detailed technical expose.)
Furthermore above the sensor is a colour filter that is a repeating structure of four filters each containing two green components and a red and blue element each. Each element of the filter overlies a single sensel. That means that light passing through the filter on the way to a particular sensel is flltered by colour - either green, red, or blue.
The sensels themselves have no idea what colour light is hitting them - they just record light in shades of gray - this is important.

The demosaicing process not only decodes the gray tones from each sensel but also interpolates the overall colour of the light to assign to that sensel by looking at light striking the sensels around them. Since the sensels around are filtered for different colour ultimately colour is assigned to each sensel by assigning values to each of the three primary colours e.g. R65 G55 B78.

Another part of raw conversion involves changing the contrast. Suffice to say for now that the original image is very flat with a gamma curve of 1.0. Generally the gamma curve applied is 2.2 which increases the contrast considerably to something a little more akin to how we see.

Each raw converter uses a different algorithm to achieve this entire process accounting for some of the subtle, and not so subtle differences, between raw converters.

My suggestion is to go and read a formal expose of raw conversion and demosaicing. I would find a link myself but I am bit pressed for time right now!

Go well!

Tony Jay

[Jack Hogan]

My objective is to know the factors that result in one image from a RAW processor to be handled differently from that produced by another RAW processing program other than saturation, sharpening, and other functionality normally considered as a part of a photo editing program.

Hi Bob,

What most people consider 'raw converters' do can actually be divided into two macro functions for simplicity:

1) objective rendering of the raw file to the chosen standard color space (this is the raw 'conversion' part, it requires virtually no human input); and
2) arbitrary PP to make the image more pleasing to the eye of the artist (can be done in any editor, like Photoshop - choose the one with the tools and the workflow you like).

The first is quantitative and objective, the second qualitative and subjective.  99% of the adjustments the average photographer performs daily on their images during what they consider to be 'raw conversion' fall squarely into category 2) and can be performed just as effectively in one or more editors/plug-ins with no loss in IQ: just pass the image from one to the other as a 16-bit or floating point TIFF.  For instance: black/white levels, curves, highlight/shadow recovery, vibrance, clarity, sharpening etc.etc.etc are all basic editing functions.  'Raw converters' simply try to bring all typical adjustments in one place and make them faster/easier to use.

Jack

[JRSmit]

Its nonetheless a fair question.

Basically your question revolves mostly around the process of demosaicing.
This is part of the process of raw file conversion done in a raw converter.

In order to explain how demosaicing works a very quick and basic explanation of a digital camera sensor and how it works is needed:
Think of the light-gathering part of the sensor as a grid with a well that collects light in each grid component. Depending on how much light hits the well each sensel (sensor element) converts that physical light hitting the sensel into a charge which is ultimately recorded as a number. This is referred to as analog-to-digital conversion. (accept for now that this is not a detailed technical expose.)
Furthermore above the sensor is a colour filter that is a repeating structure of four filters each containing two green components and a red and blue element each. Each element of the filter overlies a single sensel. That means that light passing through the filter on the way to a particular sensel is flltered by colour - either green, red, or blue.
The sensels themselves have no idea what colour light is hitting them - they just record light in shades of gray - this is important.

The demosaicing process not only decodes the gray tones from each sensel but also interpolates the overall colour of the light to assign to that sensel by looking at light striking the sensels around them. Since the sensels around are filtered for different colour ultimately colour is assigned to each sensel by assigning values to each of the three primary colours e.g. R65 G55 B78.

Another part of raw conversion involves changing the contrast. Suffice to say for now that the original image is very flat with a gamma curve of 1.0. Generally the gamma curve applied is 2.2 which increases the contrast considerably to something a little more akin to how we see.

Each raw converter uses a different algorithm to achieve this entire process accounting for some of the subtle, and not so subtle differences, between raw converters.

My suggestion is to go and read a formal expose of raw conversion and demosaicing. I would find a link myself but I am bit pressed for time right now!

Go well!

Tony Jay

Small addition: RGB values in a given colorspace!

[GrahamBy]

https://en.wikipedia.org/wiki/Demosaicing

[Tony Jay]

Small addition: RGB values in a given colorspace!

Correct!

Tony Jay

[torger]

While there are differences in demosaicing, noise reduction and other signal processing aspects, the major part that causes differences in look between raw converters is the color profile and tone curve application, that is what takes place after the basic raw conversion has been made.

Even with the same linear color in the bottom, the type of tone curve makes a large difference. With RawTherapee you can experiment with several different types of tone curves and see how big difference in look that can make. With low saturation colors the differences are typically small, but with higher saturation colors you can get very big differences.

Then we have the tradition from the film days, that color should not be neutral and realistic but rather have some sort of "look", and this is interpreted way differently between manufacturers.

[bjanes]

Another part of raw conversion involves changing the contrast. Suffice to say for now that the original image is very flat with a gamma curve of 1.0. Generally the gamma curve applied is 2.2 which increases the contrast considerably to something a little more akin to how we see.

Tony, you have provided a nice summary of demosaicing, but I take exception to your statements regarding gamma. When gamma encoding is accomplished, one applies a 1/2.2 power function to the data. When the image is output to the screen, a 2.2 power function is applied, undoing the 1/2.2 transform and restoring linearity.

This figure from Poynton's gamma FAQ illustrates the process. The contrast for a given input value is the slope of the curve. At low input values, slope is greater than one, indicating greater contrast in the shadows. At higher input values, the slope is less than unity, indicating reduced contrast.

Regards,

Bill

[Tony Jay]

Tony, you have provided a nice summary of demosaicing, but I take exception to your statements regarding gamma. When gamma encoding is accomplished, one applies a 1/2.2 power function to the data. When the image is output to the screen, a 2.2 power function is applied, undoing the 1/2.2 transform and restoring linearity.

This figure from Poynton's gamma FAQ illustrates the process. The contrast for a given input value is the slope of the curve. At low input values, slope is greater than one, indicating greater contrast in the shadows. At higher input values, the slope is less than unity, indicating reduced contrast.

Regards,

Bill

Thanks for the input Bill.
I will look into this.

Tony Jay

[Jack Hogan]

When gamma encoding is accomplished, one applies a 1/2.2 power function to the data. When the image is output to the screen, a 2.2 power function is applied, undoing the 1/2.2 transform and restoring linearity.

Good point, Bill.  Luminance from the scene (either directly, through a window, or through capture+monitor) ideally behaves linearly: twice the 'light', twice the photons hitting our eyes.  The human visual system will then perceive the arriving photons as it always does (non-linearly).

Jack

[Bart_van_der_Wolf]

Tony, you have provided a nice summary of demosaicing, but I take exception to your statements regarding gamma. When gamma encoding is accomplished, one applies a 1/2.2 power function to the data. When the image is output to the screen, a 2.2 power function is applied, undoing the 1/2.2 transform and restoring linearity.

Hi Bill,

While correct, it's important to also point out that many image manipulations, e.g. in Photoshop, are performed on gamma pre-compensated TIFF/JPEG input. That can result in color shifts when brightness and/or contrast are changed, because the linear relationship between R, G and B is 'broken' if the workingspace gamma is not linear.

But, for the OP, that's all after Raw conversion, which is a process based on linear (or linearized) gamma Raw data (usually in Bayer CFA format before it's demosaiced). After blackpoint subtraction, white balancing, and demosaicing, the postprocessing can begin (including the application of gamma pre-compensation).

Some Raw converters can also compensate for e.g. Chromatic Aberrations, before demosaicing, which will improve resolution because the RGB channels will be demosaiced while in registration/aligned.

Cheers,
Bart

[AlterEgo]

as all (?) raw converters combine (as noted) "raw conversion" and "postprocessing" the question is where the "raw conversion" ends and "postprocessing" begins... two school of thoughts : (A) demosaicking is the end of the "raw conversion" and (B) any color transform to any formal color space is the end of the "raw conversion"

[Jack Hogan]

the question is where the "raw conversion" ends and "postprocessing" begins... two school of thoughts : (A) demosaicking is the end of the "raw conversion" and (B) any color transform to any formal color space is the end of the "raw conversion"

A good question with many answers.  In my mind the raw 'conversion' part converts raw data to a standard format (e.g. a colorimetric RGB color space like sRGB) that can be understood and made more pleasing through subsequent use of any available PP tool.  So I would suggest that raw conversion includes all steps that need direct access to the raw data but stops as soon as the image is demosaiced and rendered into the working color space.   There are no sliders necessary in raw conversion, just inputs needed to describe the scene, the hardware and the output color space: white balance, color profile (or matrix, absent one of torger's) and desired color space.  Additionally some lens/camera setup information could be useful for CA correction and low-level noise reduction.

dcraw for instance is in my opinion such a 'just' raw converter.

Jack

[AlterEgo]

There are no sliders necessary in raw conversion, just inputs needed to describe the scene, the hardware and the output color space: white balance, color profile (or matrix, absent one of torger's) and desired color space.  Additionally some lens/camera setup information could be useful for CA correction and low-level noise reduction.

WB means sliders... NR & possible sharpening means again sliders... even CA can.

also some converters do apply curves before demosaicking...

however if you 1) take Adobe's workflow and 2) join the school of thought that demosaicking is the end of the raw conversion, then indeed ACR/LR are simply purely post-processing tools __UI wise__ (the only exception is the process version drop down control that shall affect things happening at least during demosaicking process).

[Torbjörn Tapani]

  Additionally some lens/camera setup information could be useful for CA correction and low-level noise reduction.

dcraw for instance is in my opinion such a 'just' raw converter.

Jack

Yes, very little mention of additional exif info. RAW files might contain proprietary information that the manufacturers RAW-converter might use to better reflect in camera settings or apply lens corrections.

[Jack Hogan]

WB means sliders... NR & possible sharpening means again sliders... even CA can.

also some converters do apply curves before demosaicking...

Fair enough, although really all you typically need is to know a little scene information, such as whether the capture was taken in the sun, the shade, cloudy.  WB Sliders are a bonus and most people do not touch them.

Low level NR requires no sliders, just camera setup info (this has nothing to do with the noise reduction performed by most raw 'converters').  Sharpening is performed on rendered data in pretty well all raw converters, as far as I know.

No curves before demosaicing, though ideally a generic curve is already part of the color profile, per torger.  Just select the scene type: Landscape, Portrait, Neutral, Standard (where have I seen those labels before?:-)

Jack

[AlterEgo]

Low level NR requires no sliders, just camera setup info (this has nothing to do with the noise reduction performed by most raw 'converters').  Sharpening is performed on rendered data in pretty well all raw converters, as far as I know.

if you consider yourself as belonging to a school of thought that considers raw conversion ending with color transform then some raw converter can in fact do both regular NR and sharpening prior to that color transform for some reasons.

No curves before demosaicing

you then probably not familiar with RPP raw converter ?

[AlterEgo]

WB Sliders are a bonus and most people do not touch them.

but they will be still present in UI regardless...

[Jack Hogan]

but they will be still present in UI regardless...

Not necessarily: dcraw does not have any sliders and I would argue that WB sliders in raw converters are all about pleasing tones as opposed to accurate ones: the former is PP, the latter is raw conversion. 

Jack

[AlterEgo]

Not necessarily: dcraw does not have any sliders

sure, as it is a command line tool - that is by defintion... however it can accept arbitrary multipliers for WB

and I would argue that WB sliders in raw converters are all about pleasing tones as opposed to accurate ones: the former is PP, the latter is raw conversion. 

WB can do both - when you aim for a pleasing result and when you aim for precise reproduction... the mere fact that 99.99% of users may allegedly not use WB sliders at all and think that WB is for a pleasing result still does not change the fact... this is not a vote, even 1 example counts to the contrary, so no point to argue