Luminous Landscape Forum
Equipment & Techniques => Digital Cameras & Shooting Techniques => Topic started by: fdisilvestro on January 14, 2013, 09:51:27 pm
-
Hi,
Just for curiosity, does anybody know what could be the reason or benefit of multiplying the values from the R and B channels by a factor? I mean, not the factors for White Balance, but a scalar multiplication of the raw values after converting to digital (what you get in a "unprocessed NEF"). This can be shown by looking at raw histograms, where you can see a periodic "hole" in the histograms of the Red and Blue channels.
The attached image shows the histogram from RawDigger for the first 128 values (0-127) of a 14-bit uncompressed NEF file at base ISO (100) from a D800. The pattern will look the same if you show any set of values. This is a clear indication of a scalar multiplication after converting to digital, and happens with other models too (I have observed this on D300 files too, using RawDigger and Rawnalize).
The factors are different for the red and blue channels, adding more curiosity to why they are doing this. From my understanding of digital image processing, they could just adjust the WB factors and achieve the same end result.
A practical implication (which I think it is negative) is that if you shot at base ISO, the green channel shows saturation at a value close to 15800 (out of 16383 for 14 bit) while the red and blue channels go all the way up to 16383. This give the false impression of the green channel saturating before the other channels, but it is just the effect of the multiplication mentioned before, which will throw values close to 15800 well above 16383 for the R and B channels. The second image shows the histogram for the highest values, where the early saturation of the green channel could be observed.
I have found explanations for other manipulations of the raw data in Nikon cameras, such as the black point offset and the "Star-killing" algorithm, but I haven't found any about this.
Regards,
Francisco
-
Are you looking at 14 bit data in 16 bit space? My software has radio buttons for looking at the histogram in 8,10,12,14,16 bits My Raws are 12 bit. If I select anything up to including 12 the file is continuous. If I select 14 or 16 it shows gaps.
Most data manipulations (photo edits) I do after conversion are in the full 16 bit tif space and the file is back to smooth when viewed in 16 bit.
-
Hi,
No data manipulation, these are the raw values as they are in the NEF file, whit the x-axis showing each individual value.
If you have a 12 bit raw file then you will have values between 0 and 4095; a 14 bit raw file will have values between 0 and 16383 and so on.
When the histogram shows "holes" in the red channel at values 32,41,50,59,68, etc. (or every 9 values) it means that there are no red pixels with those values, typical of a multiplication.
As an example, multiplying the raw values by 2 is equivalent to an exposure compensation of +1 EV, but you will end with a raw file where there are no odd values for any pixel
-
Hi,
No data manipulation, these are the raw values as they are in the NEF file, whit the x-axis showing each individual value.
If you have a 12 bit raw file then you will have values between 0 and 4095; a 14 bit raw file will have values between 0 and 16383 and so on.
When the histogram shows "holes" in the red channel at values 32,41,50,59,68, etc. (or every 9 values) it means that there are no red pixels with those values, typical of a multiplication.
As an example, multiplying the raw values by 2 is equivalent to an exposure compensation of +1 EV, but you will end with a raw file where there are no odd values for any pixel
I understand your idea that it is a multiplication. What I was proposing is that it is your view settings in the software. I don't know the software so I am suggesting the view is 16 bit 0-65535. The missing values would be rounding.
A sensor cannot fill up skipping values. You could have the occasional histogram value that is skipped but not a repeating pattern, that is a manipulation.
Your ISO gain would skip groups. What is the shot's ISO? What is that ISO as listed on DXO? Try using those values for your multiplication factor.
-
There would be no gain at your camera's base ISO. Does it vanish in RAWs at base ISO?
-
Hi,
Those are at base ISO, there are no skipped values at the histogram. This is actually a known characteristic of NEF files, but what I have never found is an explanation on why it is done
Regards
-
does anybody know what could be the reason or benefit of multiplying the values from the R and B channels by a factor?
It looks like White Balance pre-conditioning. Since the Blue and Red filters in the CFA are less sensitive than the Green filter, the R* and B* data is scaled to provide an approximately 'correct' relative output in the raw data, partly compensating for the filters' different physical properties - prior to light source dependent white balance. See below for an example.
(http://www.qsimaging.com/images/photos/KAF-8300-color-QE.jpg)
Saturation of the Green channel is what it is by design. Since the Red and Blue sensels have physically more 'highlight headroom' it makes sense to have them use the entire 14 bit range when scaled to maintain accuracy. Other than in blown highlight situations where this becomes apparent, it is uncommon to find complete (i.e. not just a saturated detail) natural scenes where the Red or Blue raw channel clip before green, before or after scaling. Take a look at white balance settings in various lighting conditions to see that.
Cheers,
Jack
-
> It looks like White Balance pre-conditioning
It is "WB preconditioning".
-
Other than in blown highlight situations where this becomes apparent, it is uncommon to find complete (i.e. not just a saturated detail) natural scenes where the Red or Blue raw channel clip before green, before or after scaling.
Hi Jack,
It is quite common for Red in flowers to saturate before the Green channel does. That would create an even bigger issue with this WB pre-compensation, leading to a requirement to under-expose the Green channel, which becomes more noisy. That would be bad news if you are in the business of flower photography (which is a serious branch of product photography in the Netherlands).
Since this post quantization multiplication doesn't add one iota of accuracy, I also wonder if there is another reason, e.g. such as improved compression (speed/efficiency).
Cheers,
Bart
-
I simply don't understand the need for scaling in a format which no user can ever see.
But then I'm really not good at all this analytical stuff, working out circuit functionality in the IC reverse engineering class I took was a pain, while most of my classmates liked it.
Edmund
Hi Jack,
It is quite common for Red in flowers to saturate before the Green channel does. That would create an even bigger issue with this WB pre-compensation, leading to a requirement to under-expose the Green channel, which becomes more noisy. That would be bad news if you are in the business of flower photography (which is a serious branch of product photography in the Netherlands).
Since this post quantization multiplication doesn't add one iota of accuracy, I also wonder if there is another reason, e.g. such as improved compression (speed/efficiency).
Cheers,
Bart
-
There are at least two reason to pre-multiply that I can think of - firstly, it's normal to premultiply before demosaicing anyway; demosaicing engines tend to work a little better if grey is grey. Secondly, if you are going to level compress data (as Nikon do in compressed NEFs), it is better to use all the available levels for data in all three channels rather than only using all the levels in one channel, and having a lot of possible levels unused in the other two channels.
Sandy
-
There are at least two reason to pre-multiply that I can think of - firstly, it's normal to premultiply before demosaicing anyway; demosaicing engines tend to work a little better if grey is grey.
Hi Sandy,
That doesn't require one doing it in the Raw file data. It could even better be postponed to a more potent processing environment which can also use more bits.
Secondly, if you are going to level compress data (as Nikon do in compressed NEFs), it is better to use all the available levels for data in all three channels rather than only using all the levels in one channel, and having a lot of possible levels unused in the other two channels.
That's along the lines of what I was also thinking. More zeroes in the LSBs (least significant bits) allows to either compress more efficiently or drop LSBs alltogether.
Such mutilation of image data is not welcome in astrophotography circles, so the question becomes, can it be avoided or is this the best Raw data quality?
Cheers,
Bart
-
Hi Sandy,
That doesn't require one doing it in the Raw file data. It could even better be postponed to a more potent processing environment which can also use more bits.
That's along the lines of what I was also thinking. More zeroes in the LSBs (least significant bits) allows to either compress more efficiently or drop LSBs alltogether.
Such mutilation of image data is not welcome in astrophotography circles, so the question becomes, can it be avoided or is this the best Raw data quality?
Cheers,
Bart
Well, NEF dates back quite a bit now - back in those days, storage was precious, and it took a long tim to move bytes, so the trade-offs were were different.
But I'd be surprised if there was actually a reduction in data quality for an uncompressed image. Simplistically, if the original data ranged from 0-100, and it's now scaled from 0-1000, then yes, you would see "holes", in fact 90% of the data would be holes - it would go 0, 10, 20, 30,.... etc. But actually, you still have the same number of real levels as the sensor/converter measured.
Sandy
-
Hi,
Thanks for your inputs. I have checked more NEFs and it seems that those multipliers are not always the same, depending on camera settings. This might be performed to compensate slight nonlinearities and being able to use the same WB multipliers in post processing.
Regards,
Francisco
-
So the idea is to start with a 'white balanced' raw file.? And because there are twice as many green pixels, you have to double the blue and red pixel data? That should increase the amplitudes - but why are there holes or gaps in the histogram? That implies those channels start out with less dynamic range and are then scaled to higher and lower values- like what happens in post with contrast increase.
The gaps remind me of early versions of photoshop - when you looked at 8 bit jpgs there were all these gaps. But if it were just the software being used to look at 14 bit files in 16 bit space, the green channel should have gaps too.
I don't understand it either.
-
That implies those channels start out with less dynamic range and are then scaled to higher and lower values
Not necessarily less dynamic range; dynamic range depends on e.g., on noise, and noise might not be same in each channel although in practice it's often similar. But less resolution (aka levels), yes.
Sandy
-
At the risk of going off topic - I don't see how 'resolution' of the number of gray shades is that different than dynamic range. Although we think of dynamic range as the number of EV between darkest and lightest (the actual number of lumens a sensor can resolve as represented by the lowest and highest number), dynamic range can also be interpreted as the number of detectable shades between darkest and lightest. And this is the 'version' of dynamic range I am using.
-
At the risk of going off topic - I don't see how 'resolution' of the number of gray shades is that different than dynamic range. Although we think of dynamic range as the number of EV between darkest and lightest (the actual number of lumens a sensor can resolve as represented by the lowest and highest number), dynamic range can also be interpreted as the number of detectable shades between darkest and lightest. And this is the 'version' of dynamic range I am using.
It's a bit of an "how many angel can dance on a pinhead" debate, but "darkest" is mostly defined by noise level - below the noise floor, you don't know how dark dark is(!), and for most sensors, the noise floor is well above the lowest level that the A/D can measure.
Sandy
-
Hey Bart,
I believe that there are many reasons why data may be scaled before it is written to the raw file, most of which are camera/sensor dependent and we can only imagine (some of them were mentioned in this thread).
It is quite common for Red in flowers to saturate before the Green channel does.
'Quite' common for Red in flowers to saturate before Green - in the R* and G* raw data channels? Are you sure? Everything I know on this topic I learned from Iliah (thanks for the spell check :-) and my experience plus some of his posts have led me do believe that it is 'uncommon' in the situation mentioned in my post:
Other than in blown highlight situations where this becomes apparent, it is uncommon to find complete (i.e. not just a saturated detail) natural scenes where the Red or Blue raw channel clip before green, before or after scaling.
In red flower shots, it is on the other hand more common for the Red Channel to saturate before Green after typical rendering in a colorimetric color space such as sRGB.
Jack
-
I don't see how 'resolution' of the number of gray shades is that different than dynamic range. Although we think of dynamic range as the number of EV between darkest and lightest (the actual number of lumens a sensor can resolve as represented by the lowest and highest number), dynamic range can also be interpreted as the number of detectable shades between darkest and lightest. And this is the 'version' of dynamic range I am using.
You are probably interested in Dxomark's Tonal Range measurement (http://www.dxomark.com/index.php/About/In-depth-measurements/Measurements/Noise) then. However, that tells you more about the capabilities of the human visual system than the sensor's.
Jack
-
So the idea is to start with a 'white balanced' raw file.?
No, as far as I understand the idea is to start with a "raw" file that will require the same multipliers in post for White Balance regardless of the parameters you chosed in the camera.
As an example, shoting the same scene, if you select different Aperture/Speed/ISO combinations where all correspond to correct exposure, you should end up with a raw file that is pretty much the same. If this is not happening at "sensor dump" data level, then it is reasonable to do some correction by the camera firmware before writing the NEF file, at least for general photography.
but why are there holes or gaps in the histogram?
That is an evidence of multiplication in the digital domain (after conversion from analog). If the encoding is linear, as in the case of most cameras, multiplicating the raw values by a constant value is equivalent to exposure compensation. In this case these small compensations (we are talking in the order of 0.1 EV) are applied individually to each channels.
That is actually what most cameras do for high ISO, it is really nothing more than multiplying the raw values after conversion. The attached image shows a small region of the raw histogram from a image taken at ISO 3200 with a nikon D800.
-
If Nikon - or anyone else - are scaling raw channels, that is quite unfortunate, and - err - dumb.
That doesn't require one doing it in the Raw file data. It could even better be postponed to a more potent processing environment which can also use more bits.
Exactly.
While scaling Real Numbers is linear, scaling finite precision numbers is not.
To illustrate the problem, consider a 3-bit RAW camera, so the RAW format supports values 0-7, but where the sensor red channel is usually in the range 0-5. Let's try scaling the red range up to 0-7 and rounding to 3-bits.
Oh dear. The input has even steps, but the output doesn't.
Also, we have a problem if we take a picture where the sensor red channel goes up to 6.
If you have some very clever RAW conversion software that analyses the gaps in the RAW red histogram, in principle, that software could undo the damage done by scaling and rounding back to 0-5 - on the basis that a histogram with regularly spaced gaps in it was statistically implausible.
OTOH, if your RAW software takes this scaled RAW data at face value, the non-linearity introduced by the scaling may introduce banding in the shadows, and emphasise sensor pattern noise in the shadows, which might not otherwise be visible.
If the manufacturers want to offer more ISO settings than their hardware has gain settings, the way to do it is to record the requested gain in the metadata, rather than by attempting to scale the "RAW" component values.
-
If the manufacturers want to offer more ISO settings than their hardware has gain settings, the way to do it is to record the requested gain in the metadata, rather than by attempting to scale the "RAW" component values.
Hi,
I agree that that would be the preferred quality preserving route to take.
Therefore I assume it must be for reasons of compression efficiency, certainly not for quality.
Cheers,
Bart
-
To illustrate the problem, consider a 3-bit RAW camera
they do this exactly because the cameras are not 3-bit RAW cameras, so there are no degradation really detectable by their target customer... hence you want to illustrate the problem by producing 2 raw files and suggested parameters of conversion in a real raw converter of your choice.
-
Hello.
I assume it must be for reasons of compression efficiency, certainly not for quality.
I don't see low that would help compression efficiency. If I understand NEF - which I may not - the same (fixed) Huffman encoding is used for all channels.
-
Hello.
I don't see low that would help compression efficiency. If I understand NEF - which I may not - the same (fixed) Huffman encoding is used for all channels.
NEF can also use level compression.
Sandy
-
they do this exactly because the cameras are not 3-bit RAW cameras, so there are no degradation really detectable by their target customer... hence you want to illustrate the problem by producing 2 raw files and suggested parameters of conversion in a real raw converter of your choice.
No.
I illustrate the problem with a 3-bit RAW format to keep the numbers in the illustration manageable.
Certainly, when we get to the mid-tones, the relative error caused in the OP's example by scaling followed by re-quantisation is unlikely to be significant or noticeable.
But the re-quantisation errors might become a real problem if we start to try to move the tone curve towards the shadows - increasing the gain in the shadows.
It seems like an unforced error if Nikon is doing that.
-
Then we could look at the whole approach to RAW compression.
Over the years, the number of logic gates that an IC designer can throw at a problem has increased exponentially, while memory bandwidth has increased much more slowly. So throwing more logic gates at compression makes sense, and folk like Luminous Landscape should be beating up Canon, Nikon et al. about why this isn't happening.
The size of RAW images is getting to be a problem, limiting [sustained] camera frame rates, and increasing storage/backup costs.
A predictive code for RAW images would let us store images with greater precision in less space, but that doesn't seem to be happening. You folk with D800's should be asking Nikon why, given the state of the art - say, five years ago - in image compression, your RAW files are so very very big.
Then I have a 10MP Samsung EX1, which I like because of its flip-and-twist display [and the fast-ish lens]. As MR put it a long time ago, it has trouble walking and chewing gum at the same time. Never mind compression, it features RAW expansion! A 10MP RAW takes > 20MBytes!
-
Hello.
I don't see low that would help compression efficiency. If I understand NEF - which I may not - the same (fixed) Huffman encoding is used for all channels.
Hi,
Even if Huffman encoding is used (I don't know if it is), having a number of LSB zeroes will compress nicely, and gaps at the same bit position in the represented numbers also help compression, as does having fewer zeroes spread in the MSBits.
IOW it helps to avoid the issues seen when compressing 16-bit numbers, where compression may cause the file size may grow!
Cheers,
Bart
-
A predictive code for RAW images would let us store images with greater precision in less space, but that doesn't seem to be happening. You folk with D800's should be asking Nikon why, given the state of the art - say, five years ago - in image compression, your RAW files are so very very big.
They are only very very big if one doesn't use Nikon's Compressed modes (http://www.openphotographyforums.com/forums/showthread.php?t=5499): about one byte per pixel, not bad for 14 bit data: the OOC 8 bit TIFF is about three times as large. However, if folks are squeamish about using even this type of conservative-though-perfect information recording (they are, oh they are), I can just imagine what it would take to convince them to use anything even remotely more aggressive...
Then I have a 10MP Samsung EX1, which I like because of its flip-and-twist display [and the fast-ish lens]. As MR put it a long time ago, it has trouble walking and chewing gum at the same time. Never mind compression, it features RAW expansion! A 10MP RAW takes > 20MBytes!
That would be about 10 x 14 / 8 = 17.5MB, plus a little overhead... yeah, no compression here ;)
Jack
-
Embedded JPEG in the Samsung?
Now if only there were a way to disable the storing of embedded JPEGs in NEFs. If I wanted JPEGs, I'd be using NEF+JPEG! Could reduce storage costs (and processing speed/copying time) by about 10-20%!
-
To see why the preconditioning is included in the flow, you just have to follow the signal flow in the Milbaut Expeed.
Some parts of the signal conditioning is made as the signal from the sensor FIRST streams into the logic unit, basically all parts that are supposed to go into the raw file type of choice (14, 14LC, 14C, 12, 12LC, 12C).
Other parts aren't applied to the signal/image until after the entire image has been one round through the buffer memory - like dark-frame subtraction (long exposure NR), jpg conversion settings, jpg-compression and so on.
To use the Compressed format, the channels need to be aligned to their respective maximum before the gamma-like curve is applied to avoid as much of the unavoidable gradation loss as possible. This means that the multiplication has to be placed before the compression choices in the raw in-flow before the first buffering. Remember that this are hardcoded signal flows.
When not in Compressed raw mode, the conditioning could theoretically be turned off - but then you'd have the peculiar problem that your WB multipliers would be different depending on if you chose Compressed or uncompresssed / LC raw files. So keep it in. It's not like they (the holes in the histogram) actually do any visible harm to the image quality. In theory they might, in practice they don't.