DxO PhotoLab 2 working space tests

[Jack Hogan]

To be formal, when discussing color folks talk about 'spaces' often without realizing/remembering that the terminology is that of linear algebra.  Conversion from one space to another is accomplished by multiplication by a 3x3 or 3x4 matrix.  A neat insight into the process is understanding that the matrix in effect gives a different point of view of the same solid in 3D space.  So it is always the same color 3D object but seen from different perspectives.

If we can shift the point of view of the captured raw data cube to that which gives us what we call an XYZ projection, and from there to sRGB's, we can just as easily shift the point of view back to XYZ's, then to Adobe RGB's, then to ProPhoto's, etc. indefinitely, without ever touching the raw data solid, which is what it is, what it was and what it will be.  All captured 'colors' are present all of the time.  They may be more or less squished or stretched but they are there nonetheless.

All these transformations are accomplished by floating point linear matrices that look like this.  Because they are linear, they are 100% reversible.  In other words, as long as you stick with floating point math, there will be no/zero/zilch practical penalty for moving between and/or working in any of these spaces.  Limitations are only introduced when working with unsigned integers at lower bit depths (say 8 bits).

So given the fact that if one sticks with floating point one can effectively use any color space one wishes without any penalty of any kind whatsoever, what color space should one use in practice in such a case?  What I do as a matter of course is to work with 14+ bit data and choose a color space that closely matches the output device that I am viewing the photo with while rendering, so that I will see what I am doing.  My monitor does close to 100% Adobe RGB, so that's the working space I typically use*.

But that's me, to each their own of course.

Jack

* This doesn't stop me from using larger color spaces on some images that I want printed, although my experience is that in such cases more than one trip to the printer is often necessary before a satisfactory result is obtained.

[32BT]

((1/255)^2.2)x65535 = 0.3327...

Which tells us that 16bits are not enough to represent 8bit perceptual steps in 16bit linear space.

In this case we seem to need at least 2 more bits to discern the first step. If you subsequently want to allow out-of-gamut, you'd want two additional bits. One for negative values, one for overshoot. That's 20bits. If you then want to store intermediate values during multiplications (in registers for example), you'd need at least 40bits. That would be the absolute minimum required precision.

If you want to be able to discern a full 8bits in linear space for each step in 8bit perceptual space, it would require 28 and 56 bits. Doing the same for gamma 2.4 is left as an exercise to the reader.

Fortunately, these days everything is 64bit or higher, so the exercise is kind of moot but perhaps insightfull.

[Jack Hogan]

((1/255)^2.2)x65535 = 0.3327...

Which tells us that 16bits are not enough to represent 8bit perceptual steps in 16bit linear space.

In this case we seem to need at least 2 more bits to discern the first step. If you subsequently want to allow out-of-gamut, you'd want two additional bits. One for negative values, one for overshoot. That's 20bits. If you then want to store intermediate values during multiplications (in registers for example), you'd need at least 40bits. That would be the absolute minimum required precision.

If you want to be able to discern a full 8bits in linear space for each step in 8bit perceptual space, it would require 28 and 56 bits. Doing the same for gamma 2.4 is left as an exercise to the reader.

I am not sure I follow.  Where do your comments fit in the following simplified sequence?

(0) scene -> (1) analog linear intensity -> (2) linear 14-bit raw -> (3) linear 16-bit RGB -> (4) gamma (1/2.2) 8-bit RGB -> (5) de-gamma (2.2) 8-bit linear intensity -> (6) analog linear intensity -> (7) eye

[32BT]

I am not sure I follow.  Where do your comments fit in the following simplified sequence?

(0) scene -> (1) analog linear intensity -> (2) linear 14-bit raw -> (3) linear 16-bit RGB -> (4) gamma (1/2.2) 8-bit RGB -> (5) de-gamma (2.2) 8-bit linear intensity -> (6) analog linear intensity -> (7) eye

Between 3 and 4, especially where display is concerned, and in general colorspace conversions that require compression. (Out-of-gamut correction for example, or a smaller space stuffed in a larger space and vv.)

[digitaldog]

Going back OT on why a wide gamut color working space is useful, (and I've provided this elsewhere in the past):

One reason we need big RGB working spaces is that they are based on theoretical emissive devices (ProPhoto RGB being very theoretical when you look at what falls outside human vision). Necessary because of their simple and predicable shapes. So while there are many more colors that can be defined in something like ProPhoto RGB than you could possibly print, we have to deal with a significant disconnect between these simple shapes of RGB working space and the vastly more complex shapes of an output color space. Simple RGB working space matrix profiles when plotted 3 dimensionally illustrate that they reach their maximum Chroma at high luminance levels which makes sense since they are based on increased Chroma by the addition of more light. The opposite is seen with print (output) color spaces where this is accomplished by adding ink: a subtractive color model. One reason we need such big RGB working space like ProPhoto RGB is due to its simple size and to counter the disconnect between mapping to the output space without potentially clipping colors. There can be issues where very dark colors of intense Chroma (which do occur in nature and we can capture with many devices) don’t map properly with a smaller working space. Many of these darker colors fall outside Adobe RGB (1998). When you encode using a smaller color space, you clip the colors to the degree that smooth gradations become solid blobs in print, again due to the dissimilar shapes and differences in how the two spaces relate to luminance. I suspect this is why Adobe picked ProPhoto RGB primaries for the processing color space in their raw converters.

Shown here:
http://www.digitaldog.net/files/sRGBvsPro3DPlot_Granger.tif

[Jack Hogan]

(0) scene -> (1) analog linear intensity -> (2) linear 14-bit raw -> (3) linear 16-bit RGB -> (4) gamma (1/2.2) 8-bit RGB -> (5) de-gamma (2.2) 8-bit linear intensity -> (6) analog linear intensity -> (7) eye

Between 3 and 4, especially where display is concerned, and in general colorspace conversions that require compression. (Out-of-gamut correction for example, or a smaller space stuffed in a larger space and vv.)

I see, in that case I believe the first non-zero value in a 16-bit linear RGB space would correspond to

(1/65535)^(1/2.2)*255 = 1.65

in the same 8-bit gamma-encoded RGB space.  Right?

Jack

[32BT]

I see, in that case I believe the first non-zero value in a 16-bit linear RGB space would correspond to

(1/65535)^(1/2.2)*255 = 1.65

in the same 8-bit gamma-encoded RGB space.  Right?

Jack

Correct, where greyscale is concerned. May be aggrevated by: additional lut correction for display calibration, or, for example, smaller source space than destination space.

EDIT: or larger source space than destination space. (Camera profiles may have virtual primaries for example).

[fdisilvestro]

((1/255)^2.2)x65535 = 0.3327...

Which tells us that 16bits are not enough to represent 8bit perceptual steps in 16bit linear space.

It would be great if more people were aware of this. I have seen many discussion around bit depth in online forums where the participants don't have a clue of the difference between linear space and perceptual or gamma encoded.

Fortunately, these days everything is 64bit or higher, so the exercise is kind of moot but perhaps insightfull.

Are you sure current image editing software are using 64 bit for the math?

[Jack Hogan]

It would be great if more people were aware of this.

That would be unnecessary as such a situation does not typically occur with decent software thanks to the linear gamma toe used by well behaved color spaces and engines (including ACE).

[Jack Hogan]

Are you sure current image editing software are using 64 bit for the math?

None use 64-bit integer arithmetic that I know of.  PS uses 8 bit unsigned and 16 bit signed, with a fair bit of FP thrown in here and there. Others here know these details better than I.

[digitaldog]

PS uses 8 bit unsigned and 16 bit signed, with a fair bit of FP thrown in here and there. Others here know the details better than I.

From: Marc Pawliger
Subject: RE: 16 bit and the info palette
Message:
The high-bit representation in Photoshop has always been "15   1" bits (32767 (which is the total number of values that can be represented by 15 bits of precision)   1).  This requires 16 bits of data to represent is called "16 bit".  It is not an arbitrary decision on how to display this data, it is displaying an exact representation of the exact data Photoshop is using, just as 0-255 is displayed for 8 bit files.

[Doug Gray]

From: Marc Pawliger
Subject: RE: 16 bit and the info palette
Message:
The high-bit representation in Photoshop has always been "15   1" bits (32767 (which is the total number of values that can be represented by 15 bits of precision)   1).  This requires 16 bits of data to represent is called "16 bit".  It is not an arbitrary decision on how to display this data, it is displaying an exact representation of the exact data Photoshop is using, just as 0-255 is displayed for 8 bit files.

I'm pretty sure, and your quote from Marc is consistent with this,  that the reason PS does this is that it allows 16 bit math using signed 16 bit ints. In the past this was much faster than floating point and there is a lot of really old, legacy code in PS. The loss of a bit using 15 instead of 16 bits (which would require unsigned like 8 bit RGB) has no material impact on color. In pretty much any colorspace the largest dE errors from the dropped bit would be under .02

[Doug Gray]

((1/255)^2.2)x65535 = 0.3327...

Which tells us that 16bits are not enough to represent 8bit perceptual steps in 16bit linear space.

This would be true if RGB(1,1,1) was discernibly different from RGB(0,0,0). It's not. The L* difference between the two is .0046 and deltaE2000 is even lower at .003. It isn't perceptible and by a very large factor.

BTW, this is why some color spaces like sRGB have a linear ramp at the front. Pure gamma encoded spaces waste bits on the front end as they produce tiny changes. sRGB uses a gamma of 2.4 when past the linear ramp.

In this case we seem to need at least 2 more bits to discern the first step. If you subsequently want to allow out-of-gamut, you'd want two additional bits. One for negative values, one for overshoot. That's 20bits. If you then want to store intermediate values during multiplications (in registers for example), you'd need at least 40bits. That would be the absolute minimum required precision.

If you want to be able to discern a full 8bits in linear space for each step in 8bit perceptual space, it would require 28 and 56 bits. Doing the same for gamma 2.4 is left as an exercise to the reader.

Fortunately, these days everything is 64bit or higher, so the exercise is kind of moot but perhaps insightfull.

This just is not so.

[32BT]

This would be true if RGB(1,1,1) was discernibly different from RGB(0,0,0). It's not. The L* difference between the two is .0046 and deltaE2000 is even lower at .003. It isn't perceptible and by a very large factor.

The problem is like this:

16bit linear = 1 results in 8bit perceptual = 2

(2/255) = 0.78 which, considered as a delta E, is most definitely visible in a grayscale. You'd see posterisation in your blacks. For grayscale we used a maximum of delta E = 0.5 as limit, but a trained eye, I'm certain, would be able to notice trouble below that threshold since it results in structured errors. Dithering is necessary to overcome part of the problem, but then you obviously shouldn't map a lot of 16bit values into a single 8bit bucket first.

[32BT]

BTW, this is why some color spaces like sRGB have a linear ramp at the front. Pure gamma encoded spaces waste bits on the front end as they produce tiny changes. sRGB uses a gamma of 2.4 when past the linear ramp.

It's very important to understand that this is pure nonsense. The pure gamma encoding simply dictates a single precise number for the gamma. How the engine actually uses that number to do its conversions and whether they waste bits in incorrect lookup table implementations is entirely a fault of the engine, not of the space.

Adding actual lookup tables within the space definition to overcome incorrect computations under the hood of some engines, has just been a really bad workaround.

[Doug Gray]

The problem is like this:

16bit linear = 1 results in 8bit perceptual = 2

It isn't clear exactl6y what you are saying. My interpretation is that you are saying that 1 (in 16 bit linear space) is about 2 in 8 bit RGB encoded with gamma=2.2. This is approximately right.  But are they perceivably different? No.

Let's take the two cases, first a 16 bit linear space RGB of (1,1,1). The DeltaE1976 between RGB(0,0,0) and RGB(1,1,1) in 16 bit linear space is .0135. dE2000 is a bit smaller.

Now for the 2.2 gamma encoded 8 bit RGB space. The deltaE76 between RGB(0,0,0) and RGB(2,2,2) is .021. This is still an order of magnitude below your perceivability threshold of .5 dE.

Lastly, looking at the deltaE76 between RGB(1,1,1) in linear space, and it's converted cousin in gamma=2.2, 8 bit space we get a value of .008, so the impact of conversion at the low end is negligible.

(2/255) = 0.78 which, considered as a delta E, is most definitely visible in a grayscale. You'd see posterisation in your blacks. For grayscale we used a maximum of delta E = 0.5 as limit, but a trained eye, I'm certain, would be able to notice trouble below that threshold since it results in structured errors. Dithering is necessary to overcome part of the problem, but then you obviously shouldn't map a lot of 16bit values into a single 8bit bucket first.

.78 certainly isn't anywhere near representing a deltaE. What exactly do you mean by that?

[Doug Gray]

It's very important to understand that this is pure nonsense. The pure gamma encoding simply dictates a single precise number for the gamma. How the engine actually uses that number to do its conversions and whether they waste bits in incorrect lookup table implementations is entirely a fault of the engine, not of the space.

Adding actual lookup tables within the space definition to overcome incorrect computations under the hood of some engines, has just been a really bad workaround.

Are you aware that the encoding algorithm for sRGB, unlike Adobe RGB,  isn't a pure gamma function? I am not talking about whatever engine is calculating transforms, but the spec itself.

Here's why pure gamma 2.2 encoding wastes bits on the low side. And let's compare it to sRGB for grins. 8 bit RGB all.
Note the difference between L*=0 or L*=100, as appropriate, is exactly the deltaE1976

sRGB (1,1,1)  L*=.27
sRGB(254,254,254)  L*=99.65

AdobeRGB (1,1,1) L*=.0046
AdobeRGB (254,254,254) L*=99.67

You have to go all the way up to RGB(6,6,6) on gamma = 2.2 to get an L* of .24. Thus values between 1 and 6 effectively do nothing. The ramp on the front end of sRGB, which is in the sRGB specification, allows these to provide more meaningful steps and also allows the rest of the curve to be gamma=2.4 which is somewhat better perceptually than 2.2.

[digitaldog]

Indeed, sRGB is using a TRC, not a gamma curve. I don't know if any other 'common' RGB working spaces do this (certainly not ColorMatch RGB, Adobe RGB (1998) or ProPhoto RGB).

[32BT]

It isn't clear exactl6y what you are saying. My interpretation is that you are saying that 1 (in 16 bit linear space) is about 2 in 8 bit RGB encoded with gamma=2.2. This is approximately right.  But are they perceivably different? No.

Let's take the two cases, first a 16 bit linear space RGB of (1,1,1). The DeltaE1976 between RGB(0,0,0) and RGB(1,1,1) in 16 bit linear space is .0135. dE2000 is a bit smaller.

Now for the 2.2 gamma encoded 8 bit RGB space. The deltaE76 between RGB(0,0,0) and RGB(2,2,2) is .021. This is still an order of magnitude below your perceivability threshold of .5 dE.

Lastly, looking at the deltaE76 between RGB(1,1,1) in linear space, and it's converted cousin in gamma=2.2, 8 bit space we get a value of .008, so the impact of conversion at the low end is negligible.
.78 certainly isn't anywhere near representing a deltaE. What exactly do you mean by that?

No, you can safely assume that the gamma 2.2 space is more or less perceptually uniform, since that is the exact idea behind gamma encoding. So, the first step in 8bit perceptual space = (2/255) = .78% thus about equivalent to a deltaE76 of .78

Please don't go overboard on the linear part of that L curve. First of all that RGB = 0 doesn't start in pure black, second that linear part was not about dark pixels in a sea of white for example.

All numbers aside: i've implemented a colormanagement engine for display purposes long ago which allowed adjustable out-of-gamut rendering. Digidog may remember, i've send him an example. I vaguely recall that too little bits gave visible posterisation up to the first  4 or 6 steps. And that's just allowing gamma 2.2, imagine allowing anything up to 3.0 or whatever photoshop allows one to save these days...

[32BT]

Are you aware that the encoding algorithm for sRGB, unlike Adobe RGB,  isn't a pure gamma function? I am not talking about whatever engine is calculating transforms, but the spec itself.

Here's why pure gamma 2.2 encoding wastes bits on the low side. And let's compare it to sRGB for grins. 8 bit RGB all.
Note the difference between L*=0 or L*=100, as appropriate, is exactly the deltaE1976

sRGB (1,1,1)  L*=.27
sRGB(254,254,254)  L*=99.65

AdobeRGB (1,1,1) L*=.0046
AdobeRGB (254,254,254) L*=99.67

You have to go all the way up to RGB(6,6,6) on gamma = 2.2 to get an L* of .24. Thus values between 1 and 6 effectively do nothing. The ramp on the front end of sRGB, which is in the sRGB specification, allows these to provide more meaningful steps and also allows the rest of the curve to be gamma=2.4 which is somewhat better perceptually than 2.2.

I don't quite understand how you get to your values. The first step in a gamma encoded space should be brighter than L since they don't have the flat bit like L has. (i.e. the derivative in 0 is infinite for gamma spaces, not so for L, therefore the deltaE should always be larger than the Y <-> L relation.)