Pages: [1]   Go Down

Author Topic: Display calibration puzzle  (Read 1827 times)

rasworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 473
Display calibration puzzle
« on: December 24, 2017, 12:09:36 pm »

I use the included BenQ software, Palette Master Element, to calibrate and profile my SW2700PT display.  I assumed the software would set the 14 bit internal monitor RGB channel LUTS so that no vcgt tag would be necessary for the video adapter LUT, i.e. the adapter would have a non-modified straight line characteristic as shown by the first attachment.

However I have found after several passes, using various white point, luminance, etc. settings, the output profile does contain a vcgt tag, and it always contains the values shown in the second attachment.  Each output value is n x 257, instead of n x 256 as for the non-modified case.  This carries all the way through the table, so that n = 255 results in 255 x 257 = 65535 instead of 255 x 256 = 65280.

I'm puzzled as to why the vendor did this - as far as I can tell it is equivalent to a non-modified table, in that my video adapter lut is 8 bits, and I'm assuming only the high order 8 bits of the output value are used, so that the end result is a straight line transfer function.

Anybody have a theory?  BTW the resulting profiles seem to be completely satisfactory, although it took four revisions of the software to get out all the kinks.

Richard Southworth

Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: Display calibration puzzle
« Reply #1 on: December 24, 2017, 06:08:42 pm »

However I have found after several passes, using various white point, luminance, etc. settings, the output profile does contain a vcgt tag, Each output value is n x 257, instead of n x 256 as for the non-modified case.  This carries all the way through the table, so that n = 255 results in 255 x 257 = 65535 instead of 255 x 256 = 65280.
x 257 is correct - this expands values fully to 16 bits.
x 256 is incorrect - the full white output is not reached for VideoLUTs over 8 bits/channel in depth.
Logged

rasworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 473
Re: Display calibration puzzle
« Reply #2 on: December 25, 2017, 11:41:15 am »

But I don't see how the extended version would apply to video LUTs over 8 bits in depth.  Using dot notation, the BenQ vcgt for inputs 0, 1, 2, 3, 4,...,255 gives outputs 0.0, 1.1, 2.2, 3.3, 4.4,...,255.255.  I assume for a 8 bit LUT the lower 8 bits, i.e. to the right of the ".", are ignored.

A deeper LUT requires a completely different sequence and therefore a differently constructed vcgt; e.g. a 10 bit LUT for linear transfer would have inputs 0, 1, 2, 3, 4,...,1023, with corresponding outputs 0.0, 0.64, 0.128, 0.192, 1.0,...,255.192.

So again I don't see any utility to their vcgt construction as opposed to the reset condition, because I don't see any way that a 8 bit vcgt tag can be used for a 10 bit (or larger) LUT.

Richard Southworth
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: Display calibration puzzle
« Reply #3 on: December 25, 2017, 06:30:05 pm »

But I don't see how the extended version would apply to video LUTs over 8 bits in depth.
Just do the math. i.e.

 Say we've got 256 entries of 16 bit in the API, and the video driver then loads the ms bits into the HW
(i.e. the way that all the major operating system API's work).

If we take the entry index and multiply by 256 we get:

   0 -> 0x0000
   1 -> 0x0100
   ..
   254 -> 0xfe00
   255 -> 0xff00

Say the VideoLUT entry size is 10 bits, that means that the full white entry gets loaded with 0x3fc rather than it's maximum value of 0x3ff.

If instead we multiply by 257 (or the simpler equivalent which is to duplicate the top 8 bits into the bottom 8 bits) we get:

   0 -> 0x0000
   1 -> 0x0101
   ..
   254 -> 0xfefe
   255 -> 0xffff

and the last entry gets loaded with 0x3ff, as it should be.
Logged

rasworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 473
Re: Display calibration puzzle
« Reply #4 on: December 26, 2017, 10:24:53 am »

Yes, I've done the math, and I understand what happens for 0..255, but for a 10 bit LUT we need to have entries for 0...1023, not 0...255.  Does the vcgt provide every fourth entry, and it's up to the system to fill in the other three values?  Still doesn't compute to me.

Richard Southworth

Added by edit - I'm assuming a video adapter with a 10 bit LUT requires 1024 entries.
« Last Edit: December 26, 2017, 10:40:30 am by rasworth »
Logged

GWGill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
  • Author of ArgyllCMS & ArgyllPRO ColorMeter
    • ArgyllCMS
Re: Display calibration puzzle
« Reply #5 on: December 26, 2017, 05:48:22 pm »

Yes, I've done the math, and I understand what happens for 0..255, but for a 10 bit LUT we need to have entries for 0...1023, not 0...255.  Does the vcgt provide every fourth entry, and it's up to the system to fill in the other three values?  Still doesn't compute to me.
A graphics card Video LUT maps the frame buffer output to the video output. So the number of entries is determined by the frame buffer depth (i.e. 8 bits/chan), and the entry size is determined by the video path depth (i.e. 8 - 12 bits/chan).
Yes, software that loads the hardware from an ICC vcgt will re-sample appropriately to match the hardware.
« Last Edit: December 26, 2017, 05:51:56 pm by GWGill »
Logged
Pages: [1]   Go Up