DSLR testing sites like DXOmark and Imaging Resource use HMI and LEDs for color

[daicehawk]

The DXOmark use a LED lightbox, the Imaging Resource HMI bulbs for the "skylight" component in the "sunlit" tests, which IMO renders the results invalid and defies the purpose  of the color accuracy measurement. Any peaky spectrum is a NO-NO in my book no matter the CRI score.
What do you guys think?

[digitaldog]

Absolutely agree. But the only light source that produces a standard illuminate is 93 million miles away. Using a man made source, the quality and matching differ. CRI is kind of a hack too. CQS (15 very colorful patches) a bit better. That doesn't tell us about the spectrum which is even a better way to evaluate the illuminant.

[daicehawk]

I see no problem with any continuous spectrum be it the sun or a man-made filtered halogen (though very inefficient and hot). The sun\sky light balance varies a lot during the day and across the seasons with the spectrum continuity being the main attribute of the high CRI no matter the test sample set reflectance spectrum.
The Blue-Yellow LEDs lack the cyan range, the HMI are just a nightmare and both must be tweaked to get a high CRI. I think it is a shame the QDots +LEDs solution has not been implemented.

[Jim Kasson]

The DXOmark use a LED lightbox, the Imaging Resource HMI bulbs for the "skylight" component in the "sunlit" tests, which IMO renders the results invalid and defies the purpose of the color accuracy measurement. Any peaky spectrum is a NO-NO in my book no matter the CRI rate.
What do you guys think?

Tests run with a particular illuminant and a particular target patch set are useful in determining the camera's response to that patch set under that illuminant. That is the high road.

It is thought that tests run with illuminants and patch sets with broad smooth spectra generalize to other broad-spectrum illuminants and other broad-spectrum patch sets. That is usually the case, but spiky camera response can throw that for a loop.

jim

[daicehawk]

Tests run with a particular illuminant and a particular target patch set are useful in determining the camera's response to that patch set under that illuminant. That is the high road.

It is thought that tests run with illuminants and patch sets with broad smooth spectra generalize to other broad-spectrum illuminants and other broad-spectrum patch sets. That is usually the case, but spiky camera response can throw that for a loop.

jim

Those are obvious things, the point of my post being such test lights invalidate any judgments about the camera response and it degree of conformity to the Luther-Ives condition.

[digitaldog]

Those are obvious things, the point of my post being such test lights invalidate any judgments about the camera response and it degree of conformity to the Luther-Ives condition.

What camera conforms to the Luther-Ives condition?

[daicehawk]

What camera conforms to the Luther-Ives condition?

That depends on the acceptance threshold and selected preferences on what colors are more important to be reproduced accurately. Please read "its degree of conformity" and do not start an argument.

[digitaldog]

That depends on the acceptance threshold and selected preferences on what colors are more important to be reproduced accurately. Please read "its degree of conformity" and do not start an argument.

So you can’t answer my question.

[Jim Kasson]

That depends on the acceptance threshold and selected preferences on what colors are more important to be reproduced accurately. Please read "its degree of conformity" and do not start an argument.

Is 'degree of conformity' a scalar, and if so, how is it defined?

Jim

[Doug Gray]

Those are obvious things, the point of my post being such test lights invalidate any judgments about the camera response and it degree of conformity to the Luther-Ives condition.

Even with a perfectly smooth, black body spectra, how well a camera can track a ColorChecker is only imperfectly correlated with how well it meets LI. A hypothetical camera and a perfect black body source, might be great with a ColorChecker and not so good with other colors, even daylight metamers of ColorChecker colors. To really characterize a camera system against LI, the gold standard is to measure its post CFA spectral responses.

But that's hard.

Of course spikey illuminants only add uncertainty. It would be interesting to compare the two sources of error.

[digitaldog]

Is 'degree of conformity' a scalar, and if so, how is it defined?

Jim

That’s an easy one Jim. It’s a scale from 1 to 10 with 10 being best. 😀

[daicehawk]

Is 'degree of conformity' a scalar, and if so, how is it defined?

Jim

I define this as "a wide color range scene including memory colors such as the sky, skin, sea water, foliage, fruits etc. shot under sunlight looks good and real on a wide gamut display merely developed with a matrix profile without having to resort to selective color correction in PS" 

[GWGill]

CQS (15 very colorful patches) a bit better.

The CRI replacement that is gaining the most interest lately is IES TM-30-15.

[daicehawk]

Even with a perfectly smooth, black body spectra, how well a camera can track a ColorChecker is only imperfectly correlated with how well it meets LI. A hypothetical camera and a perfect black body source, might be great with a ColorChecker and not so good with other colors, even daylight metamers of ColorChecker colors. To really characterize a camera system against LI, the gold standard is to measure its post CFA spectral responses.

But that's hard.

Of course spikey illuminants only add uncertainty. It would be interesting to compare the two sources of error.

The colorchecker is good in representing memory colors in a small sample set. BTW you don`t have many "daylight metamers" for orange, yellow, red and aqua in the nature. Highly saturated reflected colors are less prone to metamerism.
100% LI conformity is not the goal, the "acceptable" color accuracy is. Perfect is the enemy of good. 

[digitaldog]

Jim, was that an acceptable answer to your question?
My one question wasn't acceptably answered.
I think perhaps other than a very interesting post by GWGILL (thanks, I'll certainly look into that), this is a discussion better suited for the "Open Forums" over on DP Review where I will not enter. 

[joofa]

What camera conforms to the Luther-Ives condition?

This one.

[digitaldog]

This one.

I still have want bumps for the RX10 

[Tim Lookingbill]

The CRI replacement that is gaining the most interest lately is IES TM-30-15.

The posted video in that link is the best I've seen explaining these issues with various manufactured lighting. It made total sense on how it will be useful for me.

[Jim Kasson]

Jim, was that an acceptable answer to your question?
My one question wasn't acceptably answered.
I think perhaps other than a very interesting post by GWGILL (thanks, I'll certainly look into that), this is a discussion better suited for the "Open Forums" over on DP Review where I will not enter. 

This one: "That’s an easy one Jim. It’s a scale from 1 to 10 with 10 being best."?

 I was hoping for something objective.

Jim

[Jim Kasson]

I define this as "a wide color range scene including memory colors such as the sky, skin, sea water, foliage, fruits etc. shot under sunlight looks good and real on a wide gamut display merely developed with a matrix profile without having to resort to selective color correction in PS"

I was asking for an objective way to make that measurement. "Looks good and real" sounds subjective to me. I might think that camera A is better than camera B by that test, and you might think the reverse. Besides, you left a lot of wiggle room in the criteria for selecting the compromise matrix.

Jim