D810 support?

[Bob Rockefeller]

Or another possibility: Nikon and Canon and friends just allow any 3rd party converter company to sign an NDA, get a raw file a few weeks prior to releases. On release day, the converters understand the new data.

This makes the most sense to me. It allows the camera manufactures to evolve and improve their formats, and even add new features not covered by DNG. And it provides early purchasers the 1st class, optimized, raw support they need.

I, for one, am contemplating a D810 to "replace" my D3, but will wait until C1 has support.

[digitaldog]

My point is simple if the manufacturer, the software developer, and the user adopt a particular standard of producing raw data. Then there would be no need for any of them to make further development of their individual process and further research and development. This will not happen.

Are you sure? (I'm honestly asking as I've got no idea, I just use the data). Would producing a standard means of understanding the data force a lack of further development?
Couldn't they still produce proprietary metadata that only their converters could utilize as they do today?

[Bart_van_der_Wolf]

I, for one, am contemplating a D810 to "replace" my D3, but will wait until C1 has support.

Hi Bob,

I don't understand that. Your new D810 will presumably allow to take better images than your current setup. It will do that from day 1. The Raw files can be converted by a number of alternative Raw converters, including the one that comes with the camera (I assume). And you can at a later date still revisit the Raws, they are not going away, but at least you have got them, rather that having missed the opportunity to shoot them.

The Raw files will still be there when you unleash their full potential when your trusted Raw converter also supports a good conversion. That support has mainly to do with color calibration, which I'd rather like having done well, rather than mediocre.

Cheers,
Bart

[Bob Rockefeller]

Are you sure? (I'm honestly asking as I've got no idea, I just use the data). Would producing a standard means of understanding the data force a lack of further development?
Couldn't they still produce proprietary metadata that only their converters could utilize as they do today?

And I don't know either. But could different sensors (Bayer, Foveon, X-Trans) somehow be shoehorned into a single format? And what if someone develops yet another sensor technology? What then?

I would hate to limit manufactures.

But I still agree with the initial point: the manufactures should make it easier for their customers to get at the data. Surely they know that few people use their proprietary RAW conversion tools, preferring Lightroom, Aperture, Capture One, Iridient Developer, etc. instead. My opinion is that the manufacturer publishing their formats, in advance of their cameras' releases, perhaps under NDA, to the RAW converter developers seems reasonable.

[Iliah]

Dear Francisco,
> I was able to open the NEF available from Imaging Resource in RawDigger and RawTherapee by editing the camera model (to D800E) using a hex editor.

I think with RawDigger you do not need to edit the camera model, but depending on the version of RD you are running you may need to set black level to manual.

[Bob Rockefeller]

I don't understand that. Your new D810 will presumably allow to take better images than your current setup. It will do that from day 1. The Raw files can be converted by a number of alternative Raw converters, including the one that comes with the camera (I assume). And you can at a later date still revisit the Raws, they are not going away, but at least you have got them, rather that having missed the opportunity to shoot them.

The Raw files will still be there when you unleash their full potential when your trusted Raw converter also supports a good conversion. That support has mainly to do with color calibration, which I'd rather like having done well, rather than mediocre.

I suppose you're right. And since I was in a Best Buy this morning that actually had several D810s in stock, I got one. Now I'll wait impatiently for the C1 update that covers it.

[Iliah]

> there IS a difference such that all the raw converters that were not updated for D810 can't access the raw data
Not in the case D800 vs. D810. Raw format produced by those cameras is the same. "Prevention" is artificial, and that is why hex editor or ExifTool trick does work.

[digitaldog]

Not in the case D800 vs. D810. Raw format produced by those cameras is the same.

That's encouraging. So presumably when the newer of the two was released, did the existing raw converters simply work without any further updates?

[Iliah]

That's encouraging. So presumably when the newer of the two was released, did the existing raw converters simply work without any further updates?

Depends on how full the format support is implemented. If the raw converter is reading black level from the raw file, should be OK; especially with custom-made colour profile. If it relies on built-in tables, like dcraw for Nikon stating black level as 596, and the table is not yet in place, it is a problem (D800 has black level at zero).

[digitaldog]

Depends on how full the format support is implemented. If the raw converter is reading black level from the raw file, should be OK; especially with custom-made colour profile. If it relies on built-in tables, like dcraw for Nikon stating black level as 596, and the table is not yet in place, it is a problem (D800 has black level at zero).

Sorry, I'm still a bit confused. If the two raws are the same, why would the converter not treat them equally? The table that is or isn't in place is where?

[Iliah]

Sorry, I'm still a bit confused. If the two raws are the same, why would the converter not treat them equally? The table that is or isn't in place is where?

Raw converter should treat them equally, but equally _right_, meaning it needs to read the black level tag from makernote. If instead it uses internal converter tables, the update of the converter or a hack is needed. The table is not the right approach, because the black level is not a constant and depending on the camera settings and sensor temperature it changes.

[fdisilvestro]

Dear Francisco,
> I was able to open the NEF available from Imaging Resource in RawDigger and RawTherapee by editing the camera model (to D800E) using a hex editor.

I think with RawDigger you do not need to edit the camera model, but depending on the version of RD you are running you may need to set black level to manual.

You are right! I stand corrected.

[ErikKaffehr]

Hi,

I have no insight in raw developers, but I presume that they use a colour conversion matrix to convert from sensor RGBG values to internal working space. So, that colour conversion matrix is needed my understanding is that Adobe develops it colour conversion matrix based on lab measurements. As far as I know DNG does contain two colour conversion matrices one for illuminant D and one for A, with interpolation between the two for different colour temperature.

What I would ask if raw data from say Nikon does contain colour conversion matrices, or if these need to be measured. If the raw file doesn't contain conversion matrices, I would guess these need to be measured.

Best regards
Erik

Hi Bob,

I don't understand that. Your new D810 will presumably allow to take better images than your current setup. It will do that from day 1. The Raw files can be converted by a number of alternative Raw converters, including the one that comes with the camera (I assume). And you can at a later date still revisit the Raws, they are not going away, but at least you have got them, rather that having missed the opportunity to shoot them.

The Raw files will still be there when you unleash their full potential when your trusted Raw converter also supports a good conversion. That support has mainly to do with color calibration, which I'd rather like having done well, rather than mediocre.

Cheers,
Bart

[digitaldog]

What I would ask if raw data from say Nikon does contain colour conversion matrices, or if these need to be measured. If the raw file doesn't contain conversion matrices, I would guess these need to be measured.

My understanding is the conversion to the internal working space's source is 'assumed' in some way by some (all?) 3rd party converters unless they somehow figure out the spectral sensitivities of the sensor/filters. I have no idea how each company does this and would suspect the manufacturer has a better/different idea unless everyone is doing this identically.

Might want to check out Doug Keer's piece on this, his comments are below:

The matter of the "color space" of the sensor array itself is a fascinating one. It is discussed at length in my new technical article, "Digital Camera Sensor Colorimetry", available here:

http://dougkerr.net/Pumpkin#Sensor_Colorimetry

This article is very fat, and was actually written for the selfish purpose of recording the many fascinating things I had learned, and come to somewhat understand, during an intense week-long study of the literature covering various interlocking technical areas. (If I don't do that, it will all evaporate with a half-life of about three weeks.)

Now underway is the extraction from this heap of some articles of manageable scope and length covering particular aspect of the field. One is the matter of "What color space does the camera sensor actually operate in [essentially in what color space is the raw data to be interpreted, once we get over the fact that there are not three photodetectors at any given pixel location], and what are its primaries."

The answer to that is fascinating (perhaps even shocking), and I'll give a summary here.

• The typical camera sensor array does not maintain "metameric accuracy", which means that although there can be many light spectrums that all have the same color, the sensor does not necessarily give the same set of three outputs (implying the same color) for all of them. Not desirable, but a result of various design compromises.

• Thus, the sensor cannot be said to practice any color space at all.

• However, without actually ever saying so, we treat is as if it did practice a color space (it does "approximately"); else we could not transform its outputs, after demosaicing (or perhaps more precisely, during demosaicing), to any bona fide "delivery" color space, such as sRGB or Adobe RGB.)

• Stipulating to the notion that the sensor practices a color space, what are its primaries? Fascinatingly enough, they are all imaginary (like the primaries of the CIE XYZ color space). They could not be physically generated, and if they could, we could not see them. They have existence only in that they follow the recognized mathematical rules about what happens when we combine two or more primaries to produce an actual visible color.

• Could that conundrum have been avoided by "proper" design of the sensor? No. For a sensor to practice a color space having real (physically-realizable) primaries, its "channel" spectral response curves would have to be negative for some range of wavelengths. That is physically impossible. (We cannot design a filter that, if placed in front of a photodetector, would result in the photodetector giving a positive output for some wavelengths and a negative one for others. No, an offset won't fix that - the output must be zero for "no" stimulus.)

• Wouldn't it be nice to choose filter responses for a sensor array such that its three outputs directly related to the sRGB color space (that is, were essentially the values R, G, and B in linear form)? Nice, but not possible. Such a sensor would have real primaries (the sRGB primaries, in fact). As a corollary of the matter described just above, for that to be so, each response curve would have to be negative over some range of wavelengths, not physically possible.

I have to tell you that learning all this was somewhat of a shock to me.

[Camdavidson]

The RC version does not do a very good job of converting D810 files.  (I have shot with mine since last Thursday)  NX-D does a significantly better job with conversions.  I expect Capture One will have great profiles as will other companies.  

http://photographylife.com/nikon-d810-vs-d800e-iso-comparison

This review speaks to the difficulties of trusting the Adobe conversions.

[Iliah]

My understanding is the conversion to the internal working space's source is 'assumed' in some way by some (all?) 3rd party converters unless they somehow figure out the spectral sensitivities of the sensor/filters. I have no idea how each company does this and would suspect the manufacturer has a better/different idea unless everyone is doing this identically.

In an indirect way, Nikon are supplying their camera profiles with SDK. It is also a banal thing to calculate colour transforms using Nikon converter. Shooting a target and computing colour transforms is elementary, too.

[digitaldog]

Shooting a target and computing colour transforms is elementary, too.

I suppose if my goal was to get the output referred color numbers (why not just look at what the converter shows me?). That's not the data I'm referring to. Nor Doug if I understand his fine article.

[Iliah]

All third-party converters are using one or two of the above-mentioned methods.

[digitaldog]

All third-party converters are using one or two of the above-mentioned methods.

Maybe I'm confused or we're talking about two different parts of the process. The raw image data is in some native camera color space right? But it is not a colorimetric color space, and has no single “correct” relationship to colorimetry. How to get there and what is the native camera color space? Again, my understanding is that short of having spectral sensitivities of the filters, someone has to make a choice of how to convert values in non-colorimetric color spaces to colorimetric ones. There are good and not as good choices, but no single correct conversion (unless the “scene” you are photographing has only three independent colorants, like when we scan film). The image has a color space, but do we know what it is? If you know the camera spectral sensitivities, you may estimate the image color space quite well if the camera spectral sensitivities happen to satisfy the Ives-Luther condition which is unlikely. Only then we can determine the colorimetric color space. So do all the raw converters have this necessary data supplied to them or as I'm led to believe, they make some educated guesses and move on?

It is kind of useful that critical photographers understand this otherwise they might wonder why they get different results photographing the same scene with different cameras, even when they use the same raw converter. Or, why different conversions to scene-referred produce different results for the same scene and camera. You need to know there is judgment involved in designing the conversion otherwise you won’t know to perform evaluations to see what conversion you like best.

[Iliah]

> raw image data is in some native camera color space

Depends on how one defines colour space. Mathematically and from the point of view of physics, no, it is not a space because the metrics is not defined.

> short of having spectral sensitivities of the filters, someone has to make a choice of how to convert values in non-colorimetric color spaces to colorimetric ones

Spectral sensitivity is not the solution here. One of the reasons is that it does not help with the metameric error.

Speaking of C1, they calculate colour transforms based on shooting a good target. Myself, I determine spectral responses and shoot targets, too. Happens that profiling from targets renders better results (both in terms of dE2000 and visually).