Thanks for the feedback Eric, I really appreciate it.
A few comments follows;
If the DNG format does support calibration data I would suggest to change the implementation of your DNG converter so that it embeds calibration data rather than applying it for formats like 3FR and IIQ. Users generally don't like "cooked" raws, and the only reason people haven't complained in this case is probably because they don't know it's happening.
One reason to change the neutral axis which is not related to look is if you want to use DNG profiles for reproduction photography and want to calibrate a fixed white balance setting. I also find the mix between "colorimetric scene-referred" and "subjective output-referred" a bit strange. These limitations are said to be because the format is intended to be scene-referred, but the look table, tone curve and indeed Adobe's own profiles says otherwise -- it's output-referred. When the actual use is output-referred with subjective tunings I find it odd to limit the neutral axis. With the neutral axis limited I also find it odd why these entries are included in the LUT at all, as their values must be set to 1,1,0. Adobe Camera Raw actually ignores those entries, I've tried to set them to other values but it has no effect.
You didn't comment on the hue discontinuity point. I did not explain that in detail either, here it comes. Hue is described as a circular value in the range 0-360 degrees, which means that 0 is the same as 360, +180 is the same as -180 etc. This makes the math a little tricky you need to take special care to make those additions right so you get the right result. In the DNG pipeline there is no such care taken. This means that if you have two neighboring entries in the LUT which differ more than 180 degrees, there will be an error in the calculation. Since RGB-HSV hue gets unstable when saturation is low it's not that unlikely you hit this when designing a profile, for example if the profile is designed in a CIECAM02 coordinate system. It's not too hard to work around though so it's not a huge deal, but the profile format would be easier to work with if the pipeline would have proper circular math when adding hues. If you don't want to fix this (you might not want to as it will potentially slow down the code), it should be a documented part of the standard. Now it just looks suspiciously similar to a bug
I have indeed noted that some Adobe profiles use the forward matrix as a no-op, and that you essentially make all color correction in the LUT. The output space is still ProphotoRGB though so you're still limited by its boundaries. Of course you could make some own non-standard translation on top with other primaries, but it would not be the standard. So I still think it's just fair to say that DNG in its standard mode is limited to ProPhoto and is not intended to cover the full human gamut. If that's a problem or not is debatable of course. Personally I don't think it's a big problem, but I find it unlikely that a standard organization would choose this type of limitation today.
If using the forward-matrix as no-op becomes a de-facto standard to avoid clipping, maybe one should do like the ICC Lab LUT profile case, specify in the standard that the matrix should be a no-op when a LUT is available? At least it would be helpful for profile designers to present this option and why it's useful in the DNG spec.
Interesting note about baseline exposure that it's used for compensating changes caused by ISO settings. I think it should then be thoroughly documented in the DNG spec that baseline exposure should be used for that and nothing else, otherwise there's a risk that the tag is used for the wrong purpose. A wrong use would be if it's the same over the whole ISO range, or if there is a base offset that is the same (that is if it varies between 0.2 and 0.5, it should instead vary between 0.0 and 0.3 and the profile should add the 0.2). In the current specification there is nothing mentioned about variation over ISO.
I have a strong view that letting the baseline exposure be "anything" is bad
, so I'm certainly not satisfied with the current status, and I think I have quite strong arguments for my view. To make it a standard you need to think more about third parties so it not just becomes an internal Adobe Camera Raw format. Breaking the dependency between the DNG raw file itself and and the DCP is an important aspect of that, and to do that the baseline exposure can't work as it does now, oh well maybe it can (I haven't checked in detail how DNG converter uses it), but the documentation must state how it should be used that is only as an ISO-correcting offset (or any other factors that make exposure vary) with 0.0 as base. My point is that it should definitely not be used as a fixed offset, that should be part of the profile. Otherwise you break interoperability.
There's another aspect that probably should be documented similar to the BlackRender part, namely how the exposure offset is converted to a tone curve, which is a good idea as far as I can see especially for negative offsets (otherwise clipped data would be brought into view). There's code for that in the DNG reference SDK, perhaps it should be documented in the spec?
I can certainly live with the white-point preserving thing, I optimize my matrices with that constraint. I'm just mentioning it because ICC doesn't have this limitation and others might have a problem with that.
Considering the white balance model it's true that you can just ignore it. It just feels like one of those things that is more like an internal Adobe thing that exists in the standard for the convenience of Adobe rather than it's useful for the standard. If it's only there for Adobe to use, I don't think it should be there. I need to think more about it though, I haven't a 100% complete view on the subject of white balance / whitepoint modeling.
One particular aspect that gives me a bit of a headache when designing profiles that people use with Adobe Lightroom is that when the user has set a custom white balance Lightroom remembers the temp/tint setting and then run it backwards through the profile to calculate the multipliers. This means that if my color matrices are not exactly the same as Adobe's profiles there will be a white balance shift (that is when you switch to my profile the tint of the image changes). So I just had to add the option to copy the color matrices from existing profiles for those users that didn't want this to happen. The problem is that estimating temp/tint with a camera matrices cannot be very exact so even if Adobe and I have the most exact methods it's likely that the result will differ several hundred degrees anyway. I haven't analyzed this in depth yet to suggest how it should be done, but I think that there is a problem currently. Any aspect that makes interoperability difficult is not good for a standard.