Luminous Landscape Forum

Raw & Post Processing, Printing => Digital Image Processing => Topic started by: Jonathan Wienke on October 31, 2009, 11:37:51 pm

Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on October 31, 2009, 11:37:51 pm
I'm currently writing a lens correction and image optimization program that (if everything works as planned) will perform the following functions:

1. Removal of image blurring introduced by the lens, camera AA filter, diffraction, and RAW converter demosaic algorithm.

2. Correction of radial and longitudinal chromatic aberrations.

3. Correction of coma, spherical aberrations, and other similar distortions.

4. Correction of barrel, pincushion, and the most complex forms of mustache distortion.

5. Vignetting correction.

6. Stacking multiple DNG images for focus or exposure blending.

For best results, the approach I'm working on will require the user to shoot a specially-designed target containing an array of point sources of white light with each combination of camera, lens, aperture, and focal length to create a "blur profile" customized to that particular combination of hardware. As an alternative, my goal is to allow users to create and share blur profiles (e.g. for a Canon 1Ds wearing a 17-40/4L at f/4 and 17mm). As long as a given user's hardware matches that used to generate the blur profile, results will be good. If the user's hardware is different (e.g. the lens is softer in the corners at 17mm) than the profiled hardware, results will be poor, and the user will need to make their own custom profile. This would be similar to the owner of a printer finding that factory "canned" profiles work poorly and improving output by making a custom printer profile.

The biggest hassle I'm encountering right now is figuring out the DNG SDK. I've downloaded it and browsed through it, but there's a huge number of classes, and the object hierarchy and their intended usages aren't clear to me yet. I'm trying to figure out how to accomplish the following tasks:

1. Open a DNG file (preferably the demosaiced linear RGB flavor), and read the image and metadata (especially camera model, lens model, focal length, and aperture) into memory.

2. Once the image has been processed, write it to a new linear RGB DNG file that can be opened in any DNG-aware application, with all original tags and metadata correctly preserved.

All of the internal correction operations are done on linear RGB data in the camera's native color space, so that setting white balance, selection of color profile, etc. is unaffected by the image correction operation. The user will have the exact same color processing options, ability to select white balance, exposure adjustments, and output color space selection as when working with the original RAW(s).

Any advice from the experts would be greatly appreciated.
Title: Seeking DNG SDK Assistance
Post by: Schewe on October 31, 2009, 11:44:00 pm
Quote from: Jonathan Wienke
Any advice from the experts would be greatly appreciated.

While Eric does hang out here, Thomas doesn't. The best place to ask very specific questions relating to DNG would be the DNG Forum (http://forums.adobe.com/community/dng)...
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on October 31, 2009, 11:53:46 pm
Thanks for the pointer; I'll post an inquiry there as well. There doesn't appear to be a whole lot of traffic there though, less than 10 threads in the last 6 months in the DNG SDK forum...
Title: Seeking DNG SDK Assistance
Post by: Mark D Segal on November 01, 2009, 09:27:28 am
Jonathan,

I can't contribute anythng of technical substance to your inquiries - that's for the kind of highly expert people Jeff mentions, but I am very interested in what you are trying to do and would like to remain in loop about how it is progressing. Good luck with it.

Mark
Title: Seeking DNG SDK Assistance
Post by: madmanchan on November 01, 2009, 03:52:26 pm
Hi Jonathan,

The best place to start in terms of the code would be dng_validate.cpp. The dng_validate () routine in there will read in a negative, parse through it, and build the various image stages. This includes decompressing the raw image data, linearizing it if necessary, and render to a TIFF file if you want. From a developer's perspective, I'd suggest putting a breakpoint somewhere in that routine, feeding a DNG raw file to it, and then stepping through to see how the code flows. In terms of terminology, the code comments use the term "stage 1" image to describe the raw image data, "stage 2" to describe the linearized raw image data (in a canonical space [0, 1], or [0, 65535] if you prefer), and "stage 3" to describe the 3-channel or 4-channel data (i.e., after demosaic). If you want to do processing on linearized mosaic data, you want to grab the stage 2 image; see the Stage2Image () routine in the dng_negative class (dng_negative.h). If you want to do processing on RGB demosaiced data in the native camera color space, prior to white balance, you want the stage 3 image; see the Stage3Image () routine in the dng_negative class.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 01, 2009, 08:04:21 pm
Quote from: madmanchan
If you want to do processing on RGB demosaiced data in the native camera color space, prior to white balance, you want the stage 3 image; see the Stage3Image () routine in the dng_negative class.

Thank you, that's very helpful. The processing I'm doing is all on linear RGB data scaled 0-1 in floating-point format (this makes a lot of the deconvolution math much more straightforward), so I'll need the stage 3 data and normalize it to a maximum value of 1. Three questions:

Is the stage 3 data always scaled to the same output range (say 0-65535) regardless of the number of bits/pixel in the original RAW file, or does the range of image values vary by camera model?

How do I write the modified stage 3 data back to a new DNG file, while copying all metadata from the original DNG?

I've got the dng_validate project downloaded along with XMPCore and expat so dng_validate will build with no errors or warnings. I'd like to compile dng_sdk (along with whatever it needs from expat and XMPCore) as a either a .NET or COM .dll so I can use its functionality from Visual Basic .NET or other .NET-family languages like C#, etc. How can I do this from Visual C++?

Sorry for the stupid questions, my programming background is VB, not C, and I'm having a bit of trouble getting my head wrapped around all of the DNG SDK components and how they relate to each other.
Title: Seeking DNG SDK Assistance
Post by: madmanchan on November 02, 2009, 01:27:33 pm
Yes, the stage 3 image data is always normalized to 16 bits [0,65535], independent of camera model. That is, if you have intentionally clipped a pixel value by overexposing, that pixel component will be 65535. Typically the stage 3 image will be very "green" because it is not yet white-balanced.

To modify a DNG, you'll want to take a look at lines roughly 250 thru 320 of dng_validate.cpp. That's where you'll find sample code that writes a DNG file out to disk; the metadata should be preserved, so you don't have to worry about that. Also, the SetStage3Image () routine in the dng_negative class will let you store your modified negative data back to the dng_negative object.

Note that you'll have to save as a linear DNG since your modified data is already 3-channel (no longer in mosaic form). I believe you can do this by using

host.SetSaveLinearDNG (true)

prior to saving out the image.
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 02, 2009, 06:00:52 pm
Quote from: Jonathan Wienke
Removal of image blurring introduced by ... RAW converter demosaic algorithm.

it looks like your software is running before any raw converter, otherwise why bother to output into linear DNG ? to have a workflow like : in camera raw file -> dng converter (well only ~5% cameras out there will have native in camera DNG) -> raw converter #1 -> your software -> raw converter #2 ?  too many steps... and if you want to run it after the raw converter very few will produce non linear DNG file as their output (if any ? I know several programs that operate (modify the raw data) on DNG files w/o making them linear, but they are hardly widely used fully functional raw converters... )... and if you are running before a raw converter than how do you plan to coexist ? does DNG has a tag saying - hello people, I do contain a corrected data (besides just linearizing), please do not bother w/ certain corrections and optimizations ? just curious... or if you plan to act like DxO (raw -> DxO (corrections) -> linear DNG -> ACR/LR to deal w/ WB and some colors) then you have to be a good in demosaicing first of all... so why don't you just write a fully functional raw converter ?
Title: Seeking DNG SDK Assistance
Post by: madmanchan on November 02, 2009, 06:06:40 pm
deja, it depends on the extent of the corrections Jonathan wishes to pursue. Some of the items he describes can be expressed as metadata, via DNG opcodes -- see Chapter 7 of the DNG 1.3 spec:

http://www.adobe.com/products/dng/pdfs/dng_spec_1_3_0_0.pdf (http://www.adobe.com/products/dng/pdfs/dng_spec_1_3_0_0.pdf)
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 02, 2009, 06:09:03 pm
Quote from: madmanchan
deja, it depends on the extent of the corrections Jonathan wishes to pursue. Some of the items he describes can be expressed as metadata, via DNG opcodes -- see Chapter 7 of the DNG 1.3 spec:

http://www.adobe.com/products/dng/pdfs/dng_spec_1_3_0_0.pdf (http://www.adobe.com/products/dng/pdfs/dng_spec_1_3_0_0.pdf)

ouch, for example (w/o reading .PDF - you should be able to tell that in a blink - I mean to tell about tags/opcodes, not that I am lazy - that I know already myself) do you have a tag (DNG opcode) saying - please, raw converter, do not do any automatic CA/PF correction, it was done already ?
Title: Seeking DNG SDK Assistance
Post by: madmanchan on November 02, 2009, 06:52:47 pm
deja, there is not currently such a tag. This is a more general problem for all images, not just DNG. For example, if you have a JPEG or TIFF file lying around somewhere, it can be difficult to tell whether or not it has had some form of lens corrections applied to it. A standard metadata tag would be useful to indicate whether specific forms of processing had been applied to the image, in this case. My understanding is that such tags are under current consideration for standardization, but that could take a while.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 03, 2009, 02:34:19 am
Quote from: madmanchan
Note that you'll have to save as a linear DNG since your modified data is already 3-channel (no longer in mosaic form). I believe you can do this by using

host.SetSaveLinearDNG (true)

prior to saving out the image.

OK, thank you that is what I was looking for. Linear RGB DNG export is exactly what I want.

Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 03, 2009, 03:27:29 am
Quote from: deja
or if you plan to act like DxO (raw -> DxO (corrections) -> linear DNG -> ACR/LR to deal w/ WB and some colors) then you have to be a good in demosaicing first of all... so why don't you just write a fully functional raw converter ?

Short answer: I don't want to spend a bunch of time reinventing the wheel when there are many very good RAW converters already out there. OTOH, DXO doesn't really have a lot of competition for really good lens corrections, it's way overpriced, and it's crippled because you are locked into using "canned" blur profiles for cameras and lenses, and if DXO doesn't have profiles for your camera or lens, you're screwed--you can't use it.

If your lens' blur characteristics differ from the DXO-profiled lens, then the "canned" profiles aren't going to work all that well anyway. If a printer manufacturer only allowed the use of their canned print profiles, and didn't allow users to make their own profiles, they'd be laughed out of the market. But that is essentially DXO's business model.

Quote
it looks like your software is running before any raw converter, otherwise why bother to output into linear DNG ?

The corrections I'm doing need to be done after demosaic. The deconvolution algorithm I'm working on analyzes a linear RGB image of an array of circular light sources (ideally about 5-10 pixels in diameter in the RAW image) and analyzes the blur characteristics found. The blur in the demosaiced image is a combination of lens blur, AA filter blur, and blur introduced by the demosaic algorithm of the RAW converter. By analyzing the total blur caused by all elements of the image capture process (lens, camera, and RAW converter) all of the blur from every blur source can be removed at once.

The basic workflow is this:

1. Run DNG Converter to convert RAWs to linear-RGB DNG files. Essentially all this does is fills in the missing color channel values from the Bayer matrix (e.g. adds R & B values to a G pixel), and then scales the values 0-65535. This can be done on a batch of files.

2. Process the DNGs through my program to correct lens, AA filter, and demosaic blur, remove CA/color fringing, correct fisheye/barrel/pincushion/mustache distortion, eliminate vignetting, etc., and eventually stack multiple focus and/or exposure-bracketed images into a single output DNG. This can be a batch process also.

3. Open the processed DNGs and edit as desired.

The RGB data is still in the camera's native color space, so you still have total flexibility to set white balance, adjust exposure, process colors, etc. the same as with the original RAW, as long as the RAW converter you use can read DNGs. Using my program will require an extra step in the workflow, but my goal is to make the benefits greatly outweigh the slight inconvenience.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 03, 2009, 03:31:58 am
Quote from: madmanchan
deja, there is not currently such a tag. This is a more general problem for all images, not just DNG. For example, if you have a JPEG or TIFF file lying around somewhere, it can be difficult to tell whether or not it has had some form of lens corrections applied to it. A standard metadata tag would be useful to indicate whether specific forms of processing had been applied to the image, in this case. My understanding is that such tags are under current consideration for standardization, but that could take a while.

My solution for now at least is simply to look for lens correction opcode tags in the source DNG file, and omit them when writing the destination DNG file. If a tag is ever defined to indicate that lens corrections have already been applied, then I'll add it to the output file...
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 03, 2009, 10:30:51 am
Quote from: Jonathan Wienke
The corrections I'm doing need to be done after demosaic.

The basic workflow is this:

1. Run DNG Converter to convert RAWs to linear-RGB DNG files. Essentially all this does is fills in the missing color channel values from the Bayer matrix (e.g. adds R & B values to a G pixel), and then scales the values 0-65535. This can be done on a batch of files.

2. Process the DNGs through my program to correct lens, AA filter, and demosaic blur, remove CA/color fringing, correct fisheye/barrel/pincushion/mustache distortion, eliminate vignetting, etc., and eventually stack multiple focus and/or exposure-bracketed images into a single output DNG. This can be a batch process also.

3. Open the processed DNGs and edit as desired.

you will endup w/ a subpar demosaicing in the first place that way... that is the problem here - you will be optimizing not the best possible demosaiced data... not a problem in many cases for many people, but aren't you after the ultimate quality ?
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 03, 2009, 11:42:26 am
Quote from: deja
you will endup w/ a subpar demosaicing in the first place that way... that is the problem here - you will be optimizing not the best possible demosaiced data... not a problem in many cases for many people, but aren't you after the ultimate quality ?

Only if you consider ACR a "subpar" RAW converter. And you're not necessarily limited to ACR; any RAW converter that can export to linear-RGB DNG can be used for demosaicing. The majority of the lens corrections have to be done on demosaiced RGB data; for example, CA corrections (which involve shifting the locations of color channels relative to each other) cannot be saved back to RAW because the pixel locations no longer match the Bayer matrix pattern. Correcting barrel/fisheye distortion has the same problem, only worse because the degree of pixel-shifting es even greater.

It just makes sense to demosaic first, and then apply corrections and adjustments.
Title: Seeking DNG SDK Assistance
Post by: sandymc on November 03, 2009, 01:02:14 pm
Quote from: Jonathan Wienke
Only if you consider ACR a "subpar" RAW converter.

Umm, DNG converter has the full ACR demosaicing algorithm embedded in it, rather than a simplified, lower performance version? That would surprise me. But then, I get surprised on a regular basis.....  

But the fundamental problem you run into is actually that in order to do lens correction, you have to interpolate pixels. So if you do the correction as a separate step to the demosaicing, you have two layers of interpolation on top of each other. Of course, you can do that, but not, IMHO, a good idea. Which is why I believe that DxO will only write a linear DNG - interpolation for demosaicing and for lens correction are a single step for them. But that's only what I believe  

Sandy
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 03, 2009, 02:34:29 pm
Quote from: Jonathan Wienke
Only if you consider ACR a "subpar" RAW converter.

the question is about the quality of demosaicing that DNG converter has ?


Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 03, 2009, 06:50:03 pm
The testing I've done indicates that opening a linear-RGB DNG (demosaiced in DNG converter) and processing the source RAW directly in ACR makes zero difference--if both files are processed with the same settings, the outputs will be an exact pixel-for-pixel match. Where did you get the notion that DNG converter didn't use the same demosaic engine as ACR? If you think otherwise, do your own comparison.

 As to the single interpolation vs double objection, that's a load of crap. Demosaicing compares Bayer pixels to their immediate neighbors to guess the values of the missing channels. That requires looking at the pixels before the color channels are shifted around relative to each other (as in CA correction), or the whole concept of "adjacent pixels" gets flushed down the toilet. You have to demosaic first, then you can juggle the location of the color channels. Doing otherwise is a recipe for crappyocrity, not a quality enhancement.
Title: Seeking DNG SDK Assistance
Post by: sandymc on November 04, 2009, 01:12:59 am
Quote from: Jonathan Wienke
The testing I've done indicates that opening a linear-RGB DNG (demosaiced in DNG converter) and processing the source RAW directly in ACR makes zero difference--if both files are processed with the same settings, the outputs will be an exact pixel-for-pixel match. Where did you get the notion that DNG converter didn't use the same demosaic engine as ACR? If you think otherwise, do your own comparison.

 As to the single interpolation vs double objection, that's a load of crap. Demosaicing compares Bayer pixels to their immediate neighbors to guess the values of the missing channels. That requires looking at the pixels before the color channels are shifted around relative to each other (as in CA correction), or the whole concept of "adjacent pixels" gets flushed down the toilet. You have to demosaic first, then you can juggle the location of the color channels. Doing otherwise is a recipe for crappyocrity, not a quality enhancement.

OK, I deduce you have firmly held views on this subject(!)...................I guess we await your working code with eager anticipation so we can get to see all that quality shine through.

Good luck.

Sandy
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 04, 2009, 02:54:11 am
Quote from: Jonathan Wienke
Only if you consider ACR a "subpar" RAW converter. And you're not necessarily limited to ACR; any RAW converter that can export to linear-RGB DNG can be used for demosaicing. The majority of the lens corrections have to be done on demosaiced RGB data; for example, CA corrections (which involve shifting the locations of color channels relative to each other) cannot be saved back to RAW because the pixel locations no longer match the Bayer matrix pattern. Correcting barrel/fisheye distortion has the same problem, only worse because the degree of pixel-shifting es even greater.

It just makes sense to demosaic first, and then apply corrections and adjustments.

Actually, CA correction can be done, and done better, during demosaic rather than after.

http://www.ojodigital.com/foro/perfectraw-...erpolacion.html (http://www.ojodigital.com/foro/perfectraw-perfectblend/265272-amaze-el-nuevo-algoritmo-de-interpolacion.html)

Even so, geometric corrections might be better done after demosaic.   I'll be interested to see what comes out of your project.
Title: Seeking DNG SDK Assistance
Post by: madmanchan on November 04, 2009, 08:56:11 am
Hi Sandy, yes, you are right: the DNG SDK only has a simple bilinear interpolation demosaic routine implemented. (We have been meaning to update it with something more serviceable, but have not worked that into the already rather-packed dev schedule ...)
Title: Seeking DNG SDK Assistance
Post by: sandymc on November 04, 2009, 09:25:52 am
Quote from: madmanchan
Hi Sandy, yes, you are right: the DNG SDK only has a simple bilinear interpolation demosaic routine implemented. (We have been meaning to update it with something more serviceable, but have not worked that into the already rather-packed dev schedule ...)

Hi Eric - I can sympathize with the dev schedule!!!!

Even a "text book" AHD implementation would be really cool though......and make doing things such as the OP wants to do a lot easier.

Regards,

Sandy
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 04, 2009, 10:39:30 am
Quote from: Jonathan Wienke
The testing I've done indicates that opening a linear-RGB DNG (demosaiced in DNG converter) and processing the source RAW directly in ACR makes zero difference

well, too bad for ACR then...
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 04, 2009, 11:17:34 am
Quote from: ejmartin
Actually, CA correction can be done, and done better, during demosaic rather than after.

http://www.ojodigital.com/foro/perfectraw-...erpolacion.html (http://www.ojodigital.com/foro/perfectraw-perfectblend/265272-amaze-el-nuevo-algoritmo-de-interpolacion.html)

The samples in that thread didn't look that impressive to me; in the side-by-side comparison of the makeup brushes, AMAZE didn't appear to do as well as the other converter it was compared to. Correcting CA during demosaic may not be worse than doing so after, but I didn't see any proof it's significantly better.
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 04, 2009, 12:15:33 pm
Quote from: Jonathan Wienke
The samples in that thread didn't look that impressive to me; in the side-by-side comparison of the makeup brushes, AMAZE didn't appear to do as well as the other converter it was compared to. Correcting CA during demosaic may not be worse than doing so after, but I didn't see any proof it's significantly better.

The side-by-side with the brushes is a comparison to AMaZE prior to CA correction, the CA corrected version is lower down in the thread (post #11).  

Here's a direct comparison with ACR at 400% (using a different image). (http://theory.uchicago.edu/~ejm/pix/20d/posts/ojo/CAfix_animation.gif)  I set the ACR 4.6 CA correction slider so as to minimize CA artifacting in the blinds (there was no setting that eliminated it altogether), and it still crops up in the church steeple and the tree limbs/branches.  Anyway, regardless of the specific implementation, it seems to me that any geometric correction that can be applied in a linear space such as RAW is better done there rather than after the nonlinear operation of demosaic.   For instance, CA can corrupt the demosaic process itself, depending on the algorithm used (cf the HPHD examples from RawTherapee in the Ojodigital thread).
Title: Seeking DNG SDK Assistance
Post by: Guillermo Luijk on November 04, 2009, 07:19:46 pm
Quote from: Jonathan Wienke
All of the internal correction operations are done on linear RGB data in the camera's native color space, so that setting white balance, selection of color profile, etc. is unaffected by the image correction operation. The user will have the exact same color processing options, ability to select white balance, exposure adjustments, and output color space selection as when working with the original RAW(s).
Jon, if I understood well you plan to demosaic the RAW data with the DNG routines, and that means you should apply white balance first. You can demosaic with no WB applied, but this will probably affect the efficiency of the demosaic process in terms of getting right colours and detail.

On the other hand, applying a WB doesn't mean you cannot modify it later, but you will have to deal carefully with the highlights in the white balancing process: to preserve them will mean you will move the saturation point to some unknown (or difficult to know) place in the histogram. And this point needs to be known if additional exposure or white balance adjustments are to be done to the output DNG file.

BTW if you would like any advice on how to blend different RAW files (relative exposure calculation, and minimal progresiveness blending) feel free to ask.

Regards
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 04, 2009, 07:51:35 pm
Quote from: ejmartin
The side-by-side with the brushes is a comparison to AMaZE prior to CA correction, the CA corrected version is lower down in the thread (post #11).  

Here's a direct comparison with ACR at 400% (using a different image). (http://theory.uchicago.edu/~ejm/pix/20d/posts/ojo/CAfix_animation.gif)  I set the ACR 4.6 CA correction slider so as to minimize CA artifacting in the blinds (there was no setting that eliminated it altogether), and it still crops up in the church steeple and the tree limbs/branches.  Anyway, regardless of the specific implementation, it seems to me that any geometric correction that can be applied in a linear space such as RAW is better done there rather than after the nonlinear operation of demosaic.   For instance, CA can corrupt the demosaic process itself, depending on the algorithm used (cf the HPHD examples from RawTherapee in the Ojodigital thread).

I'm still not convinced that correcting CA during demosaic is the magic bullet you think it is. Other than the CA, the differences in the conversions were minimal, mostly due to slightly different tone curve or exposure settings.

The problem with ACR and most every other RAW converter's CA correction is that it's using a fairly simple mathematical formula (something along the lines of a simple gamma curve) to calculate the pixel shift distance needed to realign the color channels and correct the CA. The problem is that this formula is an approximation many lenses do not follow very well; therefore for many lenses there is no setting that creates the proper curve to correct the CA completely. The best you can do is spread the error fairly evenly so that no part of the image looks too horrible. The approach I'm using creates 3 spline curves with 32-64 data points each (calculated separately for each color channel) so that the color channels can be aligned to within 1/10 of a pixel or so from the center of the image all the way to the corners, regardless of whether the lens' distortion curve is simple or extremely complex.
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 04, 2009, 08:32:38 pm
Quote from: Jonathan Wienke
I'm still not convinced that correcting CA during demosaic is the magic bullet you think it is. Other than the CA, the differences in the conversions were minimal, mostly due to slightly different tone curve or exposure settings.

Most differences between good conversions are minimal, up to tone curve and color space conversion.  We're talking about the last little bit of data near Nyquist, otherwise there isn't much to discuss.

Quote
The problem with ACR and most every other RAW converter's CA correction is that it's using a fairly simple mathematical formula (something along the lines of a simple gamma curve) to calculate the pixel shift distance needed to realign the color channels and correct the CA. The problem is that this formula is an approximation many lenses do not follow very well; therefore for many lenses there is no setting that creates the proper curve to correct the CA completely. The best you can do is spread the error fairly evenly so that no part of the image looks too horrible. The approach I'm using creates 3 spline curves with 32-64 data points each (calculated separately for each color channel) so that the color channels can be aligned to within 1/10 of a pixel or so from the center of the image all the way to the corners, regardless of whether the lens' distortion curve is simple or extremely complex.

There isn't much point in aligning to 1/10 pixel spacing when 2/3 of the data has been interpolated in different directions using colors that are as much as a pixel or more away from where they should be.  No mathematical formula with a small number of parameters is going to correct for the fact that a color has been interpolated incorrectly in the horizontal direction in one place, and incorrectly in the vertical direction nearby, due to the mismatched color registration caused by CA.  That is why it is important to do the CA correction before demosaic.  

Any formula for correction that you may have can be applied before demosaic with the right approach.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 04, 2009, 10:47:46 pm
Quote from: GLuijk
Jon, if I understood well you plan to demosaic the RAW data with the DNG routines, and that means you should apply white balance first. You can demosaic with no WB applied, but this will probably affect the efficiency of the demosaic process in terms of getting right colours and detail.

Actually it doesn't. If you export a linear-RGB DNG from DNG Converter or Bridge, and then process that DNG and the original RAW file in ACR with the same settings, the result is a pixel-for-pixel match, and you have exactly the same flexibility for setting exposure, white balance, and color adjustments either way.

Quote
BTW if you would like any advice on how to blend different RAW files (relative exposure calculation, and minimal progresiveness blending) feel free to ask.

Thank you. Would you be interested in perhaps combining our projects, to create one DNG processing tool that does everything?
Title: Seeking DNG SDK Assistance
Post by: Guillermo Luijk on November 05, 2009, 02:44:05 pm
Quote from: Jonathan Wienke
Actually it doesn't. If you export a linear-RGB DNG from DNG Converter or Bridge, and then process that DNG and the original RAW file in ACR with the same settings, the result is a pixel-for-pixel match, and you have exactly the same flexibility for setting exposure, white balance, and color adjustments either way.

But how do you use Bridge to demosaic a non white balanced RAW file?
At least using DCRAW, the AHD algorithm works quite differently if applied on white balanced RAW data than when used on non white balanced data. Colour artifacts and differently interpolated detail appears when the RAW values are not balanced, surely Emil can explain why these issues happen:

White balance prior to demosaicing:
(http://www.guillermoluijk.com/tutorial/dcraw/artefactoswb1.jpg)

White balance after demosaicing:
(http://www.guillermoluijk.com/tutorial/dcraw/artefactoswb2.jpg)


Quote from: Jonathan Wienke
Thank you. Would you be interested in perhaps combining our projects, to create one DNG processing tool that does everything?

That's OK, but your workflow is based on demosaiced data; what I would like to achieve is a pure Bayer blend.

Regards
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 05, 2009, 02:56:35 pm
Quote from: GLuijk
But how do you use Bridge to demosaic a non white balanced RAW file?
Bridge can run ACR directly w/o PS
Title: Seeking DNG SDK Assistance
Post by: Guillermo Luijk on November 05, 2009, 03:04:03 pm
Quote from: deja
Bridge can run ACR directly w/o PS
I know, but it allows not to white balance the RAW file? and it allows not to convert to an output colour profile?
My ACR doesn't.
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 05, 2009, 04:32:36 pm
Quote from: GLuijk
But how do you use Bridge to demosaic a non white balanced RAW file?
At least using DCRAW, the AHD algorithm works quite differently if applied on white balanced RAW data than when used on non white balanced data. Colour artifacts and differently interpolated detail appears when the RAW values are not balanced, surely Emil can explain why these issues happen:

White balance prior to demosaicing:
http://www.guillermoluijk.com/tutorial/dcr...tefactoswb1.jpg (http://www.guillermoluijk.com/tutorial/dcraw/artefactoswb1.jpg)

White balance after demosaicing:
http://www.guillermoluijk.com/tutorial/dcr...tefactoswb2.jpg (http://www.guillermoluijk.com/tutorial/dcraw/artefactoswb2.jpg)

Take for instance the one column of pixels in the pole atop the church dome; it is either GRGRGR data or BGBGBG data.  Either B or R needs to be interpolated from an adjacent column.  If the data is approximately white balanced, the interpolation error is moderate.  If the data is not white balanced, the white balance after interpolation multiplies the interpolation error in the chrominance data by a potentially large factor, resulting in large chromatic shifts.

In the particular case of AHD, the interpolation uses color differences G-R and G-B; while the white balance is a multiplicative shift G -> G, B -> c*B.  Typical scenes with not a lot of saturation, G-B is smoothly varying and small in amplitude (it is akin to the Cb channel of YCrCb color space).  If the data is not white balanced, then AHD is interpolating RAW data with chromaticity G-B/c.  If c is large and G~B ie approximately gray, AHD is trying to interpolate data that looks to it as though the chromaticity is very large and rapidly varying across edges.  Of course it's all fake but AHD doesn't know that, and so its errors have large (fake) chromatic content along edges, that persists when the white balance is finally done after demosaic.  It ultimately traces to the mismatch between chromaticity being additive while white balance is multiplicative.  The order of operations matters when considering error propagation.

That's why, again, it's best to do as many corrections as possible while the data is in a linear space, ie prior to demosaic.  Though I imagine Jonathan remains unconvinced.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 05, 2009, 06:12:02 pm
Quote from: GLuijk
I know, but it allows not to white balance the RAW file? and it allows not to convert to an output colour profile?
My ACR doesn't.

Open Bridge.

Select a RAW.

Open RAW in ACR.

Click the Save button.

Select DNG as the output format.

Check the linear option.

The file is saved as a demosaiced linear RGB, no white balancing. The conversion settings you choose (white balance, etc) are not applied to the data, only stored as metadata tags. The only thing done to the RAW data is that it is scaled 0-65535 regardless of the number of bits/sample from the camera, and each Bayer RAW pixel value has two interpolated values added to it to create an RGB pixel value. You still have total flexibility to white balance as you choose, color processing is identical to the original RAW, you can select whatever output color space ACR supports. If you compare a conversion of the linear DNG to a conversion of the original RAW with the same settings, they will match exactly.

Which is why I don't buy the whole "WB and CA correction must be done before demosaic" argument. If that was true, the conversion of the linear DNG would have color artifacts and other problems not found in the direct RAW conversion. But don't take my word for it; I have two DNG files created from the same original RAW. The first link is to the normal, non-demosaiced DNG. The second link is to the demosaiced linear RGB DNG version of the same RAW. Open them both in ACR, and do a side-by-side comparison converting both with the same settings. The results will be identical, at least if you use the same version of ACR used to convert the linear DNG. (I'm still running CS2.)

http://www.visual-vacations.com/images/200..._0004-Bayer.dng (http://www.visual-vacations.com/images/2009/2009-10-05_0004-Bayer.dng)
http://www.visual-vacations.com/images/200...0004-LinRGB.dng (http://www.visual-vacations.com/images/2009/2009-10-05_0004-LinRGB.dng)

For downloading convenience, it may be better to download only the Bayer DNG and convert your own linear version for comparison purposes. Or use one of your own RAWs and skip the download thing altogether.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 05, 2009, 06:33:17 pm
Quote from: ejmartin
In the particular case of AHD, the interpolation uses color differences G-R and G-B; while the white balance is a multiplicative shift G -> G, B -> c*B.  Typical scenes with not a lot of saturation, G-B is smoothly varying and small in amplitude (it is akin to the Cb channel of YCrCb color space).  If the data is not white balanced, then AHD is interpolating RAW data with chromaticity G-B/c.  If c is large and G~B ie approximately gray, AHD is trying to interpolate data that looks to it as though the chromaticity is very large and rapidly varying across edges.  Of course it's all fake but AHD doesn't know that, and so its errors have large (fake) chromatic content along edges, that persists when the white balance is finally done after demosaic.  It ultimately traces to the mismatch between chromaticity being additive while white balance is multiplicative.  The order of operations matters when considering error propagation.

That's why, again, it's best to do as many corrections as possible while the data is in a linear space, ie prior to demosaic.  Though I imagine Jonathan remains unconvinced.

Obviously, ACR is using an algorithm far less stupid than what you are describing; the proof is in the DNGs in my previous post. A strategy needed to work around a limitation imposed by one particular stupid demosaic algorithm does not generalize to iron-clad proof that the work-around is necessary or even desirable in all cases for all demosaic algorithms. Look at the reflection of the umbrella in the baby's right eye. There's a small amount of CA visible. It is fixable with the lens correction sliders in ACR set at about -5,+7. The correction is equally effective in both versions of the DNG, even though the linear version has demosaicing done before white balancing. If your argument was valid, the Bayer version would be more correctable than the linear RGB version of the DNG.
Title: Seeking DNG SDK Assistance
Post by: joofa on November 05, 2009, 06:53:10 pm
Just a clarity on terminology first: By white balance I understand that only the primaries are adjusted and no cross-term multiplication happens yet (as in a color correction matrix). In my previous job when we would make specialized HD video cameras we used to do white balance before deMosaic for several reasons, so I would concur with Emil on this one, and, there were other reasons also for doing it before deMosaic. Additionally, there were some operations that we did relegate after deMosaic, so we did not want to do every operation before deMosaic.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 05, 2009, 07:05:58 pm
Quote from: joofa
Just a clarity on terminology first: By white balance I understand that only the primaries are adjusted and no cross-term multiplication happens yet (as in a color correction matrix).

White balance as I'm using the term is just multiplying all linear color channel values by a fixed value (all B values by 1.105 and all R values by 1.027, for example).
Title: Seeking DNG SDK Assistance
Post by: joofa on November 05, 2009, 07:14:34 pm
Quote from: Jonathan Wienke
White balance as I'm using the term is just multiplying all linear color channel values by a fixed value (all B values by 1.105 and all R values by 1.027, for example).

Okay, your and mine terminology is the same. I just find that term "white balancing" so confusing, because, people use it for both a diagonal matrix and a full CCM matrix.
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 05, 2009, 10:19:54 pm
Quote from: Jonathan Wienke
Obviously, ACR is using an algorithm far less stupid than what you are describing; the proof is in the DNGs in my previous post. A strategy needed to work around a limitation imposed by one particular stupid demosaic algorithm does not generalize to iron-clad proof that the work-around is necessary or even desirable in all cases for all demosaic algorithms. Look at the reflection of the umbrella in the baby's right eye. There's a small amount of CA visible. It is fixable with the lens correction sliders in ACR set at about -5,+7. The correction is equally effective in both versions of the DNG, even though the linear version has demosaicing done before white balancing. If your argument was valid, the Bayer version would be more correctable than the linear RGB version of the DNG.

Accepting for the moment what you are saying, all it demonstrates is the particular set of choices Adobe has made.  If you think that the programming of ACR is optimal for image quality, I've got news for you.  Many choices for the programming of ACR seem to be made to fit an overarching philosophy of speed of processing and convenience, rather than IQ.  For instance, if WB is held until after demosaic, then moving the CA slider does not require redoing demosaic; definitely a convenience, since demosaic delays screen update by several seconds.  The current demosaic engine in ACR has all sorts of chromatic artifacting; your observation might explain why that is so.  ACR does a lot of smearing of chroma data to cover it up; that smearing starts to fail in the presence of noise, which one can see in any high ISO conversion of ACR 5.x and before.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 06, 2009, 07:54:52 am
Quote from: ejmartin
Accepting for the moment what you are saying, all it demonstrates is the particular set of choices Adobe has made.  If you think that the programming of ACR is optimal for image quality, I've got news for you.  Many choices for the programming of ACR seem to be made to fit an overarching philosophy of speed of processing and convenience, rather than IQ.

Your examples to demonstrate this assertion haven't exactly been overwhelmingly persuasive. As a counterexample to the church tower, I give the baby's hair in my RAW; there's no color artifacting in it anywhere.

BTW, there appears to be a gaping hole in your mathematical argument:

Quote
If the data is not white balanced, then AHD is interpolating RAW data with chromaticity G-B/c. If c is large and G~B ie approximately gray, AHD is trying to interpolate data that looks to it as though the chromaticity is very large and rapidly varying across edges.

IOW, if B/c is much smaller than G, then color artifacting is inevitable, and this must be avoided by applying WB to the RAW data prior to demosaic to keep B/c as close to G as possible. You're ignoring the possibility that there can be a large variation between the scaling of color channels even when the desired WB is selected. if your assertion is correct, if you do not choose a perfectly neutral white balance (perhaps to enhance the character of the light in a sunset image or retain the colors of the stage lighting at a concert), then high-contrast-edge color artifacts will become a problem in direct proportion to the non-neutrality of the selected WB setting. Setting a deliberately "wrong" WB is no different than deferring the selection of the "right" WB until after demosaic. And shooting subjects with highly saturated colors (such as a blue flower or red laser beam on a neutral black background) can result in edges and fine details where adjacent Bayer pixel values diverge widely in the manner you described in your example, even if the RAW data has a perfectly neutral WB applied.

(G-B/c) is going to fluctuate just as much when B is small and c~1 as when G~B and c is "large". If your demosaic algorithm requires WB to be done first to avoid situations where B/c diverges greatly from G, it's going to have major problems handling situations where B/c is small and G is large even after WB. Any strategy you employ to avoid color artifacting in this instance can also be used to make pre-demosaic WB unnecessary.
Title: Seeking DNG SDK Assistance
Post by: deejjjaaaa on November 06, 2009, 11:56:14 am
Quote from: ejmartin
ACR does a lot of smearing of chroma data to cover it up; that smearing starts to fail in the presence of noise, which one can see in any high ISO conversion of ACR 5.x and before.

do you see that corrected in LR3B (and ACR v6.x when released) ? there will be new DNG converter by then too, may be w/ changes.
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 06, 2009, 12:04:49 pm
Quote from: deja
do you see that corrected in LR3B (and ACR v6.x when released) ? there will be new DNG converter by then too, may be w/ changes.

I've only played with LR3ß a little; what I've seen so far looks to be a substantial improvement over previous versions of Adobe's conversion engine.  High ISO in particular looks a lot better, especially the chroma blotching is gone.
Title: Seeking DNG SDK Assistance
Post by: joofa on November 06, 2009, 12:07:12 pm
I must stress that adhering to strict theory is not always the goal in industry. The more important issue is that the image looks pleasing to eye even if violates some linearity considerations, etc. Color correction via CCMs is done frequently on non-linear (gamma-corrected) data with coefficients that are derived from linear theory as it offers some advantages in color fidelity near saturated colors. IIRC even some SMPTE specs have "incorrect" coefficients and perhaps primary placements for such reasons and historic considerations.
Title: Seeking DNG SDK Assistance
Post by: Guillermo Luijk on November 08, 2009, 07:09:37 am
Quote from: Jonathan Wienke
Open Bridge.

Select a RAW.

Open RAW in ACR.

Click the Save button.

Select DNG as the output format.

Check the linear option.

Jesus, I had no idea my Bridge could do DNG conversions, either with and without demosaic. Good to learn new things. I am quite surprised ACR didn't generate colour artifacts when doing a non white balanced demosaicing since I always thought it was based in the AHD algorithm. However it seems to produce less sharp results:

(http://img5.imageshack.us/img5/7494/samplesl.jpg)


Anyway Jon I think there is no need to be so agressive in some of your comments to Emil's points. We are all enjoying and sharing information.

Regards.
Title: Seeking DNG SDK Assistance
Post by: ejmartin on November 08, 2009, 09:17:45 am
I suspect the DNG conversion in Bridge is the same as in DNG Converter stand-alone, in which case Eric gave the answer above:

Quote from: madmanchan
Hi Sandy, yes, you are right: the DNG SDK only has a simple bilinear interpolation demosaic routine implemented. (We have been meaning to update it with something more serviceable, but have not worked that into the already rather-packed dev schedule ...)


That would explain why it's softer.  Now, bilinear interpolation isn't exactly a linear operation, but to the extent that it is WB before or after wouldn't matter since linear operations commute with rescaling operations.  Also, one would have to know the internal workings of a converter to know whether WB is being done before demosaic -- who's to say that the WB multiplier isn't applied and then undone after demosaic, so that you are free to WB afterward in ACR/LR?    Well I suppose Eric could say...

I was incorrect before in suggesting that doing WB before demosaic would require redoing demosaic every time the WB is changed.  There is presumably an optimal value, that one can apply and then un-apply afterward, then the WB slider can act on un-WB data that has had the optimal demosaic, that need not be redone.

Jonathan, there is a difference between the WB multiplier, which is a constant, and the RGB mosaic data, which is varying across the image.
Title: Seeking DNG SDK Assistance
Post by: sandymc on November 08, 2009, 01:42:04 pm
Quote from: ejmartin
I suspect the DNG conversion in Bridge is the same as in DNG Converter stand-alone, in which case Eric gave the answer above:




That would explain why it's softer.  Now, bilinear interpolation isn't exactly a linear operation, but to the extent that it is WB before or after wouldn't matter since linear operations commute with rescaling operations.  Also, one would have to know the internal workings of a converter to know whether WB is being done before demosaic -- who's to say that the WB multiplier isn't applied and then undone after demosaic, so that you are free to WB afterward in ACR/LR?    Well I suppose Eric could say...

I was incorrect before in suggesting that doing WB before demosaic would require redoing demosaic every time the WB is changed.  There is presumably an optimal value, that one can apply and then un-apply afterward, then the WB slider can act on un-WB data that has had the optimal demosaic, that need not be redone.

Jonathan, there is a difference between the WB multiplier, which is a constant, and the RGB mosaic data, which is varying across the image.

I think that what Eric was saying is just that demosaicing in the DNG converter is not the same as ACR or bridge.

Re the white balance question, it depends on how you're doing white balance. AHD type algorithms are dependent on being able to measure "distance" in color space. If your change to white balance changes distance, then yes, you have to do the demosaic again, at least in theory. Eric has stated in the past that ACR/LR does NOT run a a AHD algorithm. Although what that means isn't too clear - there is a school of thought that ACR/LR runs a AHD derivative, but in a non-RGB space. I don't think there's much evidence one way or the other, but that would explain why ACR/LR is subject to maze type artifacts. However, we do know that in an Adobe workflow, white balance is before demosaicing - that's why a DNG has two color matrixes.

Sandy

Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 08, 2009, 06:55:02 pm
Quote from: GLuijk
Jesus, I had no idea my Bridge could do DNG conversions, either with and without demosaic. Good to learn new things. I am quite surprised ACR didn't generate colour artifacts when doing a non white balanced demosaicing since I always thought it was based in the AHD algorithm. However it seems to produce less sharp results:

I'm not excessively concerned about getting perfect sharpness directly after demosaic as long as there aren't any color artifacts and suchlike; the goal of the deconvolution I'm doing is to correct the sum of all blurring from the lens, AA filter, and RAW converter combined.
Title: Seeking DNG SDK Assistance
Post by: Jonathan Wienke on November 22, 2009, 10:36:49 am
If anyone has feature requests for my program, please post them here:

http://luminous-landscape.com/forum/index....showtopic=39453 (http://luminous-landscape.com/forum/index.php?showtopic=39453)
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 15, 2012, 12:21:26 pm
However, we do know that in an Adobe workflow, white balance is before demosaicing - that's why a DNG has two color matrixes.

Sandy

SandyMC, old topic I was rereading - Eric Chan said before ( http://www.luminous-landscape.com/forum/index.php?topic=25031.msg193382#msg193382 ) :

"...FYI, the DNG processing model performs a linearization of the original raw image values followed by demosaicing, then white balance. All of the other image stages follow. So to answer your question, all of the image ops except for linearization (which isn't under user control anyways) happens after demosaicing..."
Title: Re: Seeking DNG SDK Assistance
Post by: samueljohnchia on September 15, 2012, 09:17:48 pm
Hey guys, what about the DNG profiles that are used for camera calibration? Are those applied after demosaicing or before?
Title: Re: Seeking DNG SDK Assistance
Post by: sandymc on September 16, 2012, 02:22:14 am
SandyMC, old topic I was rereading - Eric Chan said before ( http://www.luminous-landscape.com/forum/index.php?topic=25031.msg193382#msg193382 ) :

"...FYI, the DNG processing model performs a linearization of the original raw image values followed by demosaicing, then white balance. All of the other image stages follow. So to answer your question, all of the image ops except for linearization (which isn't under user control anyways) happens after demosaicing..."

Yes, I wasn't precise. As I understand it, "white balance" data has two roles in the Adobe image pipeline. The first role is the usual one, which is to set the white balance of the image, and happens after demosaic, the same as for a "normal" image pipeline. So yes, the white balance operation occurs after demosaic.

However, white balance data may be used before demosaic as well: The second role that white balance has is in an Adobe pipeline is to avoid metamerism, etc, where Adobe use two color matrixes and interpolate them based on the color temperature. That part may/probably(?) occurs before/during demosaic in order to get an accurate measure of distance in color space for use in the demosaic process itself. Assuming they're using a demosaicing technique that needs that. (Big assumption - at the time I wrote the response above, in 2009, I was fairly sure they did. Circa 2012, I'm not nearly as sure.)

Sandy
Title: Re: Seeking DNG SDK Assistance
Post by: sandymc on September 16, 2012, 02:24:44 am
Hey guys, what about the DNG profiles that are used for camera calibration? Are those applied after demosaicing or before?

Maybe with the exception of some white balance related stuff that feeds the demosaic process as above, DNG profiles are applied after demosaic.

Sandy
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 16, 2012, 11:32:27 am
Yes, I wasn't precise. As I understand it, "white balance" data has two roles in the Adobe image pipeline. The first role is the usual one, which is to set the white balance of the image, and happens after demosaic, the same as for a "normal" image pipeline. So yes, the white balance operation occurs after demosaic.

However, white balance data may be used before demosaic as well: The second role that white balance has is in an Adobe pipeline is to avoid metamerism, etc, where Adobe use two color matrixes and interpolate them based on the color temperature. That part may/probably(?) occurs before/during demosaic in order to get an accurate measure of distance in color space for use in the demosaic process itself. Assuming they're using a demosaicing technique that needs that. (Big assumption - at the time I wrote the response above, in 2009, I was fairly sure they did. Circa 2012, I'm not nearly as sure.)

Sandy

interpolation between 2 color matrices shall depend on the particular WB that we select in UI, right ?

change in WB by you in UI shall lead to reinterpolation between 2 color matrices in ACR/LR, it can't be a fixed interpolation - can it ?


and demosaick as said by Eric happens before WB, "the usual one, which is to set the white balance of the image", then technically, you need to do the whole stuff again and again when WB sliders are changed in UI...

you need to do matrix multiplication for the whole raw data using a reinterpolated matrix, demosaick and regular WB operation after that for each slider move...

at the same time we have linear DNG (demosaick happened and for example done by genuine ACR algorithm, not by crippled /as Eric said/ DNG converter code) produced by ACR and WB in ACR on that file (linear DNG) gives exactly the same image as WB in ACR on the source raw file... so either there are no reinterpolation or everything is done on demosaicked data.

please tell me where I am wrong, because I probably am.
Title: Re: Seeking DNG SDK Assistance
Post by: sandymc on September 16, 2012, 12:03:39 pm
interpolation between 2 color matrices shall depend on the particular WB that we select in UI, right ?

change in WB by you in UI shall lead to reinterpolation between 2 color matrices in ACR/LR, it can't be a fixed interpolation - can it ?


and demosaick as said by Eric happens before WB, "the usual one, which is to set the white balance of the image", then technically, you need to do the whole stuff again and again when WB sliders are changed in UI...

you need to do matrix multiplication for the whole raw data using a reinterpolated matrix, demosaick and regular WB operation after that for each slider move...

at the same time we have linear DNG (demosaick happened and for example done by genuine ACR algorithm, not by crippled /as Eric said/ DNG converter code) produced by ACR and WB in ACR on that file (linear DNG) gives exactly the same image as WB in ACR on the source raw file... so either there are no reinterpolation or everything is done on demosaicked data.

please tell me where I am wrong, because I probably am.


My understanding circa 2009 from Eric was that yes, if you changed white balance, LR went all the way back to demosaic to rerender the image. Seemed entirely over the top to me at the time, and still does, but I've never seen a concrete example of dual illuminant making any significant different to anything(!). Maybe I misunderstood what Eric was saying.  :o

As regards linear DNG vs raw - the changes would be very, very subtle, only a pixel or two slightly different. The matrixes (should) have no impact on white balance in the sense of warmer or cooler, the changes would just be e.g., in a possibly different choice of vertical or horizontal in the demosaic step.

Regards,

Sandy
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 16, 2012, 12:44:49 pm
My understanding circa 2009 from Eric was that yes, if you changed white balance, LR went all the way back to demosaic to rerender the image. Seemed entirely over the top to me at the time, and still does, but I've never seen a concrete example of dual illuminant making any significant different to anything(!). Maybe I misunderstood what Eric was saying.  :o

you mean output from using dual illuminants vs single illumant profiles ? like you have a dual illuminant profile, you manually remove one matrix and voila - not impact ?


As regards linear DNG vs raw - the changes would be very, very subtle, only a pixel or two slightly different.

first of all - how come that very significant changes in WB in UI in ACR/LR will lead to only a pixel or two different and then - I do not see even a single pixel changed... granted I did not run that on thousands of files, but do you have a guess wha are the conditions, target or camera model where that can be seen to have a direct test to see a pixel at least ?


The matrixes (should) have no impact on white balance in the sense of warmer or cooler

you mean no impact on neither "temperature" : B (colder) - GR (warmer) nor tint : G - BR, right ?

, the changes would just be e.g., in a possibly different choice of vertical or horizontal in the demosaic step.

that sounds a little bit an overkill to use matrices to just make such a binary decision and then - how in the world ACR knows when producing a linear DNG file (means making such a decision before human being) knows which WB we shall end up in UI afterwards (if that selection of WB actually affects the step in demosaicking).
Title: Re: Seeking DNG SDK Assistance
Post by: sandymc on September 16, 2012, 01:07:28 pm
you mean output from using dual illuminants vs single illumant profiles ? like you have a dual illuminant profile, you manually remove one matrix and voila - not impact ?


first of all - how come that very significant changes in WB in UI in ACR/LR will lead to only a pixel or two different and then - I do not see even a single pixel changed... granted I did not run that on thousands of files, but do you have a guess wha are the conditions, target or camera model where that can be seen to have a direct test to see a pixel at least ?


you mean no impact on neither "temperature" : B (colder) - GR (warmer) nor tint : G - BR, right ?

that sounds a little bit an overkill to use matrices to just make such a binary decision and then - how in the world ACR knows when producing a linear DNG file (means making such a decision before human being) knows which WB we shall end up in UI afterwards (if that selection of WB actually affects the step in demosaicking).


I haven't seen a difference between dual and single illuminants in regards of demosaicing, no.

The conditions would be a edge between colors.

Yes on temperature/tint.

Completely agree on overkill......

Regards,

Sandy

Title: Re: Seeking DNG SDK Assistance
Post by: madmanchan on September 20, 2012, 11:02:24 am
The current ACR/LR demosaic does not depend on the choice of color profile or white balance.  (You can check this for yourself; if you change color profiles or white balance, the fundamental demosaic results do not change.)

The purpose of the dual-illuminant profile, like Sandy said, is to deal with different color response of sensors under different illumination.  Even such profiles can fail if shooting under a light that is spectrally rather different than the calibration lights (e.g., compact fluorescent vs tungsten).  Another approach is to build dedicated separate (single-illuminant) profiles for the illumination conditions you care about, though in your workflow that would mean you would have to select the profile manually.  This may not be a practical issue since one can use presets, sync settings, etc. to speed things up.
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 20, 2012, 10:13:19 pm
The current ACR/LR demosaic does not depend on the choice of color profile or white balance.  (You can check this for yourself; if you change color profiles or white balance, the fundamental demosaic results do not change.)

that is exactly what you (Eric) said before in quoted post - demosaick happens before any white balance, hence white balance can't affect demosaick by definition... and in such design you do not need to redo demosaicking when you change the WB in user interface... that is much faster and allows things like linear DNG... for example if you produce a linear DNG in ACR from some raw file (does not matter whether it is native in camera DNG or proprietary) then the output using the same version of ACR and any identical WB (and other parameters in UI are identical too) will be same for both conversion from that linear DNG and its original raw file, right ?

Title: Re: Seeking DNG SDK Assistance
Post by: madmanchan on September 21, 2012, 02:57:01 am
Yes.

(One clarification: linear DNG is not yet white-balanced, so you are free to change the WB later even for a linear DNG in the same way you are accustomed to with regular mosaic raw.)
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 21, 2012, 11:12:38 am
and your note about color (=.dcp camera profile) profiles basically means that any "color" transforms  (from camera's RGB numbers to ACR's internal working space) are happening after demosaicking as well, hence no matter what do you do and how do you create a profile (with or without Adobe's DNG PE - even if you type it yourself manually and compile using Sandy's DCP compiler/decompiler) it will not affect the demosaick itself (in camera's own RGB numbers, that what "the fundamental demosaic results" actually means)...

PS: about BayerGreenSplit tag - why this is not user accessible parameter ? I saw a numbers of cases when people long after ACR version release send you a sample raw file for you to correct maze artefacts... isn't it more simple to put it somewhere in UI in a camera calibration tab for example ?
Title: Re: Seeking DNG SDK Assistance
Post by: Nigel Johnson on September 24, 2012, 12:28:18 pm
...not by crippled /as Eric said/ DNG converter code...

Eric did NOT say that the DNG Converter used a bilinear interpolation demosaic routine, rather he stated that the DNG SDK used the simpler routine. Hence he did NOT say that the DNG Converter code was crippled.

Regards,
Nigel
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 24, 2012, 12:59:56 pm
Eric did NOT say that the DNG Converter used a bilinear interpolation demosaic routine, rather he stated that the DNG SDK used the simpler routine. Hence he did NOT say that the DNG Converter code was crippled.

Regards,
Nigel

let us just hope that Eric will clarify that, because I am pretty sure that in another topic it was said by him that DNG converter (not public DNG SDK) was using not the same demosaick as ACR/LR... 
Title: Re: Seeking DNG SDK Assistance
Post by: sandymc on September 24, 2012, 01:25:26 pm
Short summary:

ACR/LR > DNG converter > DNG SDK

 :)

Sandy
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 24, 2012, 01:34:33 pm
Short summary:

ACR/LR > DNG converter > DNG SDK

 :)

Sandy

exactly... I am at work now and I can't find a reference to the Eric's post, but hopefully I will...
Title: Re: Seeking DNG SDK Assistance
Post by: Nigel Johnson on September 24, 2012, 03:02:03 pm
Short summary:

ACR/LR > DNG converter > DNG SDK

Sandy,

This is not what Eric says, see the following quote (note that the bold text is my highlighting and not in the original message):

Yes, DNG is a documented image file format (basically an extension of TIFF) that can be used by anybody, but there have been revisions to it since its introduction. Just like there are multiple versions of JPEG, TIFF, MPEG, etc.

Yes, the DNG Converter will allow you to process a new raw image in an older version of ACR.

It is true that the public DNG SDK only contains a simple bilinear demosaic implementation, but the DNG Converter does not use this.

For operations that require a demosaic step (e.g., conversion to 3-channel RGB), DNG Converter uses the same demosaic algorithm as Camera Raw, assuming the same version. For example, DNG Converter 6.2 and Camera Raw 6.2 use the same demosaic algorithm. Similarly, DNG Converter 4.4 and Camera Raw 4.4 use the same (older) demosaic algorithm. Newer versions of DNG Converter offer more camera support, have bug fixes, and improved features over earlier versions. For this reason, I generally recommend using the latest available version of DNG Converter (currently 6.2, soon to be 6.3), regardless of which version of Camera Raw you are using.

Regards,
Nigel
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 24, 2012, 03:56:48 pm
Sandy,

This is not what Eric says, see the following quote


and I was not referring to that quote, which deals w/ SDK - but to another topic completely ( probably @ http://forums.adobe.com/community/cameraraw )
Title: Re: Seeking DNG SDK Assistance
Post by: Nigel Johnson on September 24, 2012, 04:24:55 pm
and I was not referring to that quote, which deals w/ SDK - but to another topic completely ( probably @ http://forums.adobe.com/community/cameraraw )

The quote deals with both the SDK and with the DNG Converter and Camera Raw (the second bold highlight). I accept that you may have another quote from elsewhere.

Nigel
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 24, 2012, 05:12:28 pm
The quote deals with both the SDK and with the DNG Converter and Camera Raw (the second bold highlight). I accept that you may have another quote from elsewhere.

Nigel

Again I might be wrong - the subject that was discussed (I am pretty sure that was later then the quote from this topic) is if Adobe DNG converter use the exact ACR/LR code when creating a preview in DNG file and the only difference between DNG converter and ACR/LR is that DNG Converter will create a JPG file (so you can't controll compression options, color space, etc) but everything else - demosaick, NR, local edits, etc - everything is the same - so you can remove preview from DNG file, take DNG converter and if you have xmp or the same embedded in DNG file then you can recreate that preview exactly as ACR/LR 'd do... and I was guessing that yes, indeed and I am pretty sure that Eric Chan said - no, demosaick (at least) is not the same... again I can't find a quote at the moment - but I am just trying to explain the context of discussion back then.
Title: Re: Seeking DNG SDK Assistance
Post by: deejjjaaaa on September 25, 2012, 10:23:19 am
I am happy to report that I was wrong and my memory failed me  :( !

http://forums.adobe.com/message/4725070#4725070

Quote
I believe that's a miscommunication.  The DNG Converter does use the same demosaic processing as ACR/LR (assuming matching version).  It's the public DNG SDK that uses a different (simpler) demosaic processing.

Yes, producing a linear DNG, then re-loading that DNG into ACR and processing it, will produce the same results, assuming all the relevant metadata is intact.