Pages: 1 [2] 3 4   Go Down

Author Topic: Seeking DNG SDK Assistance  (Read 44884 times)

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Seeking DNG SDK Assistance
« Reply #20 on: November 04, 2009, 02:54:11 am »

Quote from: Jonathan Wienke
Only if you consider ACR a "subpar" RAW converter. And you're not necessarily limited to ACR; any RAW converter that can export to linear-RGB DNG can be used for demosaicing. The majority of the lens corrections have to be done on demosaiced RGB data; for example, CA corrections (which involve shifting the locations of color channels relative to each other) cannot be saved back to RAW because the pixel locations no longer match the Bayer matrix pattern. Correcting barrel/fisheye distortion has the same problem, only worse because the degree of pixel-shifting es even greater.

It just makes sense to demosaic first, and then apply corrections and adjustments.

Actually, CA correction can be done, and done better, during demosaic rather than after.

http://www.ojodigital.com/foro/perfectraw-...erpolacion.html

Even so, geometric corrections might be better done after demosaic.   I'll be interested to see what comes out of your project.
« Last Edit: November 04, 2009, 02:57:51 am by ejmartin »
Logged
emil

madmanchan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2115
    • Web
Seeking DNG SDK Assistance
« Reply #21 on: November 04, 2009, 08:56:11 am »

Hi Sandy, yes, you are right: the DNG SDK only has a simple bilinear interpolation demosaic routine implemented. (We have been meaning to update it with something more serviceable, but have not worked that into the already rather-packed dev schedule ...)
Logged
Eric Chan

sandymc

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 350
Seeking DNG SDK Assistance
« Reply #22 on: November 04, 2009, 09:25:52 am »

Quote from: madmanchan
Hi Sandy, yes, you are right: the DNG SDK only has a simple bilinear interpolation demosaic routine implemented. (We have been meaning to update it with something more serviceable, but have not worked that into the already rather-packed dev schedule ...)

Hi Eric - I can sympathize with the dev schedule!!!!

Even a "text book" AHD implementation would be really cool though......and make doing things such as the OP wants to do a lot easier.

Regards,

Sandy
Logged

deejjjaaaa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1170
Seeking DNG SDK Assistance
« Reply #23 on: November 04, 2009, 10:39:30 am »

Quote from: Jonathan Wienke
The testing I've done indicates that opening a linear-RGB DNG (demosaiced in DNG converter) and processing the source RAW directly in ACR makes zero difference

well, too bad for ACR then...
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Seeking DNG SDK Assistance
« Reply #24 on: November 04, 2009, 11:17:34 am »

Quote from: ejmartin
Actually, CA correction can be done, and done better, during demosaic rather than after.

http://www.ojodigital.com/foro/perfectraw-...erpolacion.html

The samples in that thread didn't look that impressive to me; in the side-by-side comparison of the makeup brushes, AMAZE didn't appear to do as well as the other converter it was compared to. Correcting CA during demosaic may not be worse than doing so after, but I didn't see any proof it's significantly better.
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Seeking DNG SDK Assistance
« Reply #25 on: November 04, 2009, 12:15:33 pm »

Quote from: Jonathan Wienke
The samples in that thread didn't look that impressive to me; in the side-by-side comparison of the makeup brushes, AMAZE didn't appear to do as well as the other converter it was compared to. Correcting CA during demosaic may not be worse than doing so after, but I didn't see any proof it's significantly better.

The side-by-side with the brushes is a comparison to AMaZE prior to CA correction, the CA corrected version is lower down in the thread (post #11).  

Here's a direct comparison with ACR at 400% (using a different image).  I set the ACR 4.6 CA correction slider so as to minimize CA artifacting in the blinds (there was no setting that eliminated it altogether), and it still crops up in the church steeple and the tree limbs/branches.  Anyway, regardless of the specific implementation, it seems to me that any geometric correction that can be applied in a linear space such as RAW is better done there rather than after the nonlinear operation of demosaic.   For instance, CA can corrupt the demosaic process itself, depending on the algorithm used (cf the HPHD examples from RawTherapee in the Ojodigital thread).
« Last Edit: November 04, 2009, 07:03:36 pm by ejmartin »
Logged
emil

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Seeking DNG SDK Assistance
« Reply #26 on: November 04, 2009, 07:19:46 pm »

Quote from: Jonathan Wienke
All of the internal correction operations are done on linear RGB data in the camera's native color space, so that setting white balance, selection of color profile, etc. is unaffected by the image correction operation. The user will have the exact same color processing options, ability to select white balance, exposure adjustments, and output color space selection as when working with the original RAW(s).
Jon, if I understood well you plan to demosaic the RAW data with the DNG routines, and that means you should apply white balance first. You can demosaic with no WB applied, but this will probably affect the efficiency of the demosaic process in terms of getting right colours and detail.

On the other hand, applying a WB doesn't mean you cannot modify it later, but you will have to deal carefully with the highlights in the white balancing process: to preserve them will mean you will move the saturation point to some unknown (or difficult to know) place in the histogram. And this point needs to be known if additional exposure or white balance adjustments are to be done to the output DNG file.

BTW if you would like any advice on how to blend different RAW files (relative exposure calculation, and minimal progresiveness blending) feel free to ask.

Regards

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Seeking DNG SDK Assistance
« Reply #27 on: November 04, 2009, 07:51:35 pm »

Quote from: ejmartin
The side-by-side with the brushes is a comparison to AMaZE prior to CA correction, the CA corrected version is lower down in the thread (post #11).  

Here's a direct comparison with ACR at 400% (using a different image).  I set the ACR 4.6 CA correction slider so as to minimize CA artifacting in the blinds (there was no setting that eliminated it altogether), and it still crops up in the church steeple and the tree limbs/branches.  Anyway, regardless of the specific implementation, it seems to me that any geometric correction that can be applied in a linear space such as RAW is better done there rather than after the nonlinear operation of demosaic.   For instance, CA can corrupt the demosaic process itself, depending on the algorithm used (cf the HPHD examples from RawTherapee in the Ojodigital thread).

I'm still not convinced that correcting CA during demosaic is the magic bullet you think it is. Other than the CA, the differences in the conversions were minimal, mostly due to slightly different tone curve or exposure settings.

The problem with ACR and most every other RAW converter's CA correction is that it's using a fairly simple mathematical formula (something along the lines of a simple gamma curve) to calculate the pixel shift distance needed to realign the color channels and correct the CA. The problem is that this formula is an approximation many lenses do not follow very well; therefore for many lenses there is no setting that creates the proper curve to correct the CA completely. The best you can do is spread the error fairly evenly so that no part of the image looks too horrible. The approach I'm using creates 3 spline curves with 32-64 data points each (calculated separately for each color channel) so that the color channels can be aligned to within 1/10 of a pixel or so from the center of the image all the way to the corners, regardless of whether the lens' distortion curve is simple or extremely complex.
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Seeking DNG SDK Assistance
« Reply #28 on: November 04, 2009, 08:32:38 pm »

Quote from: Jonathan Wienke
I'm still not convinced that correcting CA during demosaic is the magic bullet you think it is. Other than the CA, the differences in the conversions were minimal, mostly due to slightly different tone curve or exposure settings.

Most differences between good conversions are minimal, up to tone curve and color space conversion.  We're talking about the last little bit of data near Nyquist, otherwise there isn't much to discuss.

Quote
The problem with ACR and most every other RAW converter's CA correction is that it's using a fairly simple mathematical formula (something along the lines of a simple gamma curve) to calculate the pixel shift distance needed to realign the color channels and correct the CA. The problem is that this formula is an approximation many lenses do not follow very well; therefore for many lenses there is no setting that creates the proper curve to correct the CA completely. The best you can do is spread the error fairly evenly so that no part of the image looks too horrible. The approach I'm using creates 3 spline curves with 32-64 data points each (calculated separately for each color channel) so that the color channels can be aligned to within 1/10 of a pixel or so from the center of the image all the way to the corners, regardless of whether the lens' distortion curve is simple or extremely complex.

There isn't much point in aligning to 1/10 pixel spacing when 2/3 of the data has been interpolated in different directions using colors that are as much as a pixel or more away from where they should be.  No mathematical formula with a small number of parameters is going to correct for the fact that a color has been interpolated incorrectly in the horizontal direction in one place, and incorrectly in the vertical direction nearby, due to the mismatched color registration caused by CA.  That is why it is important to do the CA correction before demosaic.  

Any formula for correction that you may have can be applied before demosaic with the right approach.
« Last Edit: November 04, 2009, 08:33:28 pm by ejmartin »
Logged
emil

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Seeking DNG SDK Assistance
« Reply #29 on: November 04, 2009, 10:47:46 pm »

Quote from: GLuijk
Jon, if I understood well you plan to demosaic the RAW data with the DNG routines, and that means you should apply white balance first. You can demosaic with no WB applied, but this will probably affect the efficiency of the demosaic process in terms of getting right colours and detail.

Actually it doesn't. If you export a linear-RGB DNG from DNG Converter or Bridge, and then process that DNG and the original RAW file in ACR with the same settings, the result is a pixel-for-pixel match, and you have exactly the same flexibility for setting exposure, white balance, and color adjustments either way.

Quote
BTW if you would like any advice on how to blend different RAW files (relative exposure calculation, and minimal progresiveness blending) feel free to ask.

Thank you. Would you be interested in perhaps combining our projects, to create one DNG processing tool that does everything?
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Seeking DNG SDK Assistance
« Reply #30 on: November 05, 2009, 02:44:05 pm »

Quote from: Jonathan Wienke
Actually it doesn't. If you export a linear-RGB DNG from DNG Converter or Bridge, and then process that DNG and the original RAW file in ACR with the same settings, the result is a pixel-for-pixel match, and you have exactly the same flexibility for setting exposure, white balance, and color adjustments either way.

But how do you use Bridge to demosaic a non white balanced RAW file?
At least using DCRAW, the AHD algorithm works quite differently if applied on white balanced RAW data than when used on non white balanced data. Colour artifacts and differently interpolated detail appears when the RAW values are not balanced, surely Emil can explain why these issues happen:

White balance prior to demosaicing:


White balance after demosaicing:



Quote from: Jonathan Wienke
Thank you. Would you be interested in perhaps combining our projects, to create one DNG processing tool that does everything?

That's OK, but your workflow is based on demosaiced data; what I would like to achieve is a pure Bayer blend.

Regards
« Last Edit: November 05, 2009, 02:44:39 pm by GLuijk »
Logged

deejjjaaaa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1170
Seeking DNG SDK Assistance
« Reply #31 on: November 05, 2009, 02:56:35 pm »

Quote from: GLuijk
But how do you use Bridge to demosaic a non white balanced RAW file?
Bridge can run ACR directly w/o PS
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Seeking DNG SDK Assistance
« Reply #32 on: November 05, 2009, 03:04:03 pm »

Quote from: deja
Bridge can run ACR directly w/o PS
I know, but it allows not to white balance the RAW file? and it allows not to convert to an output colour profile?
My ACR doesn't.

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Seeking DNG SDK Assistance
« Reply #33 on: November 05, 2009, 04:32:36 pm »

Quote from: GLuijk
But how do you use Bridge to demosaic a non white balanced RAW file?
At least using DCRAW, the AHD algorithm works quite differently if applied on white balanced RAW data than when used on non white balanced data. Colour artifacts and differently interpolated detail appears when the RAW values are not balanced, surely Emil can explain why these issues happen:

White balance prior to demosaicing:
http://www.guillermoluijk.com/tutorial/dcr...tefactoswb1.jpg

White balance after demosaicing:
http://www.guillermoluijk.com/tutorial/dcr...tefactoswb2.jpg

Take for instance the one column of pixels in the pole atop the church dome; it is either GRGRGR data or BGBGBG data.  Either B or R needs to be interpolated from an adjacent column.  If the data is approximately white balanced, the interpolation error is moderate.  If the data is not white balanced, the white balance after interpolation multiplies the interpolation error in the chrominance data by a potentially large factor, resulting in large chromatic shifts.

In the particular case of AHD, the interpolation uses color differences G-R and G-B; while the white balance is a multiplicative shift G -> G, B -> c*B.  Typical scenes with not a lot of saturation, G-B is smoothly varying and small in amplitude (it is akin to the Cb channel of YCrCb color space).  If the data is not white balanced, then AHD is interpolating RAW data with chromaticity G-B/c.  If c is large and G~B ie approximately gray, AHD is trying to interpolate data that looks to it as though the chromaticity is very large and rapidly varying across edges.  Of course it's all fake but AHD doesn't know that, and so its errors have large (fake) chromatic content along edges, that persists when the white balance is finally done after demosaic.  It ultimately traces to the mismatch between chromaticity being additive while white balance is multiplicative.  The order of operations matters when considering error propagation.

That's why, again, it's best to do as many corrections as possible while the data is in a linear space, ie prior to demosaic.  Though I imagine Jonathan remains unconvinced.
« Last Edit: November 05, 2009, 05:52:16 pm by ejmartin »
Logged
emil

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Seeking DNG SDK Assistance
« Reply #34 on: November 05, 2009, 06:12:02 pm »

Quote from: GLuijk
I know, but it allows not to white balance the RAW file? and it allows not to convert to an output colour profile?
My ACR doesn't.

Open Bridge.

Select a RAW.

Open RAW in ACR.

Click the Save button.

Select DNG as the output format.

Check the linear option.

The file is saved as a demosaiced linear RGB, no white balancing. The conversion settings you choose (white balance, etc) are not applied to the data, only stored as metadata tags. The only thing done to the RAW data is that it is scaled 0-65535 regardless of the number of bits/sample from the camera, and each Bayer RAW pixel value has two interpolated values added to it to create an RGB pixel value. You still have total flexibility to white balance as you choose, color processing is identical to the original RAW, you can select whatever output color space ACR supports. If you compare a conversion of the linear DNG to a conversion of the original RAW with the same settings, they will match exactly.

Which is why I don't buy the whole "WB and CA correction must be done before demosaic" argument. If that was true, the conversion of the linear DNG would have color artifacts and other problems not found in the direct RAW conversion. But don't take my word for it; I have two DNG files created from the same original RAW. The first link is to the normal, non-demosaiced DNG. The second link is to the demosaiced linear RGB DNG version of the same RAW. Open them both in ACR, and do a side-by-side comparison converting both with the same settings. The results will be identical, at least if you use the same version of ACR used to convert the linear DNG. (I'm still running CS2.)

http://www.visual-vacations.com/images/200..._0004-Bayer.dng
http://www.visual-vacations.com/images/200...0004-LinRGB.dng

For downloading convenience, it may be better to download only the Bayer DNG and convert your own linear version for comparison purposes. Or use one of your own RAWs and skip the download thing altogether.
« Last Edit: November 05, 2009, 06:16:28 pm by Jonathan Wienke »
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Seeking DNG SDK Assistance
« Reply #35 on: November 05, 2009, 06:33:17 pm »

Quote from: ejmartin
In the particular case of AHD, the interpolation uses color differences G-R and G-B; while the white balance is a multiplicative shift G -> G, B -> c*B.  Typical scenes with not a lot of saturation, G-B is smoothly varying and small in amplitude (it is akin to the Cb channel of YCrCb color space).  If the data is not white balanced, then AHD is interpolating RAW data with chromaticity G-B/c.  If c is large and G~B ie approximately gray, AHD is trying to interpolate data that looks to it as though the chromaticity is very large and rapidly varying across edges.  Of course it's all fake but AHD doesn't know that, and so its errors have large (fake) chromatic content along edges, that persists when the white balance is finally done after demosaic.  It ultimately traces to the mismatch between chromaticity being additive while white balance is multiplicative.  The order of operations matters when considering error propagation.

That's why, again, it's best to do as many corrections as possible while the data is in a linear space, ie prior to demosaic.  Though I imagine Jonathan remains unconvinced.

Obviously, ACR is using an algorithm far less stupid than what you are describing; the proof is in the DNGs in my previous post. A strategy needed to work around a limitation imposed by one particular stupid demosaic algorithm does not generalize to iron-clad proof that the work-around is necessary or even desirable in all cases for all demosaic algorithms. Look at the reflection of the umbrella in the baby's right eye. There's a small amount of CA visible. It is fixable with the lens correction sliders in ACR set at about -5,+7. The correction is equally effective in both versions of the DNG, even though the linear version has demosaicing done before white balancing. If your argument was valid, the Bayer version would be more correctable than the linear RGB version of the DNG.
« Last Edit: November 05, 2009, 06:43:29 pm by Jonathan Wienke »
Logged

joofa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 544
Seeking DNG SDK Assistance
« Reply #36 on: November 05, 2009, 06:53:10 pm »

Just a clarity on terminology first: By white balance I understand that only the primaries are adjusted and no cross-term multiplication happens yet (as in a color correction matrix). In my previous job when we would make specialized HD video cameras we used to do white balance before deMosaic for several reasons, so I would concur with Emil on this one, and, there were other reasons also for doing it before deMosaic. Additionally, there were some operations that we did relegate after deMosaic, so we did not want to do every operation before deMosaic.
« Last Edit: November 05, 2009, 07:20:08 pm by joofa »
Logged
Joofa
http://www.djjoofa.com
Download Photoshop and After Effects plugins

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Seeking DNG SDK Assistance
« Reply #37 on: November 05, 2009, 07:05:58 pm »

Quote from: joofa
Just a clarity on terminology first: By white balance I understand that only the primaries are adjusted and no cross-term multiplication happens yet (as in a color correction matrix).

White balance as I'm using the term is just multiplying all linear color channel values by a fixed value (all B values by 1.105 and all R values by 1.027, for example).
Logged

joofa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 544
Seeking DNG SDK Assistance
« Reply #38 on: November 05, 2009, 07:14:34 pm »

Quote from: Jonathan Wienke
White balance as I'm using the term is just multiplying all linear color channel values by a fixed value (all B values by 1.105 and all R values by 1.027, for example).

Okay, your and mine terminology is the same. I just find that term "white balancing" so confusing, because, people use it for both a diagonal matrix and a full CCM matrix.
Logged
Joofa
http://www.djjoofa.com
Download Photoshop and After Effects plugins

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
Seeking DNG SDK Assistance
« Reply #39 on: November 05, 2009, 10:19:54 pm »

Quote from: Jonathan Wienke
Obviously, ACR is using an algorithm far less stupid than what you are describing; the proof is in the DNGs in my previous post. A strategy needed to work around a limitation imposed by one particular stupid demosaic algorithm does not generalize to iron-clad proof that the work-around is necessary or even desirable in all cases for all demosaic algorithms. Look at the reflection of the umbrella in the baby's right eye. There's a small amount of CA visible. It is fixable with the lens correction sliders in ACR set at about -5,+7. The correction is equally effective in both versions of the DNG, even though the linear version has demosaicing done before white balancing. If your argument was valid, the Bayer version would be more correctable than the linear RGB version of the DNG.

Accepting for the moment what you are saying, all it demonstrates is the particular set of choices Adobe has made.  If you think that the programming of ACR is optimal for image quality, I've got news for you.  Many choices for the programming of ACR seem to be made to fit an overarching philosophy of speed of processing and convenience, rather than IQ.  For instance, if WB is held until after demosaic, then moving the CA slider does not require redoing demosaic; definitely a convenience, since demosaic delays screen update by several seconds.  The current demosaic engine in ACR has all sorts of chromatic artifacting; your observation might explain why that is so.  ACR does a lot of smearing of chroma data to cover it up; that smearing starts to fail in the presence of noise, which one can see in any high ISO conversion of ACR 5.x and before.
« Last Edit: November 05, 2009, 11:25:58 pm by ejmartin »
Logged
emil
Pages: 1 [2] 3 4   Go Up