Pages: [1]   Go Down

Author Topic: non-interpolating raw converter?  (Read 20804 times)

StevenByks

  • Newbie
  • *
  • Offline Offline
  • Posts: 2
non-interpolating raw converter?
« on: December 09, 2009, 06:43:11 pm »

Is there a raw converter that doesn't interpolate when it de-mosaics?  This will of course mean that the output is a lower resolution, but on the other hand it should be a heck of a lot sharper and have less artifacting.  I honestly think you'd end up with an overall higher quality image by working this way, with the only downside being a slightly smaller maximum print size.  So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

P.S.  If it doesn't already exist, I'm going to start writing one asap.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
non-interpolating raw converter?
« Reply #1 on: December 09, 2009, 07:19:44 pm »

Quote from: StevenByks
have I stumbled upon the greatest breakthrough in digital photography ever?

Sure, but do you mind demonstrating the resulting RGB values based on following raw values?

R11   G11a   R12   G12a   R13   G13a   R14   G14a
G21b   B21   G22b   B22   G23b   B23   G24b   B24
R31   G31a   R32   G32a   R33   G33a   R34   G34a
G41b   B41   G42b   B42   G43b   B43   G44b   B44
Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
non-interpolating raw converter?
« Reply #2 on: December 09, 2009, 09:57:56 pm »

Quote from: StevenByks
So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

Yes, and no. Google search "Canon sRAW", and you'll find some newer Canon cameras already do this. The problem is that to eliminate any need for interpolating, you need to downsize your pixel dimensions by 50%, turning 20MP into 5MP (albeit 5 really good quality megapixels). The problem is that existing RAW converters do a good enough job interpolating that you get most of the benefit of your 20MP even though they are interpolated. Downsizing to 5MP throws away a LOT of detail.

Before you start wasting time coding this idea, try this experiment:
  • Pick an uncropped image, process it to the best of your ability, and then print it as large as you can.
  • Make a copy of the image file you just printed, and downsize it 50% (2000 pixels > 1000 pixels, etc) using Bicubic Sharper interpolation.
  • Print the downsized image the same size as the original image.
  • Compare the prints side by side, realize the pointlessness of the exercise, and then go find something else useful to occupy the rest of your life.
« Last Edit: December 09, 2009, 09:58:38 pm by Jonathan Wienke »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
non-interpolating raw converter?
« Reply #3 on: December 09, 2009, 10:15:58 pm »

Quote from: Jonathan Wienke
The problem is that to eliminate any need for interpolating, you need to downsize your pixel dimensions by 50%, turning 20MP into 5MP (albeit 5 really good quality megapixels
Do you have some reliable information regarding the method sRaw and mRaw are created? I would be surprized if that happened without interpolation.
Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
non-interpolating raw converter?
« Reply #4 on: December 09, 2009, 10:21:12 pm »

Quote from: Panopeeper
Do you have some reliable information regarding the method sRaw and mRaw are created? I would be surprized if that happened without interpolation.

I don't know the exact details, but the sRAW files are supposedly full RGB and 50% of the pixel dimensions...
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
non-interpolating raw converter?
« Reply #5 on: December 09, 2009, 10:45:00 pm »

Quote from: Jonathan Wienke
I don't know the exact details, but the sRAW files are supposedly full RGB and 50% of the pixel dimensions...
I know, but that does not necessarily exclude involving interpolation in the calculations of the RGB values.
Logged
Gabor

Graeme Nattress

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
    • http://www.nattress.com
non-interpolating raw converter?
« Reply #6 on: December 09, 2009, 11:46:29 pm »

Such a raw conversion does a number of things:

at a per pixel level it's sharper
overall, measurably less detail
more aliasing
noisier

I tend to find that if you, say, have a 20mp image you get a better looking 5mp version from a full demosaic and downsample than a straight non-interpolating decimation to 5mp.

Graeme
Logged

EduPerez

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 700
    • Edu Pérez
non-interpolating raw converter?
« Reply #7 on: December 10, 2009, 02:58:38 am »

Quote from: StevenByks
Is there a raw converter that doesn't interpolate when it de-mosaics?  This will of course mean that the output is a lower resolution, but on the other hand it should be a heck of a lot sharper and have less artifacting.  I honestly think you'd end up with an overall higher quality image by working this way, with the only downside being a slightly smaller maximum print size.  So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

P.S.  If it doesn't already exist, I'm going to start writing one asap.

And exactly how are you going to do the demosaicing without interpolation?
Logged

kal

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 63
    • http://fotonordest.blogspot.com
non-interpolating raw converter?
« Reply #8 on: December 10, 2009, 03:27:05 am »

Quote from: StevenByks
Is there a raw converter that doesn't interpolate when it de-mosaics?  This will of course mean that the output is a lower resolution, but on the other hand it should be a heck of a lot sharper and have less artifacting.  I honestly think you'd end up with an overall higher quality image by working this way, with the only downside being a slightly smaller maximum print size.  So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

P.S.  If it doesn't already exist, I'm going to start writing one asap.

dcraw does this, at user request, see http://www.cybercom.net/~dcoffin/dcraw/

However the half-size conversion is describes as being "fast", not "sharp"...
Logged
Piero
[url=http://fotonordest.blogspot.c

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
non-interpolating raw converter?
« Reply #9 on: December 10, 2009, 11:10:56 am »

As mentioned, dcraw (command line interface) will take an RGGB Bayer quartet of raw data, average the two green values, and output the resulting triplet as RGB pixel data.  RPP (Mac only, with a functional but rudimentary GUI) is a raw converter based on dcraw that carries over this feature of allowing a "half-size" interpolation. Because the three colors were not located at the same point on the sensor, such an approach necessarily generates aliasing of high spatial frequency image content.  As Graeme says, it's better to do a full demosaic of the raw data, and then downsample.  

Canon mRAW and sRAW show little in the way of aliasing artifacts, so it is very likely that they follow this second route (especially mRAW, which downsamples by a linear factor 11/16; hard to imagine how you're going to do that without interpolation!).  It would be silly not to, since the camera is already doing the full interpolation to generate the embedded jpeg.
Logged
emil

Graeme Nattress

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
    • http://www.nattress.com
non-interpolating raw converter?
« Reply #10 on: December 10, 2009, 11:16:19 am »

When I tested sRAW on my mk III it did indeed produce vastly more aliasing than the full rez decode, but it was just luma aliasing, rather than chroma aliasing.

Graeme
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
non-interpolating raw converter?
« Reply #11 on: December 10, 2009, 12:03:56 pm »

Quote from: ejmartin
As mentioned, dcraw (command line interface) will take an RGGB Bayer quartet of raw data, average the two green values, and output the resulting triplet as RGB pixel data... Because the three colors were not located at the same point on the sensor, such an approach necessarily generates aliasing of high spatial frequency image content
If dcraw really works this way, then the aliasing is a negligable issue, for the output is not truely RGB. Putting the "red", "green" and "blue" raw channel values together as R, G and B components is not demosaicing.
Logged
Gabor

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
non-interpolating raw converter?
« Reply #12 on: December 10, 2009, 01:58:11 pm »

You can do that just by extracting one of the RGGB RAW channels and ignore the other 3. The image will be half the size in both axis, sharp and should have less aliasing artifacts than when demosaicing. The problem is it will be a monochrome image

You can do that by extracting the RAW channels in DCRAW:

dcraw -v -d -r 1 1 1 1 -4 -T file.cr2

Rescale the resulting TIFF to 50% using nearest neighbour (no idea the name nearest neighbour is given in the English version of PS). Add two extra rows/columns of pixels before rescaling to pick up the preferred RAW channel.

One sample (no sharpening at all applied, 100% crop):




Aliasing still happens (see nearly horizontal hairs on top of the girl's head). I guess the gaps between the photocells in the sub-grid used to form the image make them behave closer to delta samplers than to the more desired light averaging samplers, therefore easily producing aliasing. Accutance is higher this way though.

Regards.
« Last Edit: December 10, 2009, 07:01:20 pm by GLuijk »
Logged

StevenByks

  • Newbie
  • *
  • Offline Offline
  • Posts: 2
non-interpolating raw converter?
« Reply #13 on: December 11, 2009, 09:17:43 am »

Ok, so when I first thought of this idea the simplest and easiest way to do it was the way that Canon seems to be doing it.  You cut the height and width each in half, leaving only 25% of the original pixel count but with the advantage of giving you full color pixels with 0 interpolation.  So to use Panopeeper's example numbers, you'd get something like:

R11 G11a R12 G12a R13 G13a R14 G14a
G21b B21 G22b B22 G23b B23 G24b B24
R31 G31a R32 G32a R33 G33a R34 G34a
G41b B41 G42b B42 G43b B43 G44b B44

turns into (R/G/B):

11/19A/B21 12/1AA/22 12/1BA/23 14/1CA/24
31/39A/41 32/3AA/42 33/3BA/43 343CA/44

(without bit-depth conversion of course)

Alright, now that's a pretty quick and dirty way of doing it, but is there a better way?  The method above samples four different pixels in order to get all the color information needed to generate one pixel of output.  So I thought, how else could I sample four pixels?



The grid shows the outline of the post-conversion pixels in spatial relation to pre-conversion.  The dots with letters represent samples taken per color channel.  On the left is supposed to be a normal interpolating algorithm.  The marked pixel is supposed to be showing that for that pixel it can sample the green directly, but has to create sample values for red and blue by looking at neighboring pixels and interpolating.  To the right is my second idea.  You only lose half of the pixels along the outer edge, so on large images you retain nearly all the original pixel count.  By shifting all the pixels, each one can now sample all the color information it needs.  I think this will reduce aliasing, but at the cost of introducing a different type of artifacting.  I suspect that this method will not increase IQ above standard methods, but maybe I'll write it out in Erlang just for fun.

I'm starting to see why no companies really take this approach.  Except for Fuji, which I've heard has a camera that doesn't interpolate.  But they get around that by laying out their color filter in a different arrangement than the standard Bayer.  I think that's a pretty cool solution.  Anyway, thanks everyone for your input.
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
non-interpolating raw converter?
« Reply #14 on: December 11, 2009, 11:43:11 am »

Quote from: Panopeeper
If dcraw really works this way, then the aliasing is a negligable issue, for the output is not truely RGB. Putting the "red", "green" and "blue" raw channel values together as R, G and B components is not demosaicing.

Try it with a contrasty, near-Nyquist texture and see what you get.  The aliasing is not negligible in situations where it tends to arise (eg as moire).   While not demosaicing, it is interpolation, albeit a crude form -- the RGB information in the output data is coming from locations that are slightly shifted from the input values, and then the data is subsampled.
Logged
emil

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
non-interpolating raw converter?
« Reply #15 on: December 11, 2009, 11:50:36 am »

Quote from: StevenByks
Alright, now that's a pretty quick and dirty way of doing it, but is there a better way?  The method above samples four different pixels in order to get all the color information needed to generate one pixel of output.  So I thought, how else could I sample four pixels?



The grid shows the outline of the post-conversion pixels in spatial relation to pre-conversion.  The dots with letters represent samples taken per color channel.  On the left is supposed to be a normal interpolating algorithm.  The marked pixel is supposed to be showing that for that pixel it can sample the green directly, but has to create sample values for red and blue by looking at neighboring pixels and interpolating.  To the right is my second idea.  You only lose half of the pixels along the outer edge, so on large images you retain nearly all the original pixel count.  By shifting all the pixels, each one can now sample all the color information it needs.  I think this will reduce aliasing, but at the cost of introducing a different type of artifacting.

Suppose there is a sharp diagonal edge passing from lower left to upper right, with the green pixels along the diagonal having midtone values, the pixels to the upper left being low values, and to the lower right having high values.  Your diagonal edge will be alternating RBRBRBR averaged in with the midtone green.

There is lots of image information that this sort of simple approach throws away, and that a more sophisticated, nonlinear demosaic algorithm takes account of.
Logged
emil

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
non-interpolating raw converter?
« Reply #16 on: December 11, 2009, 02:41:34 pm »

Quote from: ejmartin
Try it with a contrasty, near-Nyquist texture and see what you get.  The aliasing is not negligible in situations where it tends to arise (eg as moire)
I meant that the aliasing is negligable, because the result is pretty worthless anyway, for the raw channel pixel values are far from the R, G and B components of RGB.

Example for the color checker rendered by using the raw pixel values as RGB components and by demosaicing:

Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
non-interpolating raw converter?
« Reply #17 on: December 11, 2009, 06:35:59 pm »

Quote from: StevenByks
To the right is my second idea.  You only lose half of the pixels along the outer edge, so on large images you retain nearly all the original pixel count.  By shifting all the pixels, each one can now sample all the color information it needs.  I think this will reduce aliasing, but at the cost of introducing a different type of artifacting.  I suspect that this method will not increase IQ above standard methods, but maybe I'll write it out in Erlang just for fun.

I guarantee you it will suck. You have no provision for taking into account the variations in the location of the source pixels, so you're going to have weird color alternations along all hard edges, whether horizontal, diagonal, or vertical. You have to do averaging of all the pixels that fall within the area covered by an output pixel, and you need to fill in the gaps in the Bayer matrix with interpolated values or you'll get weird color artifacting and aliasing. I've been playing around with interpolation stuff a LOT the last few weeks (see my thread in Digital Image Processing for more details), and very little of it is anywhere near as simple as it appears at first glance, at least if you want to maximize output sharpness without aliasing. Achieving both of those goals simultaneously is a cast-iron-b**ch.
Logged

Graeme Nattress

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 584
    • http://www.nattress.com
non-interpolating raw converter?
« Reply #18 on: December 11, 2009, 08:17:05 pm »

Jonathan, you're spot on right there. Good demosaics are not easy, and the reasons why some are good and others are not are often subtle and nuanced. Good luck with yours.

Steven, from my memory of coding up one similar to yours on the right, it was quick, but soft. It certainly wasn't bad, but it wasn't sharp.

The major thing you get with more simple demosaics is that they do nothing for (or even exaggerate) chroma moire.

Graeme
Logged

ejmartin

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 575
non-interpolating raw converter?
« Reply #19 on: December 14, 2009, 08:57:12 am »

Quote from: Panopeeper
I meant that the aliasing is negligable, because the result is pretty worthless anyway, for the raw channel pixel values are far from the R, G and B components of RGB.

Example for the color checker rendered by using the raw pixel values as RGB components and by demosaicing:

In your example, it isn't the demosaicing that causes the difference, it's the color space transform.  In uniform flat areas there will be little difference between the sampling method suggested by the OP, and a full demosaic.
Logged
emil
Pages: [1]   Go Up