B&W Sensor approaches: non-Interpolation and HDR

[Guillermo Luijk]

These last days I did an experiment, I wondered about the interest in the market for a specific B&W sensor, and I thought of 2 approaches to achieve.

The reasons for a specific B&W sensor are that in my opinion a Bayer sensor is not optimised for B&W photography. Keeping the same degree of complexity (number of photocaptors mainly) I think we can design a B&W sensor that improves the quality of results for this specific kind of photography because a Bayer sensor has design disadvatanges over a monochrome sensor for B&W such as:

- Irregular band pass filtering in 3 agressive channels (RGB)
- Lower optical efficiency because of the reduction of incident light power due to these filters filters
- High degree of interpolation (only 1 out of each 3 channel levels is captured)
- Multiple B&W conversion methods from RGB suggest none is 100% satisfactory
- The conversion process itself is an additional pain step in the B&W photographer workflow

I have thought of 2 approaches for a B&W sensor with improved quality over a Bayer version with the same number of photocells:

BASIC NON-INTERPOLATION B&W SENSOR
All captors receive the entire visible spectrum. No interpolation is needed. Would be some kind of "B&W Foveon" sensor.

HDR B&W SENSOR

By allowing some degree of interpolation, we set ND -4EV microfilters over half the photocells. 2 independent B&W images will be produced that can be blent achieving a DR expansion of 4 f-stops.

______________________

NOTE: I consider the HDR sensor interpolation rate similar and even lower than that found in a Bayer sensor:


According to Y = 0,299R + 0,587G + 0,114B:
- Bayer RGGB sensor (left): (29,9% + 58,7% * 2 + 11,4%) / 4 = 39,7% captured information
- Proposed HDR B&W sensor (right): (100% + 0%) / 2 = 50% captured information
______________________

I will do a simulation to compare both methods (non-interpolated B&W and HDR B&W). I will start from 3 Bayer images shot over a high dynamic scene at: -2EV, 0EV, +2EV and converted to B&W through Y = 0,299R + 0,587G + 0,114B rule.
The 0EV shot being the 'correct' tradeoff exposure for a single shot on that scene.

BASIC NON-INTERPOLATION B&W SENSOR
For the non-interpolation approach, the 0EV image converted to B&W will be straight the image to compare.

HDR B&W SENSOR
For the HDR approach, I will decimate 50% of the values (discard them) in the -2EV and +2EV B&W images, according to the 2 sensor distributions.
Later the discarded values will be interpolated through the simplest interpolation algorithm, median of the 4 surrounding values:





And then I will mix both images with the following blending criteria: for every pixel I keep the value from the +2EV image whenever possible. Should this value be close to saturation I would take the -2EV value. Of course exposure is corrected to match on both images prior to fusion.

Done this, I wil compare the result of both approaches in shadows, medium tones and highlights.


This is the scene under study:


This is the comparision in different areas after the 2 B&W approaches have been done: crops at 100% (shadows), 50% (medium tones), 50% (highlights)
(when applied nearest neighbour rescaling to preserve SNR):
(left: non-interpolation B&W sensor, right: HDR B%W sensor)





It seems the HDR approach performs much better than the basic in terms of DR and visible noise.
Let's find out if our simplistic interpolation algorithm (median of 4 values) for the HDR sensor means less detail accuracy:

Crops at 100%:
(up left: Bayer, up right: decimated B&W Bayer)
(down left: non-interpolation approach, down right: HDR approach)



There is some loss of sharpness in the fine detail (see the "60W" label) in the HDR approach when compared to the non-interpolated, even if the former is about to blow. Surely with a better interpolation strategy the differences would not be so visible.

This is the final aspect of the B&W HDR scene:




QUESTIONS
1. Do you think a B&W sensor has a place in the market if it really improves the quality of B&W shots of the equivalent complexity Bayer?
2. Do you think it is worth to allow some degree of interpolation and sharpness loss to enjoy the 4 f-stops DR expansion of the second approach?
3. Do you think these kind of sensors would need some orthocromatic filtering (maybe on the sensor, maybe on the lens) to produce adequate B&W images?
4. Why Leica does not design such a sensor type based camera?


In the original article (Spanish) you can see a 600% crop which reveals more clearly the sharpness loss in the high contrast fine detail areas of the "60W" label: B&W SENSORS original article (Fig. 13). But think that I used a very basic emergency interpolation scheme (median).

[John Sheehy]

QUESTIONS
1. Do you think a B&W sensor has a place in the market if it really improves the quality of B&W shots of the equivalent complexity Bayer?

I would buy one, if the price weren't too ridiculous (as specialty items often are).  I think the chances of selling a B&W digital now are probably better than they were a few years back when some were, in fact, marketed, but failed.  There are more people now doing digital, more long for the higher sensitivity, resolution, and aesthetics of B&W.

2. Do you think it is worth to allow some degree of interpolation and sharpness loss to enjoy the 4 f-stops DR expansion of the second approach?

I'm not sure I would want that as standard, unless the camera had a large number of pixels.  Besides, you can emulate that somewhat with colored light sources and color filters with Bayer cameras.  For example, with a green filter, you can clip the green channel and use its shadows while the red and blue pick up highlight detail.  Shooting under a sodium vapor lamp, which is all concentrated in a narrow frequency band, results in what you are experimenting with.  Shooting with pure-color LED light sources can give similar results.

3. Do you think these kind of sensors would need some orthocromatic filtering (maybe on the sensor, maybe on the lens) to produce adequate B&W images?[a href=\"index.php?act=findpost&pid=164649\"][{POST_SNAPBACK}][/a]

I would rather the sensor be panchromatic, and the user add filters to taste.

[Jonathan Wienke]

I'm not real excited about using a monochrome sensor. For me, much of the creative process of digital B&W is being able to experiment with various virtual color filters and spectral sensitivity curves AFTER the shot in PS rather than having to guess what the best color filtration settings might be, and committing to them  before exposure. The biggest complaints about digital B&W have more to do with Bayer pixel softness than anything else.

There is also no reason to mess with a ND checkerboard filter to increase DR; Fuji's approach of using a mix of large and small photosites is far more elegant and allows one to image photons your approach would discard.

If I was building a camera sensor from the ground up, here's how I would do it:

[John Sheehy]

Each output pixel (outlined in heavy black lines) is composed of 3 sub-pixels, (outlined in narrow black lines) which are masked with a CMY color filter array instead of the traditional GRGB Bayer pattern.

The noise increase resulting from splitting each pixel in 3 parts will be offset by combining the 3 sub-pixels back into 1 output pixel, as well as by virtue of using a CMY filter array, which allows twice as much light through as an RGB filter and gives you a noise-free 1-stop ISO boost.
[a href=\"index.php?act=findpost&pid=164780\"][{POST_SNAPBACK}][/a]

C, M, and Y bandpass filters will not automatically allow more light to pass through.  You're probably thinking that "cyan equals green plus blue; magenta equals red plus blue; and yellow equals red plus green", which is not really true.  A real bandpass filter needs to have fairly steep trade-off curves between the colors, and fairly narrow peaks, otherwise you get chromatic noise and quantization when trying to separate the colors.  The red, green, and blue bands in RGB cameras could be made wider, too, but they don't do it because it is detrimental.

[mcbroomf]

It's pretty old technology now but there's an interesting read here on what may have been the only SLR made with a dedicated b&w sensor.  Worth a read just for it's historical context.

http://www.luminous-landscape.com/reviews/...odak-760m.shtml

When shooting MF and LF for some time I only took b&w (although I started shooting colour as well before my switch to digital).  For some time after I switched to digital I thought a body like this would be great, but I now realise that given the inevitable high cost due to the low volumes I doubt I'd be prepared to pay another many $k's for a body that while it may give me higher effective resolution and DR, would mean I'd have to lug a 2nd body with me if I thought I'd come upon a scene worthy of b&w.

Mike

[Guillermo Luijk]

It's pretty old technology now but there's an interesting read here on what may have been the only SLR made with a dedicated b&w sensor.  Worth a read just for it's historical context.

http://www.luminous-landscape.com/reviews/...odak-760m.shtml

When shooting MF and LF for some time I only took b&w (although I started shooting colour as well before my switch to digital).  For some time after I switched to digital I thought a body like this would be great, but I now realise that given the inevitable high cost due to the low volumes I doubt I'd be prepared to pay another many $k's for a body that while it may give me higher effective resolution and DR, would mean I'd have to lug a 2nd body with me if I thought I'd come upon a scene worthy of b&w.

Mike
[a href=\"index.php?act=findpost&pid=164850\"][{POST_SNAPBACK}][/a]

But don't you think Mike that with the popularisation of digital cameras, the number of people interested in purely B&W photography can increase significantly in the next years? that would push a specific sensor like this to reasonable production that would make cost affordable.

Jonathan, your approach looks impressive, but I need to take some more time to swallow it.

About the neutral filters, I proposed that possibility instead of reduced photocells as in the Fuji for 3 reasons:

1. I think a sensor with a uniform grid of photocells and the checkerboard neutral filters array should be cheaper to produce (no idea if I am right or not).

2. A full size photocell would average information falling into a wider area than a reduced captor, so it would be more accurate to the scene.

3. Lastly I think it's easier to design a ND microfilter so that the -4EV cells behave in all ways the same as the 0EV ones but with a shift in exposure, than to achieve this through physically differentiated captors. After analysing several Fuji S3 RAW files I have found some differences in the behaviour and quality of its S and R photocells (of course leaving aside the exposure offset, which BTW seems to be exactly 3.6EV in all Fuji HDR shots).

Regards.

[Jonathan Wienke]

C, M, and Y bandpass filters will not automatically allow more light to pass through.  You're probably thinking that "cyan equals green plus blue; magenta equals red plus blue; and yellow equals red plus green", which is not really true.  A real bandpass filter needs to have fairly steep trade-off curves between the colors, and fairly narrow peaks, otherwise you get chromatic noise and quantization when trying to separate the colors.

But the passband for a CMY array is twice as wide as the equivalent RGB for the same cutoff steepness. Having a wider passband per-filter is going to mean more photons passing through the filter.

[mcbroomf]

Guillermo,

Well I wish it weren't so but I don't think that the numbers will swell significantly.  I think we have to recognize that we are a pretty small portion of the overall photographic community, even those that own digital SLRs.

For example I have a decent number of friends.  Almost all of the own cameras and take photos.  Only 2 of them own digital SLRs, only one of them has a 13" printer.  None of them are remotely interested in taking b&w prints although they do admire some of mine.  I recently spent Christmas in the UK with my family, and my sister who has a pretty good eye didn't even bother to bring her Canon XTi with her, just her small P&S Casio.  

I realize of course this is just a sample of 1, but I think Canon/Nikon etc have pretty savvy marketing departments and if a pure b&w camera could be sold then we would already have seen something in the p&s sector as test.  In fact this would help with ISO noise I imagine as there would be less loss from the CFA layer.

Mike

[John Sheehy]

But the passband for a CMY array is twice as wide as the equivalent RGB for the same cutoff steepness. Having a wider passband per-filter is going to mean more photons passing through the filter.
[a href=\"index.php?act=findpost&pid=164913\"][{POST_SNAPBACK}][/a]

It would also mean more chromatic noise, and color issues.

And there's no reason you couldn't have wider R,G, and B filters, with the same problems.

Cyan does not mean "a full blue band overlapped with a full green band".  It means the center of the band is between blue and green, and nothing more.

[Paul Kay]

Correct me if I am wrong, but my guess is that B&W is now the medium used by a dwindling number of real enthusiasts. Given this as a starting point, I'd say that any B&W only camera would have limited appeal unless of course it could be marketed appropriately and had an appeal which would attract more photographers into an 'enthusiast' style environment. So I'd hazard a guess and suggest that unless it was possible to get a company like Leica to take the idea up (where a 'special' camera might be seen as a potential marketing assest rather than risk) then I'm not sure that it is a feasible idea. There may well be a few manufacturer's arounf though who, if visionary enough, may, like Leica, be potentially interested in what is most decidedly a non-mainstream product. But forgetting about the theory, the prime requirement of any such B&W camera would be to find an interested manufacturer before considering the nuance of the theory, etc.. Sadly, that's probably the hardest thing to do.

[Jonathan Wienke]

It would also mean more chromatic noise, and color issues.

And there's no reason you couldn't have wider R,G, and B filters, with the same problems.

If I understand you correctly, in order to distinguish one color from another, the response for each color channel must have a bell curve shape instead of a flat top, or it won't be possible to distinguish one color from another in any area where two flat areas overlap. If the filters are adjusted to allow more light through, then one loses the ability to distinguish one color from another. Correct?

[photo570]

Mega-Vision produce 16,22 and 39 mp monochrome digi backs available in a variety of mounts. Yummy!

No I don't work for them, but will probably get one some time this year. Some one also got Phase to do a special run of P25(?)'s a while ago.

Cheers
Jason Berge.

[John Sheehy]

If I understand you correctly, in order to distinguish one color from another, the response for each color channel must have a bell curve shape instead of a flat top, or it won't be possible to distinguish one color from another in any area where two flat areas overlap. If the filters are adjusted to allow more light through, then one loses the ability to distinguish one color from another. Correct?
[a href=\"index.php?act=findpost&pid=165065\"][{POST_SNAPBACK}][/a]

It's more a matter of how sharply the sides of the curve cross their neighboring colors.  If you look at the color response graphs for the Foveon, for instance, you'll see that where the neighboring colors cross is about 70% of the peaks, whereas, with a Bayer RGB, the crossover points are down in the 25-35% range or so.  The only way to allow more light to pass through is to have shallow curves that cross very close to the peak values.

[TimothyFarrar]

Just a guess here, but wouldn't most of the market for B&W shooters still be in film (traditional darkroom methods) not digital?

Basically the market for a camera would be limited to B&W shooters who need HDR at the expense of some sharpness, and as was previously mentioned it is easy to get dynamic range in B&W at the expense of using a colored filter and a longer shutter time on current cameras. Don't get me wrong I think it is a really interesting idea, just seems like it would really be tough to market!

[Jonathan Wienke]

It's more a matter of how sharply the sides of the curve cross their neighboring colors.

OK. So going back to my sensor concept, if I change the CFA to a more standard RGB filter set with the channel crossover at ~40%, something like this:



and get rid of the CMY-to-RGB conversion step, what do you think?