In Camera HDR - theory

[bartron]

Been thinking about this one for a while and I think given the technology it is doable. The result would be reduction in blown highlights, in camera HDR and opens up the posibility of an interesting photo technique.

The problem with digital sensors is their low exposure latitude. In a contrasty situation you can chose to expose for highs, mids or low in a single exposure, or you can take multiple exposures and combine them and there are many programs that will do this for you.

It occured to me though that a digital photo is nothing but a bunch of numbers representing various things about the file. If you could add another number to this file you could eliminate problems with blown highlights and effectivle attain in-camera HDR. The number represents the time taken for a pixel to reach 100% saturation. In theory you could overexpose an image (e.g. full sunlight, f1.8, 30 second exposure) and be able to recover detail.

Obviously it wouldn't be as simple as all that...there would be formulas to take into account the way the sensor reacts to light and thus be able to recover information about the image.

Obviously this number needs to be stored somewhere, somehow. A 32 bit number can record single nano seconds for about 2 and a bit seconds, a 16 bit number can store 1 every 15,200 or so nano seconds up to 1 second. The former would tripple the amount of information to be stored and the later would double it (after the camera unscales the 12bit ccd data to 16 bit). How fine the granuality needs to be depends on how much detail on average is lost due to over exposure.

Given that storage is cheap these days, what's another 10mb of data to store?

Thoughts?...been done before?...or am I just talking out my rear end..

Cheers,

Bart

edit: didn't realise this was my first ever post (thought I'd posted here before)...hmm...will have to introduce myself somewhere.

[bbrantley]

Hi Bart,

Sounds like a good idea, but I think it falls into the "no free lunch" category.  The biggest challenge, I suspect, is having a timer to measure each sensor.  High-resolution timers are quite complex -- they need a lot of transistors and other supporting hardware to enable them.  One of the biggest hurdles with designing large photoarrays is keeping the ratio of photodetector area to control circuitry as high as possible.  Modern CMOS sensors get away with just a handful of transistors per photodetecting site.  Even if it were technically feasible and affordable to build millions of high-resolution timer and employ one at each photosite, I believe the tradeoff in light-gathering efficiency would obviate most of the advantage.

There's another catch, too.  Photodetectors are really just time-integrators.  They add up "all" the photons that come in per some unit time and report that number.  Once you hit the ceiling on their ability to count, you then throw away any *additional* information about the light flux after that.  Imagine a silly example where your photosite can count as many as 10 photons before it pegs out.  Now you open your shutter for 3 seconds and light starts flowing in.  Let's say it flows in at 20 photons per second initially...  so your high-resolution timer writes down that the sensor pegged after 0.5 seconds and you extrapolate that to 60 photons for the entire 3-second exposure.  Well, that's probably only a really nice guess if the light flux is constant!  In a non-linear scenario (say, the inbound photon flow increases immensely during seconds two and three) you've lost the ability to capture what's really going on.  I think your timing-interpolation approach would work well only in highly static situations.  

It is a fun thought experiment, though.

Ben