is there a dolby NR system for digital imaging?

[bwana]

In the good old days, you could hear tape hiss when a particularly quiet passage of music was being played. This happened even on the best reel to reel tape recorders because of the low signal to noise ratio of the quiet music. The electronics of the day introduced a certain amount of noise into the signal in decoding the tape and amplifying it for output to the line out jacks. Dolby got around this by processing the audio signal. Low amplitude signals were amplified by a variable amount to raise the amplitude to some arbitrarily high level. At the same time, the amplification factor was recorded on the tape as well. When the music was played back, the amplitude was diminished by the same factor. This 'dolby decoding' was the last step in playback. As a result, all the tape hiss that was added during processing was made much quieter and almost inaudible.

It seems the same thing could be done with a digital image sensor. Per pixel signal amplification of low signals (prior to analog to digital conversion of the signal) would allow storing a better signal in the raw file without clipping the bright tones. The amplification factor would be recorded in the raw and during raw conversion, the proper luminance values would be restored. What I am describing could be labeled an 'intelligent expose to the right' algorithm.

Is this being used in any sensor today?

[Steve House]

If I understand it correctly, Nikon's 'Active D-Lighting' does something like that.

[bwana]

I thought that was only for jpegs? I'll have to investigate.

[bwana]

Ok this what I found:

D-lightning is different from Active D-lightning, D-lightning was a feature of older cameras and it is only a tone curve that raises the shadows and lowers the highlights. Active D-lightning is a feature of newer cameras, it can alter exposure by 1/3 or 2/3 stops to avoid blown highlights and then applies a custom tone curve to optimize DR. In both DL and ADL The RAW data is numerically intact and the tone curve is applied during conversion, When ADL/DL is enabled NEF files are tagged and Nikon sw can read this tag and apply the correct tone curve. Adobe sw cannot read this tag and therefore ADL/DL is ignored. Since DL is just a curve it can be set off in the camera and later applied in Nikon Capture if desired, but ADL has to be set in camera and cannot be turned on later. This is not just due to the fact that ADL alters exposure but also because in some models like D700/D3 there is an "auto" mode which records a custom tone curve optimized using scene information which is gathered when camera records the photo.

So it seems there is no amplification prior to analog digital conversion, just exposure decrease. A custom curve is generated and that is used automatically by Nikon sw after demosaicising.

[Wayne Fox]

I’m not an engineer, but couldn’t help taking a stab at this.

With the audio, the signal is separate from the noise, the noise of the tape is the result of the recording medium, and is not part of the original signal.  Thus you can process the signal before you put it on the recording medium which is the source of the noise.

With imaging, the noise results from the same place that the signal does, it really doesn’t ever exist in a state which is separate from the signal.  So tough to process it, otherwise you could just eliminate it because the recording medium (digital bits) doesn’t have noise property.

[kirkt]

I’m not an engineer, but couldn’t help taking a stab at this.

With the audio, the signal is separate from the noise, the noise of the tape is the result of the recording medium, and is not part of the original signal.  Thus you can process the signal before you put it on the recording medium which is the source of the noise.

With imaging, the noise results from the same place that the signal does, it really doesn’t ever exist in a state which is separate from the signal.  So tough to process it, otherwise you could just eliminate it because the recording medium (digital bits) doesn’t have noise property.

You can characterize the noise of the recording medium (the sensor) with a dark frame - as is done in astrophotography.  I would guess that there is nothing stopping one from creating sensor noise profiles, albeit very specific to that condition associated with the time of image capture (because of camera settings, ambient temperature, sensor temperature, exposure time, etc.).  This is not an "intelligent" system as suggested by the OP, just a compensatory mechanism.

kirk

[Jim Kasson]

You can characterize the noise of the recording medium (the sensor) with a dark frame - as is done in astrophotography.  I would guess that there is nothing stopping one from creating sensor noise profiles, albeit very specific to that condition associated with the time of image capture (because of camera settings, ambient temperature, sensor temperature, exposure time, etc.).  This is not an "intelligent" system as suggested by the OP, just a compensatory mechanism.

That would work for dark current. A modification of that would work for pixel response non-uniformity, which is important in the highlights. However, at most normal photographic tone values and SNRs, the dominant noise is photon, or shot noise. That is a property of the light falling on the sensor and of the sensor area, and can't be subtracted out.

TANSTAAFL.

Jim

[Bart_van_der_Wolf]

That would work for dark current. A modification of that would work for pixel response non-uniformity, which is important in the highlights. However, at most normal photographic tone values and SNRs, the dominant noise is photon, or shot noise. That is a property of the light falling on the sensor and of the sensor area, and can't be subtracted out.

Hi Jim,

Although correct, photon shot noise can't be simply subtracted, it can be reduced by averaging multiple exposures (requires relatively stationary subjects, e.g. product shots). In fact, read noise can also be averaged with CMOS devices by reading out the signal multiple times, which is virtually non-destructive unlike with CCDs.

Cheers,
Bart

[jrsforums]

I, also, am not an audio engineer.

However, I do not look at the sensor as a recording medium, but a capture medium, such as a microphone.  There is inherent noise in the mic capture and the analog path.

As I remember, Dolby originally worked on reducing the inherent noise in the recording medium, tapes and cassettes and optical film recording.  Later moving on to support compression to increase perceptible sound on multiple channels with reduced bandwidth.

I don't believe reducing the noise in a mic or sensor and their signal paths is in any way similar to what Dolby Labs did.....but...what do I know :-)

John

[Jim Kasson]

Although correct, photon shot noise can't be simply subtracted, it can be reduced by averaging multiple exposures (requires relatively stationary subjects, e.g. product shots). In fact, read noise can also be averaged with CMOS devices by reading out the signal multiple times, which is virtually non-destructive unlike with CCDs.

Bart, you are so right. http://blog.kasson.com/?p=2854

Another acronym: GMTA.

Good point about read noise.

Jim

[bwana]

...the dominant noise is photon, or shot noise. ...

Jim

I did not know this. I thought read noise was a biggee. And read noise can be minimized by raising the iso since that is analog amplification before ADConversion. Whereas increasing exposure in post makes things way worse. Here is the explanation that I go by (with pix too!)
http://photo.stackexchange.com/questions/35136/is-it-better-to-shoot-with-a-higher-iso-or-use-lower-iso-and-raise-the-exposure

So yes, more photons are needed, not just better photon massage (= amplifying the the almost empty photon wells before ADC and then restoring them on raw conversion)

[pluton]

I, also, am not an audio engineer.

However, I do not look at the sensor as a recording medium, but a capture medium, such as a microphone.  There is inherent noise in the mic capture and the analog path.

As I remember, Dolby originally worked on reducing the inherent noise in the recording medium, tapes and cassettes and optical film recording.  Later moving on to support compression to increase perceptible sound on multiple channels with reduced bandwidth.

I don't believe reducing the noise in a mic or sensor and their signal paths is in any way similar to what Dolby Labs did.....but...what do I know :-)

John

Good point: The transducers in audio are the microphone and the loudspeaker.
The analogue in digital imaging would be the lens/sensor and printer or display.

[Jim Kasson]

I did not know this. I thought read noise was a biggee. And read noise can be minimized by raising the iso since that is analog amplification before ADConversion. Whereas increasing exposure in post makes things way worse. Here is the explanation that I go by (with pix too!)
http://photo.stackexchange.com/questions/35136/is-it-better-to-shoot-with-a-higher-iso-or-use-lower-iso-and-raise-the-exposure

In most of the modern sensor chips with on-chip ADCs (all the ones I've tested), most of the read noise is before the ADC. Also, for photographic quality, in most circumstances, you want SNRs of 10 or better, which means raw counts of 100 or more to keep the photon noise below that. Read noise at base ISO is usually a few counts, and uncorrelated noise adds as the square root of the sum of the squares, which means the photon noise dominates.

Raising the ISO can help with Canons, I hear. I haven't tested them. With the modern Sony chips in Nikons and Sonys, raising the ISO past a certain point doesn't help any more, and you're better off doing it in post.

Here are some examples with the Sony a7. If you poke around my blog, you can see results for the a7R, the Nikon D800E, and the Nikon D4. You can also see results for the Leica M240, and the Leica M9. Surprisingly, in the M9, turning up the ISO past 640 hurts IQ significantly.

http://blog.kasson.com/?p=4928

http://blog.kasson.com/?p=5063

Jim

[Simon J.A. Simpson]

I WAS a sound engineer (BBC TV).

To put it very simply professional/domestic Dolby audio analogue signal processing relied on pre–processing the signal relative to the noise (and other analogue recording artefacts) and then post processing both in an a equal and opposite direction to effectively push the noise/artefacts lower in strength relative to the signal.

Dolby audio digital is a method (or, rather, methods) of digitally encoding multiple channels of cinema sound for imaging on film* or, latterly, so it can be streamed from a prerecorded source or over the internet.

I'm NOT, however, a digital imaging engineer but I would observe that the artefacts we refer to as picture noise arise out of the randomness of quantising (thus pixels of a smooth blue sky are rendered in slightly differing shades of blue) as well as analogue noise generated by the photosensitive chip, especially when the signal is amplified under low light conditions.  I would expect most camera manufacturers to incorporate some kind of noise reduction into the electronics of their chips which would not be the same as Nikon's Active D Lighting which I believe is software, not electronics, driven.

IMHO I would consider that trying to reduce the noise that we perceive in our images using a method à la Dolby to be extremely difficult, given the random nature of that ‘noise’, as it would be like trying to apply a pre/post correction which would effect the whole image – all colours and potentially levels of brightness.


* Just for interest, and for those that don't know, Dolby 5.1 digital sound is encoded and imaged on 35mm film as small squares, in between the sprocket holes of the film, complete with a small Dolby logo in the middle (apparently used for registration purposes of the reader).  See here: http://en.wikipedia.org/wiki/Dolby_Digital

[bwana]

Thank you Mr. Kasson. The best part of your blog was the idea of rewriting some of the raw conversion software to combine 4 pixels into one to avoid demosaicisng artifacts (color noise). Another one of your great ideas is the different color filter array that only has 1 green instead of 2 - that would give 3 'subpixels' to give one full color pixel. A nikon d800E with this kind of Kasson array would effectively have 12 megapixels. The pentax 50 megapixel chip would be more than 16 megapixels. These would be awesome lowlight machines. Such a Nikon D800K (for Kasson) would shame a D800S. In addition, the d800K would be great at 4 k hd video - the 12 megapixels could fill a native 4k frame.

Perhaps Nikon is already thinking of doing this and that's why they have abandoned their regular raw converter?