Pages: [1]   Go Down

Author Topic: RawDigger - a program to explore raw data  (Read 11242 times)

Iliah

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 770
RawDigger - a program to explore raw data
« on: February 21, 2012, 12:35:10 pm »

RawDigger is a computer program intended for viewing, studying, and analyzing raw files from digital photo cameras and some digital video cameras.

RawDigger allows the user to:

- View the real (that is, prior to any processing) pixel values in the raw file.
- Plot and save raw data histograms per channel for both the entire file as well as an arbitrary rectangular area of the file (selection). The histograms can be saved as graphics or as a table providing for convenience of further external processing.
- Calculate the statistics per channel (minimum, maximum, average, root-mean-square deviation) for the entire file.
- Calculate the statistics per channel and save the results in a processing-friendly form  - CSV, CGATS (for an arbitrary number of rectangular areas of the file).
- Save the compression (tone) curves of raw files in a tabulated form (for cameras and raw data formats where such curves are used).

System Requirements:
The program runs on a computer with Windows (XP, Vista, 7), 32 or 64-bit, 1GB memory, CPU supporting SSE2 or higher (Intel Pentium 4 or newer). Screen resolution should be 1024x768 or better.

OS X and possibly Linux will be supported at later stage.
Currently, user reports are that one can use Wine to run RawDigger under OS X and Linux (configurations reported as tested are OS X 10.6.8 and Wine-1.2.3; Wine (1.3.37-0ubuntu1~ppa1~lucid1) and Debian testing.

The beta version of RawDigger relies on Internet connection.

Please visit http://www.rawdigger.com to download the program. We currently offer two examples of use:
http://www.rawdigger.com/houtouse/lightmeter-calibration
http://www.rawdigger.com/howtouse/pixel-capacity-and-amplifier-gain

Suggestions and bug reports are most welcome.
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: RawDigger - a program to explore raw data
« Reply #1 on: February 24, 2012, 12:02:06 pm »

RawDigger is a computer program intended for viewing, studying, and analyzing raw files from digital photo cameras and some digital video cameras.

I am surprised that no one has commented on Iliah's post. Rawdigger is most useful in checking the calibration of one's meter/exposure system and determination of ETTR exposure as compared to spot meter readings and the appearance of the camera histogram and blinking highlights. One would think this would be important in a forum where ETTR was popularized.

To determine the light meter calibration, one can expose a gray card (or white card for that matter) and determine the corresponding raw value and determine the percent saturation of the sensor. The following data are for the Nikon D3.

Here is a reading from the central area of the gray card. The green channels are at 496 in a 12 bit file, and the saturation is 12.4% (assuming saturation at a raw value of 4000). Log (0.124, base 2) = -3. So one can spot meter the area in the scene where one wants to retain highlight detail and increase exposure by 3 stops, placing the metered area at clipping.



To compare the camera histogram and blinking highlights, I bracketed exposures of a Stouffer step wedge. Exposures at 1/20, 1/25, and 1/30 second at f/8 produced these results. The camera histogram and blinking highlights are shown.



One then looks at the raw data for these exposures, using RawDigger.













I will leave it to the reader to interpret the results.

Regards,

Bill
Logged

Iliah

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 770
Re: RawDigger - a program to explore raw data
« Reply #2 on: February 24, 2012, 01:02:42 pm »

Dear Bill,

Have you looked at the asymmetry of the per channel histograms (mostly both green channels), and at the progression of the sigma with the change of the exposure?
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: RawDigger - a program to explore raw data
« Reply #3 on: February 24, 2012, 04:25:31 pm »

Dear Bill,

Have you looked at the asymmetry of the per channel histograms (mostly both green channels), and at the progression of the sigma with the change of the exposure?

Iliah,

To some extent, but could you tell me what to look for? With the green channels, clipping seems to occur with saturation of the sensor rather than overflow in the ADC. As saturation approaches, the sigma begins to decrease as the histograms are clipped to the right. The onset of clipping is what Bill Claff terms as the white level. I would think that one should not exceed the white level if one wants the best color accuracy.

In the Stouffer series, Step 4 is below the white level and Step 3 is above it and shows 4 histograms and a smaller standard deviation if the statistics readout. The data are shown below. What insights can you offer?

Regards,

Bill









Logged

Iliah

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 770
Re: RawDigger - a program to explore raw data
« Reply #4 on: February 24, 2012, 04:47:04 pm »

Dear Bill,

If you take a completely (all 4 channels) blown-out shot, and process it without black frame, what is the statistics? Can you look at the histogram in linear mode too?

Do I see it right, and on the histogram for the selection on the step #4 the maximum is not yet reached in the green channels, yet the right side of the hump is truncated and the hump is asymmetrical?
« Last Edit: February 24, 2012, 04:49:24 pm by Iliah »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: RawDigger - a program to explore raw data
« Reply #5 on: February 25, 2012, 09:24:36 am »

Dear Bill,

If you take a completely (all 4 channels) blown-out shot, and process it without black frame, what is the statistics? Can you look at the histogram in linear mode too?

Do I see it right, and on the histogram for the selection on the step #4 the maximum is not yet reached in the green channels, yet the right side of the hump is truncated and the hump is asymmetrical?

Iliah,

As you requested.  There is some non-homogeneity in the green channels, and it is not possible, even with massive overexposure, to cause them to assume the same value.

Regards,

Bill





Logged

Iliah

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 770
Re: RawDigger - a program to explore raw data
« Reply #6 on: February 25, 2012, 09:41:26 am »

Dear Bill,

Red and Blue channels seem (IMHO only seem) to behave better due to white balance preconditioning, while on Green channels you can see peaks that may be caused by  multi-channel read-outs using http://www.analog.com/en/audiovideo-products/cameracamcorder-analog-front-ends/ad9974/products/product.html
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Re: RawDigger - a program to explore raw data
« Reply #7 on: February 25, 2012, 12:59:17 pm »

Dear Bill,

Red and Blue channels seem (IMHO only seem) to behave better due to white balance preconditioning, while on Green channels you can see peaks that may be caused by  multi-channel read-outs using http://www.analog.com/en/audiovideo-products/cameracamcorder-analog-front-ends/ad9974/products/product.html

Iliah,

Yes, that seems to be what is happening. Here is a part of the green1 channel split off by Iris with the threshold set to bring out the pattern. One can see tracks corresponding to multiple readout. In addition, the sensor has abrupt transitions suggestive that it is composed of portions welded together, with one such region just to the left of the threshold dialog box.

Regards,

Bill

Logged
Pages: [1]   Go Up