What info can be lost once you have completed raw conversion?

[Fine_Art]

If you really want to be particular about data loss you need a program that uses scientific image formats. This will avoid losing rounding errors in image manipulations making operations truly reversible.

I use an astronomy program Images Plus which can do RAw conversion to 32 bit floating point fit format. This is widely used in scientific imaging.

http://fits.gsfc.nasa.gov/fits_viewer.html

[Tim Lookingbill]

If you really want to be particular about data loss you need a program that uses scientific image formats. This will avoid losing rounding errors in image manipulations making operations truly reversible.

I use an astronomy program Images Plus which can do RAw conversion to 32 bit floating point fit format. This is widely used in scientific imaging.

http://fits.gsfc.nasa.gov/fits_viewer.html

Yes, but how good is it at producing a pleasing image for photographers.

An electron microscope (according to what I saw on the Discovery Science channel) has now gotten so powerful they can show us what an atom looks like as perfectly lined up fuzzy balls in a grid pattern, but I wouldn't want to frame it and put it on my wall.

[Fine_Art]

Its really limited by the ability of the person. All image editing functions can be done in 32 bit floating point space. You will never get combing.

No, it does not have a "secret sauce" function that makes your image wonderful. You have to do it, as in know what you are doing with luminance and color curves, etc.

Maybe people should look at http://clarkvision.com/, his site is what got me into using the program. He uses it.

[hjulenissen]

If you really want to be particular about data loss you need a program that uses scientific image formats. This will avoid losing rounding errors in image manipulations making operations truly reversible.

I use an astronomy program Images Plus which can do RAw conversion to 32 bit floating point fit format. This is widely used in scientific imaging.

http://fits.gsfc.nasa.gov/fits_viewer.html

Floating-point implementation does not avoid round-off errors, consequently arithmetic implemented in float32 is generally not "truly reversible".

-h

[Fine_Art]

Well, Im no expert on the implementation, but it seems to me any rounding error on a 32 bit floating point file is going to be so small you will never see it on a histogram, a display, or out of a printer made in the next 10 years.



http://en.wikipedia.org/wiki/Single_precision

[Fine_Art]

[Gollum clips his data]

"It's lost! It's lost! my precious is lost!
We could use recovery.
We don'ts have recovery!
We could stretch it, what's left.
No. No. That would make very nasty banding. And combing. We don't wants that do we Precious.
In 32bit FP we won'ts see it.
Won'ts see it?
Look, the histogram is zoomed to 14 bit view on a 12 bit RAW after very strong curves. There should be huge holes!
That's impossible! Gollum. Gollum. The edge of the sun is so smooth! How does it do its?!
Its the working space my Love, the working space. We do the maths in a large working space!

[hjulenissen]

I dont know where you get your kicks from, I merely pointed out that your statement was clearly wrong:

If you really want to be particular about data loss you need a program that uses scientific image formats. This will avoid losing rounding errors in image manipulations making operations truly reversible.

If you want to argue that _you_ are unable to _see_ visible artifacts running _your algorithms of choice_ on _your images_ employing float32, then that is fine.

32-bit float can be insufficient for certain algorithms and applications. 8-bit integer can be more than enough for other algorithms and applications. As always, single marketing numbers does not tell the whole story.

-h

[jeremypayne]

That would make very nasty banding.

Can you show us some practical examples where using a 32bit FP implementation you were able to achieve a visibly superior results?

[Fine_Art]

Can you show us some practical examples where using a 32bit FP implementation you were able to achieve a visibly superior results?

I could do a one shot raw conversion in 16 bit tif vs 32 bit fp, then display the difference. I will do that later. Look at the edge of the sun in the screenshot. I duplicated a portion of the shot to test a curve that would be called a strong hyperbolic stretch of the data. Do you see banding?

Another example that comes to mind immediately is a several shot bracket merged to one HDR. I can take several shots then combine them to one. They all fit in a 32 bit fit. I can save the file to fit then save again to jpg or tif with tone mapping.

[Fine_Art]

Ok, Here is my first HDR attempt in this program. I had to spend a fair amount of 1/2 hr figuring out how to do it.

This was a 3 shot bracket on a bobbing ferry hand held to the window. The image is nothing special, it was just an opportune shot to pass time. You can see a black edge around 2 sides where the shots had to be aligned. I could have done translate and rotate but it was fast enough I don't think there was much rotate.

The whites were badly clipped in one shot. The whole lower tree area was black or near black in the other. You get the idea. It was done in part of the time since the prior post.

Now, we have all seen many gross HDRs put on the web for display as final images. This looks pretty realistic considering it was a fast cobble together with a lot of time spent trying to align the shots.

[bjanes]

Its really limited by the ability of the person. All image editing functions can be done in 32 bit floating point space. You will never get combing.

No, it does not have a "secret sauce" function that makes your image wonderful. You have to do it, as in know what you are doing with luminance and color curves, etc.

Maybe people should look at http://clarkvision.com/, his site is what got me into using the program. He uses it.

ImagesPlus is an excellent but specialized program. It is more suited for processing of astronomical images than for general photography and is often used for sensor analysis using raw files. However, it is less suitable for general photography. Among other considerations, it does not appear to be color managed. It converts my D800e images into an untagged color space. ACR/LR (among others) is better suited for general photography. What about printing with profiles and softproofing?

Iris is another astronomical program favored by techies for analysis of raw files. It can split the CFA image into its channels (RGGB), perform mathematical operations, and perform demosaicing, WB and gamma corrections. It also does Richardson-Lucy deconvoluiton and VanClittret filtering. It also is not color managed. For general photography, I will stick to ACR/LR.

Regards,

Bill

[Tim Lookingbill]

Ok, Here is my first HDR attempt in this program. I had to spend a fair amount of 1/2 hr figuring out how to do it.

This was a 3 shot bracket on a bobbing ferry hand held to the window. The image is nothing special, it was just an opportune shot to pass time. You can see a black edge around 2 sides where the shots had to be aligned. I could have done translate and rotate but it was fast enough I don't think there was much rotate.

The whites were badly clipped in one shot. The whole lower tree area was black or near black in the other. You get the idea. It was done in part of the time since the prior post.

Now, we have all seen many gross HDRs put on the web for display as final images. This looks pretty realistic considering it was a fast cobble together with a lot of time spent trying to align the shots.

Now show how the 32 bit floating point benefited the image which was the reason for asking you to post examples.

[Fine_Art]

ImagesPlus is an excellent but specialized program. It is more suited for processing of astronomical images than for general photography and is often used for sensor analysis using raw files. However, it is less suitable for general photography. Among other considerations, it does not appear to be color managed. It converts my D800e images into an untagged color space. ACR/LR (among others) is better suited for general photography. What about printing with profiles and softproofing?

Iris is another astronomical program favored by techies for analysis of raw files. It can split the CFA image into its channels (RGGB), perform mathematical operations, and perform demosaicing, WB and gamma corrections. It also does Richardson-Lucy deconvoluiton and VanClittret filtering. It also is not color managed. For general photography, I will stick to ACR/LR.

Regards,

Bill

It converts the file as straight linear data. Gamma is under the color menu, you have to do it yourself.

You are absolutely right it is not good for going to print. It will not understand a color calibrated screen. It will not provide a soft-proof for printing. You will have to send the document to another program for that. I have always used PS for that.

[Fine_Art]

Now show how the 32 bit floating point benefited the image which was the reason for asking you to post examples.

The benefit of having your data manipulated in a 32bit fp should be clear. I believe the latest version of PS uses the same for it's HDR for the same reason.

One thing about that file is the light tones in the snow were not gamma adjusted. Quite often tones near clipping look so similar they cannot be distinguished. By giving the user control of gamma after a straight linear conversion the program allowed me to show the whites as the camera sees them. Tone mapped,if you will over a large luminance range. I was able to combine that with the same at the low end from another shot which I used a very strong curves on instead of gamma. The curve was hyperbolic, almost a right angle. 2 shots with 12 bit data were put together over 32 bit fp and output to a full 16 bit tif. Maybe I could have output to a 32 bit fit, I'm not sure, I was struggling to learn the features.

[bjanes]

It converts the file as straight linear data. Gamma is under the color menu, you have to do it yourself.

You are absolutely right it is not good for going to print. It will not understand a color calibrated screen. It will not provide a soft-proof for printing. You will have to send the document to another program for that. I have always used PS for that.

Gamma is easy enough to do, but you seem to miss the point that ImagesPlus in not color managed. Do you render into ProPhotoRGB? And what camera profile do you use.

Bill

[Fine_Art]

Gamma is easy enough to do, but you seem to miss the point that ImagesPlus in not color managed. Do you render into ProPhotoRGB? And what camera profile do you use.

Bill

When I go to print I render to AdobeRGB. It is bigger than what my printer (Epson R1800) can handle. The camera profile is the same. That is in PS, I would not try to print from IPlus.

[hjulenissen]

ImagesPlus is an excellent but specialized program. It is more suited for processing of astronomical images than for general photography and is often used for sensor analysis using raw files. However, it is less suitable for general photography. Among other considerations, it does not appear to be color managed. It converts my D800e images into an untagged color space. ACR/LR (among others) is better suited for general photography. What about printing with profiles and softproofing?

Iris is another astronomical program favored by techies for analysis of raw files. It can split the CFA image into its channels (RGGB), perform mathematical operations, and perform demosaicing, WB and gamma corrections. It also does Richardson-Lucy deconvoluiton and VanClittret filtering. It also is not color managed. For general photography, I will stick to ACR/LR.

Regards,

Bill

I suggest MATLAB. Most anything that can be done to a dataset in a usual programming language (such as c++, java and whatever photoshop is implemented in) can be done in MATLAB. The trade-off is that implementation effort is lower, while execution time is higher. There are specialized "toolboxes" for e.g. image processing. It even defaults to 64-bit float precision :-)

Calling dcraw (used as a proprietary raw format parser only, not for its development capabilities), I have setup simple development scripts using on the order of 50 lines of script code. Using only well-defined, mathematical vector-matrix operations.

fname = 'IMG_4311';
system(['./dcraw -v -i ', fname, '.CR2'])
system(['./dcraw -D -4 -T ', fname, '.CR2']);
im = double(imread([fname, '.tiff']));
im = im(1:2*floor(end/2), 1:2*floor(end/2)); %crop to even multiple of 2 for simplicity

%% normalization
im = im - 2^11;%remove bias
im = im ./ (2^14-2^11);%normalize

%% separate channels, naiive demosaicing
kernel1 = [1 2 1;...
           2 4 2;...
           1 2 1]./4;
kernel2 = [0 1 0;...
           1 4 1;...
           0 1 0]./4;     
r = zeros(size(im));
r(1:2:end-1,1:2:end-1)  = im(1:2:end-1,1:2:end-1);
r = conv2(r, kernel1, 'same');

g = zeros(size(im));
g(1:2:end-1,2:2:end  ) = im(1:2:end-1,2:2:end  );
g(2:2:end  ,1:2:end-1) = im(2:2:end  ,1:2:end-1);
g = conv2(g, kernel2, 'same');

b = zeros(size(im));
b(2:2:end  ,2:2:end  )  = im(2:2:end  ,2:2:end  );
b = conv2(b, kernel1, 'same');


%% white balance
white_patch = [1800 1200];%manually found "white patch"
r = r./r(white_patch(1), white_patch(2));
g = g./g(white_patch(1), white_patch(2));
b = b./b(white_patch(1), white_patch(2));

%% assembly and clipping
im2 = cat(3, r, g, b);
im2(im2<0) = 0;
im2(im2>1) = 1;

%% plot
figure
image(im2)

Results in the attached image. No part of my script is claimed to be "high quality" or even free of bugs. The sole purpose of my post is to show the relative ease of doing everything yourself if you are interested in the technical details.

-h