Pages: [1] 2 3   Go Down

Author Topic: Comparing RAW-to-JPEG -vs- JPEG only  (Read 16860 times)

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« on: December 20, 2008, 05:15:52 pm »

Here's something I haven't seen elsewhere that I can remember - a comparison of a RAW converted to JPEG and an image shot as JPEG.

These are not the same image - they were taken side by side a few seconds apart, with the camera first set to shoot RAW, and for the second image set to shoot JPEG only.  The camera was a Panasonic FZ50, 1/1.8 CCD.  The RAW image was converted to JPEG using Silkypix, with no processing other than a slight crop and reduction to the 2048 x 1536 size.  Same crop and reduction with the image taken as JPEG.

I have heard (and can't verify) that the difference is the number of color bits in the RAW-to-JPEG process being more than what I got from the in-camera JPEG.  I've done several tests like this with the Panasonic LX3, and could not see any difference.  Of course, there's an advantage in using RAW for post processing, but that aside, it's surprising to me that I don't see a default difference in the RAW-to-JPEG conversion with the LX3 and Silkypix, using the same defaults as were used for the FZ50 images.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #1 on: December 20, 2008, 06:03:19 pm »

With Raw, you're controlling the rendering. Every Raw converter will produce, at least by default, a different rendering.

JPEG is rendered in camera from the Raw. You have virtually no control over this (expect for Picture styles).

Its the norm that the two don't match although the camera manufacturers and now, some 3rd party converters like ACR/LR attempt to match, at least as an initial default, the JPEG but its usually not spot on.

You might want to glance at this article:
http://www.color.org/ICC_white_paper_20_Di...ment_basics.pdf
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #2 on: December 20, 2008, 07:53:23 pm »

Quote from: digitaldog
With Raw, you're controlling the rendering. Every Raw converter will produce, at least by default, a different rendering.

JPEG is rendered in camera from the Raw. You have virtually no control over this (expect for Picture styles).

Its the norm that the two don't match although the camera manufacturers and now, some 3rd party converters like ACR/LR attempt to match, at least as an initial default, the JPEG but its usually not spot on.

You might want to glance at this article:
http://www.color.org/ICC_white_paper_20_Di...ment_basics.pdf
It's a good beginner's article, but doesn't (probably can't) say much about how different sensors record the data, what the camera manufacturer does with the sensor data before producing the RAW image, etc.  I was hoping someone would remember the info I read a couple years ago about the number of bits of color space (or something like that) that are available in RAW's -vs- JPEG's, or if that's not always different, whether Panasonic, for example, is not doing as good a job with their LX3 product as they did with their FZ50.

The main reason I bring this up is I took about 7,000 RAW photos in the Bolsa Chica wetlands between 2/05 and 6/07 using the FZ50, and I have a pretty good feel for the image quality I was getting with that camera.  But I'm not getting as good quality with the LX3, even though the CCD is larger, the lens is supposedly better, and the electronics much improved after two years of research.  Since Panasonic made such a *big* deal about the LX3 being designed by and for photographers rather than the people who usually buy digicams, and since nearly all photo blog owners sang that chorus very loudly, I expected a lot more from the LX3, but don't see the results.  Maybe I just fell for a scam, paying twice what the little gadget was worth.  God help the people who buy into these promotions when the kit costs $5000 instead of $500.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #3 on: December 20, 2008, 08:23:07 pm »

Quote from: dalethorn
I was hoping someone would remember the info I read a couple years ago about the number of bits of color space (or something like that) that are available in RAW's -vs- JPEG's

You need to understand, that

1. you are comparing a JPEG image with another JPEG image, thus the ultimate limitations are the same, no matter which converter created that JPEG,

2. the in-camera JPEG too has been created from the raw data, which then has been discarded.

Thus you are looking for a non-existent or negligable factor. The reason of recording raw data is, that that this way you have many options before the conversion in JPEG, and you can convert it in other, non-lossy formats as well, fore example for greater bit-depth, not only in JPEG.

See it so: creating JPEG in-camera is a blindfolded raw conversion: you decide about the adjustments before you see the shot.
Logged
Gabor

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #4 on: December 20, 2008, 09:33:43 pm »

Quote from: Panopeeper
You need to understand, that

1. you are comparing a JPEG image with another JPEG image, thus the ultimate limitations are the same, no matter which converter created that JPEG,

2. the in-camera JPEG too has been created from the raw data, which then has been discarded.

Thus you are looking for a non-existent or negligable factor. The reason of recording raw data is, that that this way you have many options before the conversion in JPEG, and you can convert it in other, non-lossy formats as well, fore example for greater bit-depth, not only in JPEG.

See it so: creating JPEG in-camera is a blindfolded raw conversion: you decide about the adjustments before you see the shot.
I either don't understand what you're saying, or you're confused.  You can't possibly believe that the in-camera converter for a $500 all-in-one camera with a 1/1.8 CCD will produce a JPEG that's the same as the separate RAW converter software.

And I do distinctly remember an article saying something about in-camera JPEG's not using all of the color bits that a separate converter can apply, but then, that's a technical issue that I may have to look into elsewhere.
Logged

Farmer

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2848
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #5 on: December 20, 2008, 10:08:32 pm »

Quote from: dalethorn
I either don't understand what you're saying, or you're confused.  You can't possibly believe that the in-camera converter for a $500 all-in-one camera with a 1/1.8 CCD will produce a JPEG that's the same as the separate RAW converter software.

And I do distinctly remember an article saying something about in-camera JPEG's not using all of the color bits that a separate converter can apply, but then, that's a technical issue that I may have to look into elsewhere.

Sure it can.  You said yourself that you didn't post process the shot - it was just a straight up conversion.  The point of raw (and it's raw, not RAW - it's not an acronym :-) is that it allows you to do more with the image.  If you're not taking advantage of that then there's every reason to expect similar results are possible.  The manufacturer, after all, knows a lot about their camera specifically whereas raw conversion software is designed for many cameras.

A fair test of what you want to compare would be the JPEG from the camera compared to the best possibly processing of the raw file, rather than just the default processing.

Bear in mind, too, that a technically good shot won't suffer to the degree that a poor shot would - so if you start with sharp, correctly exposed images then the in-camera conversion will be more successful and there's less demand for raw to gain extra benefit.

As for how much of the data the in-camera conversion uses, this will vary from camera to camera.  Yes, I'd expect my A700 would have more processing power than your FZ50 and if you look at JPEG files produced using the DRO enhancment it's clear that the A700 is doing more but then it costs more, so that's hardly surprising.  Similarly, I'd bet that the processing in a 1DSmkIII or a D3x is also superior to the FZ50 - the accompanying price tag really demands it.

Overall, I'm not sure what you're getting at, though.  Are you worried that your raw convertor isn't doing a good job or are you surprised by the capabilities of your camera?  As I said, compare the best in-camera JPEG with the best post-processed raw and you'll start to see difference, particularly if the original image has some technical issues.
Logged
Phil Brown

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #6 on: December 20, 2008, 10:23:53 pm »

Quote from: dalethorn
I either don't understand what you're saying, or you're confused
Well, let's try to figure out, which one.

Quote
You can't possibly believe that the in-camera converter for a $500 all-in-one camera with a 1/1.8 CCD will produce a JPEG that's the same as the separate RAW converter software
Why would this depend on the price of the camera and/or on the size of the CCD? Do you think that one or two frames per sec is something extraordinary load? Modern DSLRs are creating up to TEN images per second, larger ones than the FZ-50.

Quote
And I do distinctly remember an article saying something about in-camera JPEG's not using all of the color bits that a separate converter can apply, but then, that's a technical issue that I may have to look into elsewhere
Not to keep you from looking into that elsewhere, but as long as you don't understand even your questions, you will have difficulty understanding the answers.

The "color bits" (your term), which are not used by the in-camera converter do not represent quality but quantity. Your camera creates a raw image of 3671x2748 pixels, but you get to see only 3648x2736. This is, because Panasonic decided (with good reason) to declare a size, which is

a. a multiple of 8 pixels in both dimensions,

b. it maintains the proportion 4:3.

However, if the camera creates a raw file, then all pixels are there and the raw converter could create a somewhat larger image. LR/ACR will create the "standard" size (as Panasonic declared it), but I can imagine, that for example Raw Shooter Premium 2007 created a larger image (they messed up my files too with funny sizes).
Logged
Gabor

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #7 on: December 20, 2008, 10:24:56 pm »

Quote from: Farmer
Sure it can.  You said yourself that you didn't post process the shot - it was just a straight up conversion.  The point of raw (and it's raw, not RAW - it's not an acronym :-) is that it allows you to do more with the image.  If you're not taking advantage of that then there's every reason to expect similar results are possible.  The manufacturer, after all, knows a lot about their camera specifically whereas raw conversion software is designed for many cameras.

A fair test of what you want to compare would be the JPEG from the camera compared to the best possibly processing of the raw file, rather than just the default processing.

Bear in mind, too, that a technically good shot won't suffer to the degree that a poor shot would - so if you start with sharp, correctly exposed images then the in-camera conversion will be more successful and there's less demand for raw to gain extra benefit.

As for how much of the data the in-camera conversion uses, this will vary from camera to camera.  Yes, I'd expect my A700 would have more processing power than your FZ50 and if you look at JPEG files produced using the DRO enhancment it's clear that the A700 is doing more but then it costs more, so that's hardly surprising.  Similarly, I'd bet that the processing in a 1DSmkIII or a D3x is also superior to the FZ50 - the accompanying price tag really demands it.

Overall, I'm not sure what you're getting at, though.  Are you worried that your raw convertor isn't doing a good job or are you surprised by the capabilities of your camera?  As I said, compare the best in-camera JPEG with the best post-processed raw and you'll start to see difference, particularly if the original image has some technical issues.
Well, the best post processing issue is another subject I wasn't interested in here.  The photos I posted *did* show a substantial difference even though they were shot side by side with the same camera and settings, seconds apart.  That seems to contradict what you said.  I'm suspecting that the reason the LX3 JPEG's don't look different from the RAW's is some bad design by Panasonic, *not* doing the best RAW that they can (RW2 actually, where the FZ50 was RAW).
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #8 on: December 20, 2008, 10:28:54 pm »

Quote from: Farmer
As for how much of the data the in-camera conversion uses, this will vary from camera to camera
LOL, a blind leader of the blind.
Logged
Gabor

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #9 on: December 20, 2008, 10:40:39 pm »

Quote from: Panopeeper
LOL, a blind leader of the blind.
Well, that's really observant (not).  You still have blindly (your term) missed the point - that the FZ50 images are very different and the LX3 images (not posted here for reasons stated) are not.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #10 on: December 20, 2008, 10:50:02 pm »

Quote from: dalethorn
You still have blindly (your term) missed the point - that the FZ50 images are very different and the LX3 images (not posted here for reasons stated) are not.
There is no "point" until you upload the full size JPEGs created in-camera and by Silkipix, as well as the raw file. THEN we can talk about the differences.
Logged
Gabor

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #11 on: December 20, 2008, 10:55:22 pm »

Quote from: Panopeeper
There is no "point" until you upload the full size JPEGs created in-camera and by Silkipix, as well as the raw file. THEN we can talk about the differences.
You need to learn to read.  I did not post the RAW file because there was no point to it.  After all, it *is* my point, not yours.
Logged

Farmer

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2848
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #12 on: December 20, 2008, 11:15:33 pm »

Quote from: Panopeeper
LOL, a blind leader of the blind.

How so, Gabor?  Different cameras will process the data internally to bake a JPEG.  Without having the manufacturer's specifications I couldn't tell you what those differences are, but do you know for sure that all the cameras make use of the full bit-depth and how they do it when producing a raw?  Do all 14bit capture cameras use all 14bits to render and then convert to 8bit or do they discard to 8bit first?  

My point stands that the question of how much data is used will vary from camera to camera.  You can make snide remarks all you like, but that's an (obvious) fact.
Logged
Phil Brown

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #13 on: December 21, 2008, 12:24:11 am »

Quote from: Farmer
Different cameras will process the data internally to bake a JPEG
Certainly; however, the subject is the processing of the raw data.


Quote
do you know for sure that all the cameras make use of the full bit-depth and how they do it when producing a raw?  Do all 14bit capture cameras use all 14bits to render and then convert to 8bit or do they discard to 8bit first?
There are some problems in this workflow. This is not a question of bit depth; the raw data has to be "developed" through several stages, and the result will not become an 8bit JPEG by way of ignoring four or six bits.

(Btw, the Panasonic FZ-50 creates 12bit data.)

I am confident, that the camera firmware uses all available data for the demosaicing, noise reduction, white balance, saturation, contrast, ...

It is important to realize, that the greater bit depth does not play much (or any) role in direct JPEG creation, because the increased bit depth is needed when making "strong" adjustments in post processing. Still, I don't think raw data will be ignored, simply because that would be more difficult than taking the pixel values as they are.

The 8bit JPEG is the result of a complex process at the very end of this procedure. That is the stage, when huge differences can arise, namely from the JPEG "quality" setting. Dalethorn posted above

The RAW image was converted to JPEG using Silkypix, with no processing other than a slight crop and reduction to the 2048 x 1536 size

However, the JPEG quality is not among the factors one can equalize, and that can account for lots of differences.
Logged
Gabor

Farmer

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2848
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #14 on: December 21, 2008, 01:00:44 am »

Quote from: Panopeeper
Certainly; however, the subject is the processing of the raw data.



There are some problems in this workflow. This is not a question of bit depth; the raw data has to be "developed" through several stages, and the result will not become an 8bit JPEG by way of ignoring four or six bits.

(Btw, the Panasonic FZ-50 creates 12bit data.)

I am confident, that the camera firmware uses all available data for the demosaicing, noise reduction, white balance, saturation, contrast, ...

It is important to realize, that the greater bit depth does not play much (or any) role in direct JPEG creation, because the increased bit depth is needed when making "strong" adjustments in post processing. Still, I don't think raw data will be ignored, simply because that would be more difficult than taking the pixel values as they are.

The 8bit JPEG is the result of a complex process at the very end of this procedure. That is the stage, when huge differences can arise, namely from the JPEG "quality" setting. Dalethorn posted above

The RAW image was converted to JPEG using Silkypix, with no processing other than a slight crop and reduction to the 2048 x 1536 size

However, the JPEG quality is not among the factors one can equalize, and that can account for lots of differences.

Thank you - this is a good reply.  If I'm wrong and someone explains it to me, then I learn something.  In this case, it has extended my understanding, so thank you again.

And, FWIW, I mentioned 14bit just as an example, not to claim that the FZ50 was 14bit :-)

Regarding bit depth, I would have thought that a camera that had limited processing power (so probably not current DSLRs) would consider reducing to 8bit from their native capture earlier in the process to reduce processer overhead.  Your suggestion is otherwise, and does make sense.  Some cameras that capture in JPEG have a faster capture rate than their raw equivelant.  Is this just write speed or does processing time come into it, I wonder?
Logged
Phil Brown

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #15 on: December 21, 2008, 01:38:15 am »

Quote from: Farmer
Some cameras that capture in JPEG have a faster capture rate than their raw equivelant.  Is this just write speed or does processing time come into it, I wonder?
I am not sure if this can be answered generally. However, at least some cameras can create dozens of JPEGs or dozens of raw continuously, but less raws than JPEG, even though creating the raw file of compressed raw data is less computing intensive than creating the JPEG files. Apparently the buffer is the limit, i.e. the limit ultimately is write speed.

It is not useful to think about the raw processing and JPEG conversion in terms of general computers, like PCs. Though the camera can not host an equivalent computing power, it has a huge advantage: the specialized task. Without knowing any details for example of the DIGICs, I am sure that these processors are specialized for image processing. That means, that they have special instructions for complex tasks; this way they can easily surpass the rate of desktop computers.

For example the "customary" JPEG encoding method (the one used in the JPEG images) is very, but VERY computing intensive. Even the JPEG encoding of Canon and Nikon and many other raw data (which consists of part of the "complete" JPEG encoding) is so computing intensive, that processing (reading and displaying) an uncompressed raw file takes much less time on a PC than the same with a compressed file, even though the amount of data to read is hugely different. This can be tested by ACR, via converting a raw in compressed and uncompressed DNG (the DNG compression is the same as the one in CR2 and in NEF, etc.). I know that this is surprizing; I have seen several times posting the "natural" assumption, that processing of the compressed files ought to be faster than that of the uncompressed files, due to the difference of data to read from disk.

However, a processor, which carries out a chunk of bit fiddling and cosine calculations, etc. can turn around the proportions. Therefor it is not so surprizing, that the camera can surpass commercial computers by a large factor in image processing.
Logged
Gabor

dalethorn

  • Guest
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #16 on: December 21, 2008, 06:34:25 am »

Quote from: Panopeeper
.....Though the camera can not host an equivalent computing power, it has a huge advantage: the specialized task. Without knowing any details for example of the DIGICs, I am sure that these processors are specialized for image processing. That means, that they have special instructions for complex tasks; this way they can easily surpass the rate of desktop computers.....
.....However, a processor, which carries out a chunk of bit fiddling and cosine calculations, etc. can turn around the proportions. Therefor it is not so surprizing, that the camera can surpass commercial computers by a large factor in image processing.
Where I come from a processor is hardware, and I find it hard to believe that Nikon etc. are making processors with floating point hardware superior to any of the many PC numeric chips. Algorithms in software are another matter, but again, it doesn't make sense that camera software for rendering JPEG's would be much better than PC software. So for the specialized camera chip - who would be fabricating the actual chips for all cameras, and how could all of those designs have escaped the various PC chip makers? A whole different set of hardware chip mfr's, just for cameras? Note I said mfr's.
Logged

michael

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5084
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #17 on: December 21, 2008, 08:40:16 am »

Dale,

You've come to ask a question and then when you get some answers (from knowledgeable people) you argue with them.

It's clear that you don't understand the fundamentals of of raw image processing and jpg conversion, so my suggestion is that you keep asking questions, keep reading, but don't be belligerent when you get answers that you have trouble with.

Finally, have you ever heard of ASICs? Application Specific IC's?

These are semiconductors that are specifically designed to do one things really well and really fast. That's what's in a digital cameras to convert raw data into JPGs. They can, as has been pointed out, do as many as 10 images a second, and then are held back by buffer size more than anything else. Even a fast desktop PC with the fastest general purpose CPU would be hard pressed to match this.

Hope this partially answers your question.

Michael

Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #18 on: December 21, 2008, 12:33:08 pm »

Quote from: dalethorn
It's a good beginner's article, but doesn't (probably can't) say much about how different sensors record the data, what the camera manufacturer does with the sensor data before producing the RAW image, etc.  I was hoping someone would remember the info I read a couple years ago about the number of bits of color space (or something like that) that are available in RAW's -vs- JPEG's, or if that's not always different, whether Panasonic, for example, is not doing as good a job with their LX3 product as they did with their FZ50.

Most digital cameras encode the raw data in a linear fashion (gamma = 1.0), where the output data number is proportional to the luminance, and use a bit depth of 12 or 14 bits. JPEGs are usually 8 bits per channel with a gamma of 2.2. Twelve, fourteen, and 8 bits correspond to 4096, 16384, and 256 levels respectively. When converting from a gamma of 1.0 to 2.2, levels are lost as can be demonstrated by Bruce Lilndbloom's levels calculator. As Norman Koren explains, an 8 bit image at a gamma of 2.2 can usually encode the dynamic range of a typical photograph, but just barely. If your exposure or tone curve is less than optimal, your image will suffer when you edit it. However, if you are shooting raw at 12 or 14 bits, you may still have enough levels to get a good image. Although white balance can be adjusted in JPEGs, it is best done with the raw data where you have more levels to work with.

In view of the above, if your exposure, white balance, and tone curve are perfect, a JPEG may be fine; however, if these parameters are less than perfect, a raw file gives you much more flexibility.

Color gamut is also a consideration. Most digital cameras offer only aRGB or sRGB with JPEG output. Modern digital cameras can capture a considerably higher range of colors than these spaces can accommodate, and modern inkjet printers can print some of these colors. To take advantage of the capabilities of your camera, it is best to shoot raw and render into 16 bit ProPhotoRGB or a similar wide gamut space. This option is not available in JPEGs.

Finally, sharpening and noise reduction are important considerations and these parameters are baked into a JPEG conversion. The camera applies a global unsharp mask to the image, whereas modern sharpening techniques use masks and blending options to produce better results.

Bill

Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Comparing RAW-to-JPEG -vs- JPEG only
« Reply #19 on: December 21, 2008, 01:04:32 pm »

Quote from: dalethorn
I find it hard to believe that Nikon etc. are making processors with floating point hardware superior to any of the many PC numeric chips
Not "superior" but "specialized".

Quote
Algorithms in software are another matter, but again, it doesn't make sense that camera software for rendering JPEG's would be much better than PC software
What I tried to explain unsuccessfully is, that the advantage of special purpose processors is not in a faster hardware nor in better algorythm (there are no secrets in these algorythms) but in the implementation of the algorythms in hardware. Although even individual processor inctructions can be made faster in special purpose processors than in the generic ones, that's not the point.

Both processing of the raw image, i.e. converting it in an RGB one, and the conversion of that in JPEG involve complex algorythms operating on a group of pixels (on up to 1024 pixels in the JPEG conversion). Carrying out these steps requires thousands of processor level instructions in loops. Implementing these algorythms in hardware can shorten the processsing time to a fraction. The same is true regarding the Huffman encoding of the raw data if it is involved at all.

Quote
So for the specialized camera chip - who would be fabricating the actual chips for all cameras
This is irrelevant regarding the topic, but I think Canon manufacturer their own processors (like the DIGICs, I don't know what is in the smaller cameras). I guess so does Sony. Some have to buy it from specialized chip manufacturers. Who is making the chips for cell phones, computerized toys, washing machines, GPS, cars, etc.?
Logged
Gabor
Pages: [1] 2 3   Go Up