Jpeg compression

[David Sutton]

I'm trying to get an idea of how much data is lost in jpeg compression. So I took a few test shots with the camera set to raw+jpeg(L) to compare. File sizes 11.67 and 3.02mb. (or MB - I can never remember). Looking at them on screen (1:1) I'd guess about 10 - 20% of the image has gone down the photographic toilet bowl with jpeg. Some fine detail lost and subtle colours smoothed over.
To try to compare "apples with apples" I converted the raw file to a compressed 8 bit tiff and got a file of 12.15mb. Too big a difference I would have thought compared to what I'm seeing.
So I tried converting both the jpeg and raw to 8 bit tiffs and both came out to exactly 28.85 mb. Checked this twice, so no joy there.
Curiously, when I converted the raw file to a jpeg in LR, it gave a file size of 4.74mb. 4.75 to 3.02 mb is a 28% loss I think, which is closer to my guesstimate. But I can't see why this should be so. Unless LR's "large" conversion is bigger than my camera's "large".
So before I start printing to look at what can be seen there, I wondered if anyone can help with suggestions/opinions? Cheers, David

[Panopeeper]

Before answering your questions, I'd like to know, which camera you are talking about, in order to relate the posted raw file size.

[David Sutton]

Canon 40D

[Gordon Buck]

Some data is lost but most is probably only compressed.  Try placing the jpg on top of the raw as a Photoshop layer and subtract to see the difference.

[Panopeeper]

To try to compare "apples with apples" I converted the raw file to a compressed 8 bit tiff and got a file of 12.15mb. Too big a difference I would have thought compared to what I'm seeing

Compared to what? To the raw or to the JPEG? Or did you mean compared to the quality?

The raw file contais 10Mpix, 12bit depth, the TIFF contains 10Mpix, 3*8bit depth. Both are compressed.

The size is in now eway an indicator of quality, it can be even the opposite way: noisy images are less compressible than clean ones. On the other hand, large smooth, uniform surfaces can be compressed very good.

So I tried converting both the jpeg and raw to 8 bit tiffs and both came out to exactly 28.85 mb. Checked this twice, so no joy there

You did not specify compression. Btw, LZW is faster and in 8-bit form sometimes better than ZIP; however, ZIP is much better than LZW in 16bit (I hope I remember correctly and it is not just the other way).

Curiously, when I converted the raw file to a jpeg in LR, it gave a file size of 4.74mb. 4.75 to 3.02 mb is a 28% loss I think, which is closer to my guesstimate

There is no basis for guestimation. I know the JPEG construction in intimate details and I would refrain from guessing.

The JPEG encoding is governed by usually 128 parameters plus some basic decisions. The concepts of "small", "large", "low quality", "high quality" are fictions. Each program has its own sets of parameters from which to select depending on whatever basis. Some programs display a selection in percentage; pure nonsense, there is nothing to do in percentage there (but the poor user has to make some decision).

My suggestion is to archive everything in raw, archive the adjustment parameters (sidecar file), generate TIFFs, post process those and keep the results in that form. From there you can create JPEGs of different sizes and different qualities, after some final adjustments, particularly sharpening. You need different sizes and different sharpenings for printing, for high-quality web presentation and for lower quality.

[David Sutton]

Some data is lost but most is probably only compressed.  Try placing the jpg on top of the raw as a Photoshop layer and subtract to see the difference.
[a href=\"index.php?act=findpost&pid=181253\"][{POST_SNAPBACK}][/a]

A very helpful suggestion. Thank you very much. David

[David Sutton]

Compared to what? To the raw or to the JPEG? Or did you mean compared to the quality?

The raw file contais 10Mpix, 12bit depth, the TIFF contains 10Mpix, 3*8bit depth. Both are compressed.

The size is in now eway an indicator of quality, it can be even the opposite way: noisy images are less compressible than clean ones. On the other hand, large smooth, uniform surfaces can be compressed very good.
You did not specify compression. Btw, LZW is faster and in 8-bit form sometimes better than ZIP; however, ZIP is much better than LZW in 16bit (I hope I remember correctly and it is not just the other way).
There is no basis for guestimation. I know the JPEG construction in intimate details and I would refrain from guessing.

The JPEG encoding is governed by usually 128 parameters plus some basic decisions. The concepts of "small", "large", "low quality", "high quality" are fictions. Each program has its own sets of parameters from which to select depending on whatever basis. Some programs display a selection in percentage; pure nonsense, there is nothing to do in percentage there (but the poor user has to make some decision).

My suggestion is to archive everything in raw, archive the adjustment parameters (sidecar file), generate TIFFs, post process those and keep the results in that form. From there you can create JPEGs of different sizes and different qualities, after some final adjustments, particularly sharpening. You need different sizes and different sharpenings for printing, for high-quality web presentation and for lower quality.
[a href=\"index.php?act=findpost&pid=181280\"][{POST_SNAPBACK}][/a]

That is some useful information. Thank you. I haven't used jpegs for a long time and am preparing a small comparison for some photo enthusiasts on the differences that show up when printing from a jpeg vs a raw file.

[David Sutton]

It's been a useful exercise. Printed three 8x10s on an Epson Stylus RX630 multifunction- not exactly a high end printer. One photo from the raw exported as a 16 bit prophoto tiff, one jpeg in Adobe rgb and one jpeg in srgb. It's proven hard to tell the two colour spaces apart, and both jpegs look,well, garish. Not a big surprise, but what I wasn't expecting is how bad the detail in the jpegs look compared to the tiff, worse than appeared on screen. I thought I would need to do some enlargements to show the difference, but at 8x10 the jpegs frankly stink.
Also used "save as" ten times on one of the jpegs. One screen the resulting loss of quality was quite apparent (but I needed at least 1:1 to see it). Interestingly, this was very hard to reproduce in print, and I haven't really succeeded in doing this. David

[AJSJones]

A quick way to assess the effect is to use the preview image when saving as jpeg in Photoshop - and slide the quality slider, then click the preview on/off box.  You can still zoom in and out of the image.

[Morgan_Moore]

am preparing a small comparison for some photo enthusiasts on the differences that show up when printing from a jpeg vs a raw file.
[a href=\"index.php?act=findpost&pid=181336\"][{POST_SNAPBACK}][/a]

I would think the differences in print can be minimal unless the jpg compression is massive

but its how you get to that jpg that counts

Shoot a potrait under tungsten RAW and JPG

Try and take out the yellow cast in both files then print them - big difference

Take a wide dynamic range image

Layer a dark raw conversion and a light raw conversion to make one print file

dodge and burn the camera jpg to make the second print file - big difference

The adantage of raw is that one is storing more information than is printable

I often deliver jpg to the client (10 or 12)- but that is a jpg made right at the end of any raw processing or retouching of the Tiff from Raw file


Of course there a visible differences with a jpg saved at 1 or 2

And jpgs cannot handle further file manipulation like Tiffs can

JPG v RAW is about getting to the route to the last file (that is printed or delivered to the client)


----


Think about this : compression does not inherently damage files - compression is just an efficient way outputting a set instructions

If you want to paint your whole wall red you can instruct you painter in two ways

'Bill - paint the wall red' (a compressed instruction)

'Bill paint brick A1 red, B1 red  ect to Brick z90 red' (an uncompressed instruction)

Both get the same result - the first was quicker and far more efficient

Tiff is uses the second method JPG the first

Now using the second method will become more accurate the more complex a pattern you want painted on the bricks on your wall - or if you change your mind about the pattern

Even an instruction like

'Bill paint the top third of the wall red and the rest blue' still gets the required simple pattern with a compressed instruction

it just start going wrong when you say 'Bill paint a face on the wall' - that is ambigous instruction and could be interpreted inaccurately - you still get a face -but maybe not quite the one you wanted - but it you stand far enough away from the wall it might start to like like the one you wanted and meet your requirements

I think that that shows how jpg does not always mean bad because as long as the instruction outresolves the output device there will be no visible difference


SMM

[Justinr]

Just as an aside, Jon Tarrant of BJP asked a similar question a couple of years back and came to the conclusion that after re-saving a JPEG 18-19 times no further information is lost.

Justin.

[NikoJorj]

I'm trying to get an idea of how much data is lost in jpeg compression.
[{POST_SNAPBACK}][/a]

My 2c's : what you see between "unprocessed" jpegs and raw (I mean not edited by you) is merely the difference between the default in-camera jpeg treatment, and the default in-computer RAW treatment.

The information lost with reasonable jpg compression is essentially invisible to the naked eye, and really shows itself only after significant edits.
See [a href=\"http://www.luminous-landscape.com/tutorials/jpg-follies.shtml]http://www.luminous-landscape.com/tutorial...g-follies.shtml[/url] for examples!

[Tim Gray]

For 99% of my files I shoot RAW and process in 16 (14bits) until I'm ready to save the file - then it gets saved in JPG, quality 11, sbsequent output sharpening, up rezzing (if any) and printing, including soft proofing is against the JPG.  I will also do minor tweaks (if necessary) against the JPG, opening and resaving at quality 11.  

What I did to get a sense of the lossyness of successive JPG compressions is take the final jpg from the above workflow, open and resave it 10 times.  Take the last version, copy it into the first and set blend mode to difference - at this point there is no visible difference even at 100%.  If you want to magnify the difference (because there is some) run auto levels (you can see slight evidence of the difference on the very left of the histogram prior to running auto levels).  Bottom line, even if I have to adjust the file a couple of times from the JPG there's no visible degradation due to JPG compression.  If the file requires major surgery then it's back to the original RAW.

The other 1% (I still start out with this workflow) are the files that end up being printed large for more formal disply - prints of 17x25.  I take the initial "draft proof" and reprocess from the beginning, but keep all the layers and that gets saved as PSD.

I don't hold this out as any kind of best practice, but regardless of how cheap memory is, this seem to me to be a reasonably balance between quality and managing terabytes of archives.

[Panopeeper]

Misguided "tests" of the JPEG encoding lossiness

I saw above  the result of repeated saving as a kind of "endurance test". Well, this proves nothing at all.

The JPEG encoding is a combination of several lossy and lossless steps. One of the causes ofthe lossyness is the replacement of the original values with parameters (coefficients) of a function called discrete cosine transform (a kind of Fourier transform). This is an approximation, which is very lossy, but once it is done, it can be repeated with minimal loss, even rounding errors will return. However, this process is carried out on blocks of 8x8 pixels, from the top left of the image. Repeating the process does not cause harm, as long as these blocks remain unchanged. On the other hand, the any shift in the box boundaries requires new transformation with the associated losses.

The other lossy step is the quantization of the coefficients. "Quantization" is an euphemism, it means plainly reducing, in most cases throwing away the coefficients. There is a set of 2*64 values (divisors) governing  this reduction; when selecting a certain "compression quality", the program picks a different set of divisors (the higher the divisors, the less data remains, the lower the resulting quality will be). There is no standard for these values, every program does it at the will of the designer. Repeating the encoding process with the same divisors does not increase the lossiness, but changing the divisors can be disasterous.

Accordingly, new loseses are caused without changing the image content by

- changing the boundaries of the 8x8 boxes. Example: rotation (if the image dimensions are not a multiple of ; cropping or adding one to seven pixels from/to the top or left edge,

- changing the compression quality. This is the most important point: increasing the "quality" setting of an already encoded image reduces the quality.

So, if one wants to know the effect of multiple JPEG encoding, one should make such changes.

[Justinr]

Panopeeper

Some of that is a bit beyond me but I think I get your drift. The article was written three years ago at least and my memory is not the best one ever known. I believe Jon set out to investigate the claims that repeated saving of images as jpegs resulted in continued deterioration of the files. After 19 attempts he felt that there was no longer any loss of detail. The parameters he set himself are lost in the mists of time but I know that he is a physicist by education so I imagine he had some idea of how to approach the subject.

Many thanks for the post anyway, it certainly pointed the way to a little more understanding of the issue.

Justin.