How to convince Sony to do lossless raw on A7RII and others?

[dreed]

In various interviews that Sony folks have done since the release of the A7RII, they have all indicated that they have heard of the desire for lossless (or uncompressed?) raw images. Some have gone as far as to say that they'll listen to what customers want before doing anything.

So that then raises the question, how do we (the customer) make ourselves heard to them (Sony management) on this topic?

[BernardLanguillier]

At least by not buying the camera until it does support lossless compressed raw?

Cheers,
Bernard

[Herbc]

+1

[Jack Hogan]

Other than the delta business, has anybody shown lost image information from Sony's 11 bit 'gamma' compression?  The more I think about this type of compression (also used in one of Nikon's compressed NEF modes) the more I am becoming convinced that if done properly it just drops storage bits - but not one bit (:-) of IQ.  Aggressive PP included.  Any examples of lost information, anybody?

[ErikKaffehr]

+1

Erik

Other than the delta business, has anybody shown lost image information from Sony's 11 bit 'gamma' compression?  The more I think about this type of compression (also used in one of Nikon's compressed NEF modes) the more I am becoming convinced that if done properly it just drops storage bits - but not one bit (:-) of IQ.  Aggressive PP included.  Any examples of lost information, anybody?

[MatthewCromer]

Someone was complaining about using Sony to photograph clouds. Claimed the highlight compression reduced tonalities in the whites, or something.

[Telecaster]

Someone was complaining about using Sony to photograph clouds. Claimed the highlight compression reduced tonalities in the whites, or something.

Bah…I've done some cloud pic-taking with my A7r (lotsa opportunities in the new reality of five-month spring, no summer, here in Michigan) and have bashed the RAWs about pretty hard. Nada in terms of clear tonal compression/posterizing issues. That is, issues beyond what any normal 12- or 14-bit file will exhibit if you go too far. For that matter I don't think I've seen any real-world photos that clearly show such issues…just RawDigger plots.

I'd like to see Sony offer a loss-less RAW option. But I won't lose any sleep over it.

-Dave-

[BrianVS]

Just don't buy the camera until the firmware does justice to the sensor.

Maybe the other camera manufacturers are bribing the decision makers of the Sony division to screw-up the image using the lossy compression. Makes Nikon and Canon look better for still photographs. For video- will not matter.

[Jim Kasson]

In various interviews that Sony folks have done since the release of the A7RII, they have all indicated that they have heard of the desire for lossless (or uncompressed?) raw images. Some have gone as far as to say that they'll listen to what customers want before doing anything.

So that then raises the question, how do we (the customer) make ourselves heard to them (Sony management) on this topic?

I've gone to the trouble of writing software to simulate the round trip through the Sony compansion process, and passed both simulated and real images through the sim. I can see errors by subtracting the original image from the companded one, but they are quite small. Under some extreme circumstances, I can see errors by looking at companded images alone. However, my conclusion is that the tone curve/delta mod algorithm will hardly ever cause visible problems in real images, and it's not something I worry much about.

Would I like to see lossless compression offered in addition or instead? Heck, yes.

If they don't change their compression, will I buy more Sony cameras with their lossy algorithm in them? Also, heck, yes.

I think the fact that the a7x cameras change from 13 to 12 bit precision when you switch the shutter to continuous, silent, or bulb is much more worrisome, and I don't see that getting much ink at all.

Jim

[LKaven]

Other than the delta business, has anybody shown lost image information from Sony's 11 bit 'gamma' compression?  The more I think about this type of compression (also used in one of Nikon's compressed NEF modes) the more I am becoming convinced that if done properly it just drops storage bits - but not one bit (:-) of IQ.  Aggressive PP included.  Any examples of lost information, anybody?

(with apologies to Franz Kafka:)

Sony: "We just made the finest 35mm image sensor in the world today"
Me: "Great!"
Sony: "Ok!  You can have images that look just like images from the finest 35mm image sensor in the world today"
Me: "I thought you were going to give me the finest 35mm image sensor in the world?"
Sony: "We totally are...well, we're giving you the next best thing, and you totally can't tell the difference!"
Me: "But...but..."
Sony: "Well, we can't exactly let you use the highest quality setting on the finest 35mm image sensor in the world..."
Me: "But...but..."
Sony: "then you might not think that there was something left that you hadn't tried"

[ErikKaffehr]

Hi Jim,

I would essentially agree. Some oddities observed may come from colour profiles, like this one:


This banding is visible with most DNG-profiles I have generated, but not with Adobe Standard profile in LR. It is also visible in Capture One 7 with their default profile. DCP profiles generated with Anders Torger's DcamProf don't show this issue.

I guess lossy compression gets a lot of blame for this kind of artefacts, but the sample is taken from an "uncompressed" IIQ file on a P45+ back, the ones with 16 bits of fidelity…

Best regards
Erik

I've gone to the trouble of writing software to simulate the round trip through the Sony compansion process, and passed both simulated and real images through the sim. I can see errors by subtracting the original image from the companded one, but they are quite small. Under some extreme circumstances, I can see errors by looking at companded images alone. However, my conclusion is that the tone curve/delta mod algorithm will hardly ever cause visible problems in real images, and it's not something I worry much about.

Would I like to see lossless compression offered in addition or instead? Heck, yes.

If they don't change their compression, will I buy more Sony cameras with their lossy algorithm in them? Also, heck, yes.

I think the fact that the a7x cameras change from 13 to 12 bit precision when you switch the shutter to continuous, silent, or bulb is much more worrisome, and I don't see that getting much ink at all.

Jim

[Jack Hogan]

(with apologies to Franz Kafka:)

Sony: "We just made the finest 35mm image sensor in the world today"
Me: "Great!"
Sony: "Ok!  You can have images that look just like images from the finest 35mm image sensor in the world today"
Me: "I thought you were going to give me the finest 35mm image sensor in the world?"
Sony: "We totally are...well, we're giving you the next best thing, and you totally can't tell the difference!"
Me: "But...but..."
Sony: "Well, we can't exactly let you use the highest quality setting on the finest 35mm image sensor in the world..."
Me: "But...but..."
Sony: "then you might not think that there was something left that you hadn't tried"

I hear you Luke, and I think it makes good marketing sense for Sony to put all fears to bed by providing a lossless compressed mode as well (expect some possible compromises if they do, though, e.g. slower FPS).

But on a practical note, since I seem to recall that we share a fondness for Jazz, I remember participating in A/B listening tests with audio equipment more expensive than a car by like minded supposedly golden eared folks when MP3 was starting to enter the mainstream.  With well encoded material some could hear differences at 128k bps, a few could on very specific rare passages at 192k (top hats come to mind), by 320k nobody (nobody) had a clue as to whether the original master or the compressed version was playing.

I can't speak for the delta thing, but since in 10+ years nobody that I am aware of has been able to show loss of visual information with 12/14-bit captured raw data 'gamma' coded to 10/12-bits in Nikon's 'lossy' NEF compression - the only type of raw file available to many Nikon cameras today including, I believe, the new and outstanding D5500 - I think we are probably talking the visual equivalent of 320kbps audio: nobody can see that the file is smaller, even after aggressive PP.  The data may be different, but the underlying visual information recorded is the same in both file formats.

I am interested in this subject, so I would be happy to be disproved if evidence is shown to the contrary.  Anyone?

Jack

PS BTW, the parallel with audio breaks down because there they actually do throw away recorded audio information on the basis that humans cannot hear it.  The digital photography case for raw data 'gamma' compression is much stronger because here they are simply not using up bits for stuff that the camera cannot record.  Take a look at the market balance example here for an idea of what's happening.

[uaiomex]

+1

At least by not buying the camera until it does support lossless compressed raw?

Cheers,
Bernard

[geesbert]

I am doing this for years now, but alas with limited success:

- I am not buying a Nikon unless they change the fucus ring direction
- I am not buying a Hasselblad unless thy stop selling their spacey Sonys
- I am not buying a Phase one unless they offer a true liveview/mirrorless camera option (Alpa doesn't count)

There are so many cameras I am not buying...

I am sure they are listening!

At least by not buying the camera until it does support lossless compressed raw?

Cheers,
Bernard

[scooby70]

I don't know if you guys spotted this but in a recent interview with Sony engineer and manager Kimio Maki...

"5) Uncompressed RAW: Sony RAW is compressed, not uncompressed. But if we’re getting a lot of requests for it, we should make such a kind of no-compression raw. We recognize the customer’s requirement, and actually we are working on it. And yes Sony could provide that via simple firmware upgrade!"

See here...

http://www.sonyalpharumors.com/sony-interview-and-a-good-news-they-are-working-on-uncompressed-raw/#disqus_thread

It's never been an issue for me but if they do release a firmware update I'm download it and it'll hopefully banish even the thought that compression could be an issue.

[Jim Kasson]

I don't know if you guys spotted this but in a recent interview with Sony engineer and manager Kimio Maki...

"5) Uncompressed RAW: Sony RAW is compressed, not uncompressed. But if we’re getting a lot of requests for it, we should make such a kind of no-compression raw. We recognize the customer’s requirement, and actually we are working on it. And yes Sony could provide that via simple firmware upgrade!"

See here...

http://www.sonyalpharumors.com/sony-interview-and-a-good-news-they-are-working-on-uncompressed-raw/#disqus_thread

It's never been an issue for me but if they do release a firmware update I'm download it and it'll hopefully banish even the thought that compression could be an issue.

As I've said in other conversations, if Sony actually does offer only uncompressed raw in addition to cRAW, that will qualify as malicious obedience in my book. I don't want uncompressed raw. What I want is losslessly compressed raw.

I"m from Missouri on the firmware-only part of what he's saying. The fact that the a7x precision drops from 13 to 12 bits under some conditions feels like hardware, not firmware. The tone curve looks like it's designed for hardware implementation. And I question whether the ADC is really 14 bits in the first place, given the fact that you can only use 13 of them at best. Who would design a 14-bit ADC if they were going to toss the LSB on the floor?

Jim

[Telecaster]

As I've said in other conversations, if Sony actually does offer only uncompressed raw in addition to cRAW, that will qualify as malicious obedience in my book. I don't want uncompressed raw. What I want is losslessly compressed raw.

The Sony guy must mean loss-less RAW. (But if not…are these people high?!) Why would anyone care about uncompressed loss-y RAW when the compressed variety saves space while throwing away no additional data? I also know of no requests for such, as opposed to many & sustained pleas for loss-less RAW.

-Dave-

[BernardLanguillier]

I am doing this for years now, but alas with limited success:

- I am not buying a Nikon unless they change the fucus ring direction
- I am not buying a Hasselblad unless thy stop selling their spacey Sonys
- I am not buying a Phase one unless they offer a true liveview/mirrorless camera option (Alpa doesn't count)

Keep trying!

Cheers,
Bernard

[LKaven]

I hear you Luke, and I think it makes good marketing sense for Sony to put all fears to bed by providing a lossless compressed mode as well (expect some possible compromises if they do, though, e.g. slower FPS).

But on a practical note, since I seem to recall that we share a fondness for Jazz, I remember participating in A/B listening tests with audio equipment more expensive than a car by like minded supposedly golden eared folks when MP3 was starting to enter the mainstream.  With well encoded material some could hear differences at 128k bps, a few could on very specific rare passages at 192k (top hats come to mind), by 320k nobody (nobody) had a clue as to whether the original master or the compressed version was playing.

I can't speak for the delta thing, but since in 10+ years nobody that I am aware of has been able to show loss of visual information with 12/14-bit captured raw data 'gamma' coded to 10/12-bits in Nikon's 'lossy' NEF compression - the only type of raw file available to many Nikon cameras today including, I believe, the new and outstanding D5500 - I think we are probably talking the visual equivalent of 320kbps audio: nobody can see that the file is smaller, even after aggressive PP.  The data may be different, but the underlying visual information recorded is the same in both file formats.

I am interested in this subject, so I would be happy to be disproved if evidence is shown to the contrary.  Anyone?

Glad to know you like jazz!  I've done a lot of recordings, and my observations have to do with material that I recorded and mastered originally.  I know I can tell the difference when I move from 20+ bit lossless masters to 16 bit lossless.  The sound field flattens out spatially, and you start to lose reverb tails.  When I move to MP3 from there, even at high quality settings, the reverb tails are severely truncated.  There are some tests where knowing the material well can help to point out differences, as opposed to blind A/B testing on unfamiliar material.  Some of the task involves knowing what to attend to.

I'd say the same might be true of lossy compression in digital photography.  People who are used to extensively working over their own images in a way that depends upon the information "holding up" may notice more issues working with lossy compression.  I really don't know, since I'm used to just the D800 files and not the Sony files.  But I wonder how well the lossy files stand up to HP filtering, and other fine-grained processes used in retouching?

As you say, on general principle, Sony should provide the highest quality setting on the best 35mm image sensor in the world just as a matter of course.

[scooby70]

I suppose this matters more than I thought just not to me when I see no evidence of it in real world images. Perhaps the A7 series just isn't for everyone.