How to convince Sony to do lossless raw on A7RII and others?

[uaiomex]

I am not buying another Canon FF dslr unless they offer an articulated screen.
Trying really hard, seriously.
Thank god Sony for gas relief. 
Eduardo

I am doing this for years now, but alas with limited success:

- I am not buying a Nikon unless they change the fucus ring direction
- I am not buying a Hasselblad unless thy stop selling their spacey Sonys
- I am not buying a Phase one unless they offer a true liveview/mirrorless camera option (Alpa doesn't count)

There are so many cameras I am not buying...

I am sure they are listening!

[ErikKaffehr]

Hi Jim,

I am pretty sure that the ADC is ramp compare type and I guess that ramp time can be programmed. The number of bits thing, I don't know. It seems that Nikon gets better DR out of the "same" basic design than Sony.

Regarding using 12 bits in serial modes, I think it should be user selectable.

Best regards
Erik

As I've said in other conversations, if Sony actually does offer only uncompressed raw in addition to cRAW, that will qualify as malicious obedience in my book. I don't want uncompressed raw. What I want is losslessly compressed raw.

I"m from Missouri on the firmware-only part of what he's saying. The fact that the a7x precision drops from 13 to 12 bits under some conditions feels like hardware, not firmware. The tone curve looks like it's designed for hardware implementation. And I question whether the ADC is really 14 bits in the first place, given the fact that you can only use 13 of them at best. Who would design a 14-bit ADC if they were going to toss the LSB on the floor?

Jim

[Paulo Bizarro]

In various interviews that Sony folks have done since the release of the A7RII, they have all indicated that they have heard of the desire for lossless (or uncompressed?) raw images. Some have gone as far as to say that they'll listen to what customers want before doing anything.

So that then raises the question, how do we (the customer) make ourselves heard to them (Sony management) on this topic?

There are several ways:
1. Send individual/group petition/feedback to Sony Imaging division, or Sony CEO.
2. Do as above, but with clear examples and image samples of image quality impact.
3. Don't buy the cameras.

Personally, I am glad this issue does not affect my photography whatsoever, so to me it is "much ado about nothing". I guess not many people are seeing any impact, as the cameras are selling rather well?

[pegelli]

I would like Sony to apply a loss-less compression scheme (uncompressed if that's the only way to do it, but preferably not).

Not so much because my pictures would show the effect (I don't do star trails or other ultra high contrast scenes) but only because the majority of complaints about this will then go away.

There are a few people for which the effect is real, but it's my guess that 99% of the complaints come from people who are looking to complain just "for the heck of it"

[Otto Phocus]

The best way to let any company know your feelings as a customer is to write them a letter.  Not an E-mail.  But a physical letter.  Why?  E-mails are easy.  But if a customer takes the time to write a traditional letter and mail it, it lets the company know that this customer is serious enough about the issue to go through the effort of writing a letter.

An E-mail easily goes into a queue.  A physical letter actually sits on someone's desk.  Sure, a letter can be thrown away, but in that case it is far easier to delete an E-mail so probably nothing will work.

Just make sure that the letter is written in a professional manner with clear statements on the what and why of your request.  Keep the emotions out of it and, it should not need to be said, don't include any threats!

 

[chez]

I've made hundreds of large prints from my A7R and have never noticed any issues in my prints. This compression issue has really been jacked up by the typical Internet hype.

[Otto Phocus]

I've made hundreds of large prints from my A7R and have never noticed any issues in my prints. This compression issue has really been jacked up by the typical Internet hype.

That's a real good and cogent issue.  Just because a compression is lossy, does not mean it is losing anything important.

Has anyone found any examples of how Sony's "lossy" compression prevented them from doing something to the image that they could have done if the data were losslessly compressed?

As some pointed eared guy once said on TV: a difference that makes no difference is no difference.

[pegelli]

http://diglloyd.com/blog/2014/20140214_1-SonyA7-artifacts-star-trails.html Here's a link to an article which clearly shows the artifacts in a real life example. It is extreme and probably the vast majority of the people using Sony camera's will never encounter such a scenario but you can't say it's non-existent or will never be visible. So for people involved with such high contrast edges in their photography I really hope Sony will fix it with a firmware update, they need and use it. And as a side effect the "moaners" who just like to complain without ever experiencing the effect will also have to shut up  

[shadowblade]

That's a real good and cogent issue.  Just because a compression is lossy, does not mean it is losing anything important.

Has anyone found any examples of how Sony's "lossy" compression prevented them from doing something to the image that they could have done if the data were losslessly compressed?

As some pointed eared guy once said on TV: a difference that makes no difference is no difference.

Occasional posterisation when dealing with even skies, when subject to heavy curves. I suspect this is because the brightest portions of the compressed RAW files do not have as many levels as a 14-bit file should (i.e. 8k levels for the brightest stop).

[shadowblade]

The Sony guy must mean loss-less RAW. (But if not…are these people high?!) Why would anyone care about uncompressed loss-y RAW when the compressed variety saves space while throwing away no additional data? I also know of no requests for such, as opposed to many & sustained pleas for loss-less RAW.

-Dave-

Uncompressed files are faster to work with and faster to save and open, since there's no need to compress or decompress the file. Slightly larger file size, but storage space is cheap and high-resolution files already push performance envelopes.

Potentially, this could also translate to higher frame rates or larger buffers in camera, in addition to faster editing.

[shadowblade]

I think the fact that the a7x cameras change from 13 to 12 bit precision when you switch the shutter to continuous, silent, or bulb is much more worrisome, and I don't see that getting much ink at all.

Jim

I wasn't aware that was the case.

Which particular shooting modes are affected? Does it also include the bracketed exposures? If so, I might have to avoid bracketing exposures and just adjust shutter speed manually between shots...

[Jim Kasson]

I wasn't aware that was the case.

Which particular shooting modes are affected? Does it also include the bracketed exposures? If so, I might have to avoid bracketing exposures and just adjust shutter speed manually between shots...

It does include the continuous bracketed exposures. Automatic bracketing is implemented on the Sony alpha 7 cameras as a shutter, or drive,  mode,.  There are two modes, continuous and single shot. If you use continuous mode, the raw (pre-compression) bit depth is changed from 13 bits to 12 bits, just like when the camera operates in continuous shutter mode without auto-bracketing.

If you’re doing handheld HDR, you want continuous auto-bracketing. If you use single shot auto-bracketing, there will be greater camera motion between shots, more work for the auto-registration feature of your HDR program to do, and less chance that it will do the job precisely.

But with the a7 cameras inflicting the bit-depth tax when you use continuous auto-bracketing, you’re going to lose about a stop of DR just for invoking that mode. So, if you select as a bracketing sequence 0, +0.3, -0.3 stops, you’re actually going to be worse off than if you’d not used auto-bracketing in the first place.

To my way of thinking, the minimum continuous auto-bracketing span that makes any sense is 2 stops, and even it is less than a one-stop improvement over no auto-bracketing at all. So use 0, -1, +1 or more when you use continuous auto-bracketing.

Here's a link to a post with the above information plus links to tests that prove my assertions.

http://blog.kasson.com/?p=8819

Jim

[shadowblade]

It does include the continuous bracketed exposures. Automatic bracketing is implemented on the Sony alpha 7 cameras as a shutter, or drive,  mode,.  There are two modes, continuous and single shot. If you use continuous mode, the raw (pre-compression) bit depth is changed from 13 bits to 12 bits, just like when the camera operates in continuous shutter mode without auto-bracketing.

If you’re doing handheld HDR, you want continuous auto-bracketing. If you use single shot auto-bracketing, there will be greater camera motion between shots, more work for the auto-registration feature of your HDR program to do, and less chance that it will do the job precisely.

But with the a7 cameras inflicting the bit-depth tax when you use continuous auto-bracketing, you’re going to lose about a stop of DR just for invoking that mode. So, if you select as a bracketing sequence 0, +0.3, -0.3 stops, you’re actually going to be worse off than if you’d not used auto-bracketing in the first place.

To my way of thinking, the minimum continuous auto-bracketing span that makes any sense is 2 stops, and even it is less than a one-stop improvement over no auto-bracketing at all. So use 0, -1, +1 or more when you use continuous auto-bracketing.

Here's a link to a post with the above information plus links to tests that prove my assertions.

http://blog.kasson.com/?p=8819

Jim

That's good to know. And I hope they fix that, even if it comes at the expense of shooting speed or buffer length. After all, it's an easy firmware adjustment, and Sony's success with the A7 series is significantly dependent on refugees from Canon seeking better image quality.

Why they don't use a 14-bit RAW bit-depth escapes me.

Hopefully this is all included in the A9... along with a 59MP sensor.

[Jack Hogan]

Occasional posterisation when dealing with even skies, when subject to heavy curves. I suspect this is because the brightest portions of the compressed RAW files do not have as many levels as a 14-bit file should (i.e. 8k levels for the brightest stop).

Excellent, let's see  the relevant raw file.

[eronald]

Uncompressed files are faster to work with and faster to save and open, since there's no need to compress or decompress the file. Slightly larger file size, but storage space is cheap and high-resolution files already push performance envelopes.

Potentially, this could also translate to higher frame rates or larger buffers in camera, in addition to faster editing.

I think they cannot get the necessary throughput without the compression, and in particular current consumption and thermal effects might go up.

Edmund

[ErikKaffehr]

Hi,

Jack discusses this here: http://www.strollswithmydog.com/how-many-bits-to-fully-encode-my-image/

My understanding is that Jack means that shot noise masks any possible banding in the sky. Shot noise is natural distribution of photons.

Let's assume 8000 levels (1 stop below saturation)

Shot noise will be 89 levels, meaning that 65% of the pixels will vary within 7911 and 8089. The remaining 35% of the pixels will have even wider variation. At 8000 levels the Sony ARW uses 1 digital number for 16 levels, so the 65% range from 7911 - 8089 will be coded with eleven number. That will give a good representation, and I don't think banding is possible. Sony is far from alone using this type of compression, I guess both Leica and Phase One uses it.

The probable cause of banding that can be observed is more likely colour management issues.

The artefacts below are caused by Colour Profile issues. They go away with original Adobe Standard DCP profile:

Just to say that the above image is coming from a P45+ back used with my usual DNG Colour Profile generated by DNG Profile Editor. Capture One 7.3 has similar issues with that image.

Now, Sony has an additional compression that I don't like, the Delta coding, but the Delta coding is lossless in area without very large gradients. The artifacts caused by the Delta compression look like this:


Best regards
Erik

Excellent, let's see  the relevant raw file.

[shadowblade]

I think they cannot get the necessary throughput without the compression, and in particular current consumption and thermal effects might go up.

Edmund

It shouldn't.

A compressed file requires the camera to do a lot more than an uncompressed file - it needs to compress it first before saving it, which requires a fair bit of processor power (just see how long it takes to open or save large files in Photoshop, even with a full-powered processor).

[ErikKaffehr]

Hi,

Adobe Photoshop uses a ZIP type compression that is much slower, also the camera ASICs probably have specific hardware to speed conversion. Consider that Alpha 99 SLT can convert 6 frames a second to JPG and write to a fast memory card unitil card is filled or battery exhausted. A fast PC would never come any near!

Best regards
Erik

It shouldn't.

A compressed file requires the camera to do a lot more than an uncompressed file - it needs to compress it first before saving it, which requires a fair bit of processor power (just see how long it takes to open or save large files in Photoshop, even with a full-powered processor).

[LKaven]

Now, Sony has an additional compression that I don't like, the Delta coding, but the Delta coding is lossless in area without very large gradients.

Wouldn't the division from 14 bits to 11 bits and subsequent re-inflation to 14 bits will result in missing codes, even without consideration of the delta coding?

[LKaven]

Adobe Photoshop uses a ZIP type compression that is much slower, also the camera ASICs probably have specific hardware to speed conversion. Consider that Alpha 99 SLT can convert 6 frames a second to JPG and write to a fast memory card unitil card is filled or battery exhausted. A fast PC would never come any near!

This seems like a very fast compression, done on the fly.  I think decompressing/compressing would be less expensive in time than the extra storage access time and bandwidth by far.

This compression might even be done on the sensor as a way to speed up frame reads.