Luminous Landscape Forum

Equipment & Techniques => Cameras, Lenses and Shooting gear => Topic started by: dreed on June 20, 2015, 03:48:05 am

Title: How to convince Sony to do lossless raw on A7RII and others?
Post by: dreed on June 20, 2015, 03:48:05 am
In various interviews that Sony folks have done since the release of the A7RII, they have all indicated that they have heard of the desire for lossless (or uncompressed?) raw images. Some have gone as far as to say that they'll listen to what customers want before doing anything.

So that then raises the question, how do we (the customer) make ourselves heard to them (Sony management) on this topic?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on June 20, 2015, 05:38:53 am
At least by not buying the camera until it does support lossless compressed raw?

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Herbc on June 20, 2015, 11:42:40 am
+1
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 20, 2015, 02:17:44 pm
Other than the delta business, has anybody shown lost image information from Sony's 11 bit 'gamma' compression?  The more I think about this type of compression (also used in one of Nikon's compressed NEF modes) the more I am becoming convinced that if done properly it just drops storage bits - but not one bit (:-) of IQ.  Aggressive PP included.  Any examples of lost information, anybody?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 20, 2015, 03:19:52 pm
+1

Erik


Other than the delta business, has anybody shown lost image information from Sony's 11 bit 'gamma' compression?  The more I think about this type of compression (also used in one of Nikon's compressed NEF modes) the more I am becoming convinced that if done properly it just drops storage bits - but not one bit (:-) of IQ.  Aggressive PP included.  Any examples of lost information, anybody?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: MatthewCromer on June 20, 2015, 03:29:56 pm
Someone was complaining about using Sony to photograph clouds. Claimed the highlight compression reduced tonalities in the whites, or something.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Telecaster on June 20, 2015, 03:57:01 pm
Someone was complaining about using Sony to photograph clouds. Claimed the highlight compression reduced tonalities in the whites, or something.

Bah…I've done some cloud pic-taking with my A7r (lotsa opportunities in the new reality of five-month spring, no summer, here in Michigan) and have bashed the RAWs about pretty hard. Nada in terms of clear tonal compression/posterizing issues. That is, issues beyond what any normal 12- or 14-bit file will exhibit if you go too far. For that matter I don't think I've seen any real-world photos that clearly show such issues…just RawDigger plots.

I'd like to see Sony offer a loss-less RAW option. But I won't lose any sleep over it.

-Dave-
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 20, 2015, 10:25:26 pm
Just don't buy the camera until the firmware does justice to the sensor.

Maybe the other camera manufacturers are bribing the decision makers of the Sony division to screw-up the image using the lossy compression. Makes Nikon and Canon look better for still photographs. For video- will not matter.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 20, 2015, 11:56:21 pm
In various interviews that Sony folks have done since the release of the A7RII, they have all indicated that they have heard of the desire for lossless (or uncompressed?) raw images. Some have gone as far as to say that they'll listen to what customers want before doing anything.

So that then raises the question, how do we (the customer) make ourselves heard to them (Sony management) on this topic?

I've gone to the trouble of writing software to simulate the round trip through the Sony compansion process, and passed both simulated and real images through the sim. I can see errors by subtracting the original image from the companded one, but they are quite small. Under some extreme circumstances, I can see errors by looking at companded images alone. However, my conclusion is that the tone curve/delta mod algorithm will hardly ever cause visible problems in real images, and it's not something I worry much about.

Would I like to see lossless compression offered in addition or instead? Heck, yes.

If they don't change their compression, will I buy more Sony cameras with their lossy algorithm in them? Also, heck, yes.

I think the fact that the a7x cameras change from 13 to 12 bit precision when you switch the shutter to continuous, silent, or bulb is much more worrisome, and I don't see that getting much ink at all.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on June 21, 2015, 12:05:54 am
Other than the delta business, has anybody shown lost image information from Sony's 11 bit 'gamma' compression?  The more I think about this type of compression (also used in one of Nikon's compressed NEF modes) the more I am becoming convinced that if done properly it just drops storage bits - but not one bit (:-) of IQ.  Aggressive PP included.  Any examples of lost information, anybody?

(with apologies to Franz Kafka:)

Sony: "We just made the finest 35mm image sensor in the world today"
Me: "Great!"
Sony: "Ok!  You can have images that look just like images from the finest 35mm image sensor in the world today"
Me: "I thought you were going to give me the finest 35mm image sensor in the world?"
Sony: "We totally are...well, we're giving you the next best thing, and you totally can't tell the difference!"
Me: "But...but..."
Sony: "Well, we can't exactly let you use the highest quality setting on the finest 35mm image sensor in the world..."
Me: "But...but..."
Sony: "then you might not think that there was something left that you hadn't tried"
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 21, 2015, 10:45:06 am
Hi Jim,

I would essentially agree. Some oddities observed may come from colour profiles, like this one:
(http://echophoto.dnsalias.net/ekr/Articles/MFDLook/20150407-CF046278.jpg)

This banding is visible with most DNG-profiles I have generated, but not with Adobe Standard profile in LR. It is also visible in Capture One 7 with their default profile. DCP profiles generated with Anders Torger's DcamProf don't show this issue.

I guess lossy compression gets a lot of blame for this kind of artefacts, but the sample is taken from an "uncompressed" IIQ file on a P45+ back, the ones with 16 bits of fidelity…

Best regards
Erik



I've gone to the trouble of writing software to simulate the round trip through the Sony compansion process, and passed both simulated and real images through the sim. I can see errors by subtracting the original image from the companded one, but they are quite small. Under some extreme circumstances, I can see errors by looking at companded images alone. However, my conclusion is that the tone curve/delta mod algorithm will hardly ever cause visible problems in real images, and it's not something I worry much about.

Would I like to see lossless compression offered in addition or instead? Heck, yes.

If they don't change their compression, will I buy more Sony cameras with their lossy algorithm in them? Also, heck, yes.

I think the fact that the a7x cameras change from 13 to 12 bit precision when you switch the shutter to continuous, silent, or bulb is much more worrisome, and I don't see that getting much ink at all.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 21, 2015, 12:25:43 pm
(with apologies to Franz Kafka:)

Sony: "We just made the finest 35mm image sensor in the world today"
Me: "Great!"
Sony: "Ok!  You can have images that look just like images from the finest 35mm image sensor in the world today"
Me: "I thought you were going to give me the finest 35mm image sensor in the world?"
Sony: "We totally are...well, we're giving you the next best thing, and you totally can't tell the difference!"
Me: "But...but..."
Sony: "Well, we can't exactly let you use the highest quality setting on the finest 35mm image sensor in the world..."
Me: "But...but..."
Sony: "then you might not think that there was something left that you hadn't tried"

I hear you Luke, and I think it makes good marketing sense for Sony to put all fears to bed by providing a lossless compressed mode as well (expect some possible compromises if they do, though, e.g. slower FPS).

But on a practical note, since I seem to recall that we share a fondness for Jazz, I remember participating in A/B listening tests with audio equipment more expensive than a car by like minded supposedly golden eared folks when MP3 was starting to enter the mainstream.  With well encoded material some could hear differences at 128k bps, a few could on very specific rare passages at 192k (top hats come to mind), by 320k nobody (nobody) had a clue as to whether the original master or the compressed version was playing.

I can't speak for the delta thing, but since in 10+ years nobody that I am aware of has been able to show loss of visual information with 12/14-bit captured raw data 'gamma' coded to 10/12-bits in Nikon's 'lossy' NEF compression - the only type of raw file available to many Nikon cameras today including, I believe, the new and outstanding D5500 - I think we are probably talking the visual equivalent of 320kbps audio: nobody can see that the file is smaller, even after aggressive PP.  The data may be different, but the underlying visual information recorded is the same in both file formats.

I am interested in this subject, so I would be happy to be disproved if evidence is shown to the contrary.  Anyone?

Jack

PS BTW, the parallel with audio breaks down because there they actually do throw away recorded audio information on the basis that humans cannot hear it.  The digital photography case for raw data 'gamma' compression is much stronger because here they are simply not using up bits for stuff that the camera cannot record.  Take a look at the market balance example here (http://www.strollswithmydog.com/how-many-bits-to-fully-encode-my-image/) for an idea of what's happening.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: uaiomex on June 21, 2015, 12:26:06 pm
+1

At least by not buying the camera until it does support lossless compressed raw?

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: geesbert on June 21, 2015, 04:12:19 pm
I am doing this for years now, but alas with limited success:

- I am not buying a Nikon unless they change the fucus ring direction
- I am not buying a Hasselblad unless thy stop selling their spacey Sonys
- I am not buying a Phase one unless they offer a true liveview/mirrorless camera option (Alpa doesn't count)

There are so many cameras I am not buying...

I am sure they are listening!

At least by not buying the camera until it does support lossless compressed raw?

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: scooby70 on June 21, 2015, 04:24:27 pm
I don't know if you guys spotted this but in a recent interview with Sony engineer and manager Kimio Maki...

"5) Uncompressed RAW: Sony RAW is compressed, not uncompressed. But if we’re getting a lot of requests for it, we should make such a kind of no-compression raw. We recognize the customer’s requirement, and actually we are working on it. And yes Sony could provide that via simple firmware upgrade!"

See here...

http://www.sonyalpharumors.com/sony-interview-and-a-good-news-they-are-working-on-uncompressed-raw/#disqus_thread

It's never been an issue for me but if they do release a firmware update I'm download it and it'll hopefully banish even the thought that compression could be an issue.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 21, 2015, 04:35:12 pm
I don't know if you guys spotted this but in a recent interview with Sony engineer and manager Kimio Maki...

"5) Uncompressed RAW: Sony RAW is compressed, not uncompressed. But if we’re getting a lot of requests for it, we should make such a kind of no-compression raw. We recognize the customer’s requirement, and actually we are working on it. And yes Sony could provide that via simple firmware upgrade!"

See here...

http://www.sonyalpharumors.com/sony-interview-and-a-good-news-they-are-working-on-uncompressed-raw/#disqus_thread

It's never been an issue for me but if they do release a firmware update I'm download it and it'll hopefully banish even the thought that compression could be an issue.

As I've said in other conversations, if Sony actually does offer only uncompressed raw in addition to cRAW, that will qualify as malicious obedience in my book. I don't want uncompressed raw. What I want is losslessly compressed raw.

I"m from Missouri on the firmware-only part of what he's saying. The fact that the a7x precision drops from 13 to 12 bits under some conditions feels like hardware, not firmware. The tone curve looks like it's designed for hardware implementation. And I question whether the ADC is really 14 bits in the first place, given the fact that you can only use 13 of them at best. Who would design a 14-bit ADC if they were going to toss the LSB on the floor?

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Telecaster on June 21, 2015, 04:51:22 pm
As I've said in other conversations, if Sony actually does offer only uncompressed raw in addition to cRAW, that will qualify as malicious obedience in my book. I don't want uncompressed raw. What I want is losslessly compressed raw.

The Sony guy must mean loss-less RAW. (But if not…are these people high?!) Why would anyone care about uncompressed loss-y RAW when the compressed variety saves space while throwing away no additional data? I also know of no requests for such, as opposed to many & sustained pleas for loss-less RAW.

-Dave-
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on June 21, 2015, 06:22:10 pm
I am doing this for years now, but alas with limited success:

- I am not buying a Nikon unless they change the fucus ring direction
- I am not buying a Hasselblad unless thy stop selling their spacey Sonys
- I am not buying a Phase one unless they offer a true liveview/mirrorless camera option (Alpa doesn't count)

Keep trying! ;)

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on June 21, 2015, 06:46:33 pm
I hear you Luke, and I think it makes good marketing sense for Sony to put all fears to bed by providing a lossless compressed mode as well (expect some possible compromises if they do, though, e.g. slower FPS).

But on a practical note, since I seem to recall that we share a fondness for Jazz, I remember participating in A/B listening tests with audio equipment more expensive than a car by like minded supposedly golden eared folks when MP3 was starting to enter the mainstream.  With well encoded material some could hear differences at 128k bps, a few could on very specific rare passages at 192k (top hats come to mind), by 320k nobody (nobody) had a clue as to whether the original master or the compressed version was playing.

I can't speak for the delta thing, but since in 10+ years nobody that I am aware of has been able to show loss of visual information with 12/14-bit captured raw data 'gamma' coded to 10/12-bits in Nikon's 'lossy' NEF compression - the only type of raw file available to many Nikon cameras today including, I believe, the new and outstanding D5500 - I think we are probably talking the visual equivalent of 320kbps audio: nobody can see that the file is smaller, even after aggressive PP.  The data may be different, but the underlying visual information recorded is the same in both file formats.

I am interested in this subject, so I would be happy to be disproved if evidence is shown to the contrary.  Anyone?

Glad to know you like jazz!  I've done a lot of recordings, and my observations have to do with material that I recorded and mastered originally.  I know I can tell the difference when I move from 20+ bit lossless masters to 16 bit lossless.  The sound field flattens out spatially, and you start to lose reverb tails.  When I move to MP3 from there, even at high quality settings, the reverb tails are severely truncated.  There are some tests where knowing the material well can help to point out differences, as opposed to blind A/B testing on unfamiliar material.  Some of the task involves knowing what to attend to.

I'd say the same might be true of lossy compression in digital photography.  People who are used to extensively working over their own images in a way that depends upon the information "holding up" may notice more issues working with lossy compression.  I really don't know, since I'm used to just the D800 files and not the Sony files.  But I wonder how well the lossy files stand up to HP filtering, and other fine-grained processes used in retouching?

As you say, on general principle, Sony should provide the highest quality setting on the best 35mm image sensor in the world just as a matter of course.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: scooby70 on June 21, 2015, 07:23:13 pm
I suppose this matters more than I thought just not to me when I see no evidence of it in real world images. Perhaps the A7 series just isn't for everyone.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: uaiomex on June 22, 2015, 12:52:23 am
I am not buying another Canon FF dslr unless they offer an articulated screen.
Trying really hard, seriously.
Thank god Sony for gas relief.  :D
Eduardo


I am doing this for years now, but alas with limited success:

- I am not buying a Nikon unless they change the fucus ring direction
- I am not buying a Hasselblad unless thy stop selling their spacey Sonys
- I am not buying a Phase one unless they offer a true liveview/mirrorless camera option (Alpa doesn't count)

There are so many cameras I am not buying...

I am sure they are listening!

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 02:21:04 am
Hi Jim,

I am pretty sure that the ADC is ramp compare type and I guess that ramp time can be programmed. The number of bits thing, I don't know. It seems that Nikon gets better DR out of the "same" basic design than Sony.

Regarding using 12 bits in serial modes, I think it should be user selectable.

Best regards
Erik

As I've said in other conversations, if Sony actually does offer only uncompressed raw in addition to cRAW, that will qualify as malicious obedience in my book. I don't want uncompressed raw. What I want is losslessly compressed raw.

I"m from Missouri on the firmware-only part of what he's saying. The fact that the a7x precision drops from 13 to 12 bits under some conditions feels like hardware, not firmware. The tone curve looks like it's designed for hardware implementation. And I question whether the ADC is really 14 bits in the first place, given the fact that you can only use 13 of them at best. Who would design a 14-bit ADC if they were going to toss the LSB on the floor?

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Paulo Bizarro on June 22, 2015, 04:49:25 am
In various interviews that Sony folks have done since the release of the A7RII, they have all indicated that they have heard of the desire for lossless (or uncompressed?) raw images. Some have gone as far as to say that they'll listen to what customers want before doing anything.

So that then raises the question, how do we (the customer) make ourselves heard to them (Sony management) on this topic?

There are several ways:
1. Send individual/group petition/feedback to Sony Imaging division, or Sony CEO.
2. Do as above, but with clear examples and image samples of image quality impact.
3. Don't buy the cameras.

Personally, I am glad this issue does not affect my photography whatsoever, so to me it is "much ado about nothing". I guess not many people are seeing any impact, as the cameras are selling rather well?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: pegelli on June 22, 2015, 06:11:07 am
I would like Sony to apply a loss-less compression scheme (uncompressed if that's the only way to do it, but preferably not).

Not so much because my pictures would show the effect (I don't do star trails or other ultra high contrast scenes) but only because the majority of complaints about this will then go away.

There are a few people for which the effect is real, but it's my guess that 99% of the complaints come from people who are looking to complain just "for the heck of it"
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Otto Phocus on June 22, 2015, 07:56:46 am
The best way to let any company know your feelings as a customer is to write them a letter.  Not an E-mail.  But a physical letter.  Why?  E-mails are easy.  But if a customer takes the time to write a traditional letter and mail it, it lets the company know that this customer is serious enough about the issue to go through the effort of writing a letter.

An E-mail easily goes into a queue.  A physical letter actually sits on someone's desk.  Sure, a letter can be thrown away, but in that case it is far easier to delete an E-mail so probably nothing will work.

Just make sure that the letter is written in a professional manner with clear statements on the what and why of your request.  Keep the emotions out of it and, it should not need to be said, don't include any threats!

 
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: chez on June 22, 2015, 07:59:04 am
I've made hundreds of large prints from my A7R and have never noticed any issues in my prints. This compression issue has really been jacked up by the typical Internet hype.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Otto Phocus on June 22, 2015, 08:09:12 am
I've made hundreds of large prints from my A7R and have never noticed any issues in my prints. This compression issue has really been jacked up by the typical Internet hype.

That's a real good and cogent issue.  Just because a compression is lossy, does not mean it is losing anything important.

Has anyone found any examples of how Sony's "lossy" compression prevented them from doing something to the image that they could have done if the data were losslessly compressed?

As some pointed eared guy once said on TV: a difference that makes no difference is no difference.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: pegelli on June 22, 2015, 08:39:24 am
http://diglloyd.com/blog/2014/20140214_1-SonyA7-artifacts-star-trails.html (http://diglloyd.com/blog/2014/20140214_1-SonyA7-artifacts-star-trails.html) Here's a link to an article which clearly shows the artifacts in a real life example. It is extreme and probably the vast majority of the people using Sony camera's will never encounter such a scenario but you can't say it's non-existent or will never be visible. So for people involved with such high contrast edges in their photography I really hope Sony will fix it with a firmware update, they need and use it. And as a side effect the "moaners" who just like to complain without ever experiencing the effect will also have to shut up  :)
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 11:34:58 am
That's a real good and cogent issue.  Just because a compression is lossy, does not mean it is losing anything important.

Has anyone found any examples of how Sony's "lossy" compression prevented them from doing something to the image that they could have done if the data were losslessly compressed?

As some pointed eared guy once said on TV: a difference that makes no difference is no difference.

Occasional posterisation when dealing with even skies, when subject to heavy curves. I suspect this is because the brightest portions of the compressed RAW files do not have as many levels as a 14-bit file should (i.e. 8k levels for the brightest stop).
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 11:37:36 am
The Sony guy must mean loss-less RAW. (But if not…are these people high?!) Why would anyone care about uncompressed loss-y RAW when the compressed variety saves space while throwing away no additional data? I also know of no requests for such, as opposed to many & sustained pleas for loss-less RAW.

-Dave-

Uncompressed files are faster to work with and faster to save and open, since there's no need to compress or decompress the file. Slightly larger file size, but storage space is cheap and high-resolution files already push performance envelopes.

Potentially, this could also translate to higher frame rates or larger buffers in camera, in addition to faster editing.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 11:56:49 am
I think the fact that the a7x cameras change from 13 to 12 bit precision when you switch the shutter to continuous, silent, or bulb is much more worrisome, and I don't see that getting much ink at all.

Jim

I wasn't aware that was the case.

Which particular shooting modes are affected? Does it also include the bracketed exposures? If so, I might have to avoid bracketing exposures and just adjust shutter speed manually between shots...
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 22, 2015, 12:37:01 pm
I wasn't aware that was the case.

Which particular shooting modes are affected? Does it also include the bracketed exposures? If so, I might have to avoid bracketing exposures and just adjust shutter speed manually between shots...

It does include the continuous bracketed exposures. Automatic bracketing is implemented on the Sony alpha 7 cameras as a shutter, or drive,  mode,.  There are two modes, continuous and single shot. If you use continuous mode, the raw (pre-compression) bit depth is changed from 13 bits to 12 bits, just like when the camera operates in continuous shutter mode without auto-bracketing.

If you’re doing handheld HDR, you want continuous auto-bracketing. If you use single shot auto-bracketing, there will be greater camera motion between shots, more work for the auto-registration feature of your HDR program to do, and less chance that it will do the job precisely.

But with the a7 cameras inflicting the bit-depth tax when you use continuous auto-bracketing, you’re going to lose about a stop of DR just for invoking that mode. So, if you select as a bracketing sequence 0, +0.3, -0.3 stops, you’re actually going to be worse off than if you’d not used auto-bracketing in the first place.

To my way of thinking, the minimum continuous auto-bracketing span that makes any sense is 2 stops, and even it is less than a one-stop improvement over no auto-bracketing at all. So use 0, -1, +1 or more when you use continuous auto-bracketing.

Here's a link to a post with the above information plus links to tests that prove my assertions.

http://blog.kasson.com/?p=8819

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 12:41:50 pm
It does include the continuous bracketed exposures. Automatic bracketing is implemented on the Sony alpha 7 cameras as a shutter, or drive,  mode,.  There are two modes, continuous and single shot. If you use continuous mode, the raw (pre-compression) bit depth is changed from 13 bits to 12 bits, just like when the camera operates in continuous shutter mode without auto-bracketing.

If you’re doing handheld HDR, you want continuous auto-bracketing. If you use single shot auto-bracketing, there will be greater camera motion between shots, more work for the auto-registration feature of your HDR program to do, and less chance that it will do the job precisely.

But with the a7 cameras inflicting the bit-depth tax when you use continuous auto-bracketing, you’re going to lose about a stop of DR just for invoking that mode. So, if you select as a bracketing sequence 0, +0.3, -0.3 stops, you’re actually going to be worse off than if you’d not used auto-bracketing in the first place.

To my way of thinking, the minimum continuous auto-bracketing span that makes any sense is 2 stops, and even it is less than a one-stop improvement over no auto-bracketing at all. So use 0, -1, +1 or more when you use continuous auto-bracketing.

Here's a link to a post with the above information plus links to tests that prove my assertions.

http://blog.kasson.com/?p=8819

Jim

That's good to know. And I hope they fix that, even if it comes at the expense of shooting speed or buffer length. After all, it's an easy firmware adjustment, and Sony's success with the A7 series is significantly dependent on refugees from Canon seeking better image quality.

Why they don't use a 14-bit RAW bit-depth escapes me.

Hopefully this is all included in the A9... along with a 59MP sensor.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 22, 2015, 02:53:08 pm
Occasional posterisation when dealing with even skies, when subject to heavy curves. I suspect this is because the brightest portions of the compressed RAW files do not have as many levels as a 14-bit file should (i.e. 8k levels for the brightest stop).

Excellent, let's see  the relevant raw file.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on June 22, 2015, 03:00:32 pm
Uncompressed files are faster to work with and faster to save and open, since there's no need to compress or decompress the file. Slightly larger file size, but storage space is cheap and high-resolution files already push performance envelopes.

Potentially, this could also translate to higher frame rates or larger buffers in camera, in addition to faster editing.

I think they cannot get the necessary throughput without the compression, and in particular current consumption and thermal effects might go up.

Edmund
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 03:34:14 pm
Hi,

Jack discusses this here: http://www.strollswithmydog.com/how-many-bits-to-fully-encode-my-image/

My understanding is that Jack means that shot noise masks any possible banding in the sky. Shot noise is natural distribution of photons.

Let's assume 8000 levels (1 stop below saturation)

Shot noise will be 89 levels, meaning that 65% of the pixels will vary within 7911 and 8089. The remaining 35% of the pixels will have even wider variation. At 8000 levels the Sony ARW uses 1 digital number for 16 levels, so the 65% range from 7911 - 8089 will be coded with eleven number. That will give a good representation, and I don't think banding is possible. Sony is far from alone using this type of compression, I guess both Leica and Phase One uses it.

The probable cause of banding that can be observed is more likely colour management issues.

The artefacts below are caused by Colour Profile issues. They go away with original Adobe Standard DCP profile:
(http://echophoto.dnsalias.net/ekr/Articles/MFDLook/20150407-CF046278.jpg)
Just to say that the above image is coming from a P45+ back used with my usual DNG Colour Profile generated by DNG Profile Editor. Capture One 7.3 has similar issues with that image.

Now, Sony has an additional compression that I don't like, the Delta coding, but the Delta coding is lossless in area without very large gradients. The artifacts caused by the Delta compression look like this:
(http://www.rawdigger.com/sites/www.rawdigger.com/files/Posterization/image01.png)

Best regards
Erik

Excellent, let's see  the relevant raw file.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 03:35:11 pm
I think they cannot get the necessary throughput without the compression, and in particular current consumption and thermal effects might go up.

Edmund

It shouldn't.

A compressed file requires the camera to do a lot more than an uncompressed file - it needs to compress it first before saving it, which requires a fair bit of processor power (just see how long it takes to open or save large files in Photoshop, even with a full-powered processor).
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 03:43:38 pm
Hi,

Adobe Photoshop uses a ZIP type compression that is much slower, also the camera ASICs probably have specific hardware to speed conversion. Consider that Alpha 99 SLT can convert 6 frames a second to JPG and write to a fast memory card unitil card is filled or battery exhausted. A fast PC would never come any near!

Best regards
Erik

It shouldn't.

A compressed file requires the camera to do a lot more than an uncompressed file - it needs to compress it first before saving it, which requires a fair bit of processor power (just see how long it takes to open or save large files in Photoshop, even with a full-powered processor).
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on June 22, 2015, 03:45:25 pm
Now, Sony has an additional compression that I don't like, the Delta coding, but the Delta coding is lossless in area without very large gradients.

Wouldn't the division from 14 bits to 11 bits and subsequent re-inflation to 14 bits will result in missing codes, even without consideration of the delta coding?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on June 22, 2015, 03:49:22 pm
Adobe Photoshop uses a ZIP type compression that is much slower, also the camera ASICs probably have specific hardware to speed conversion. Consider that Alpha 99 SLT can convert 6 frames a second to JPG and write to a fast memory card unitil card is filled or battery exhausted. A fast PC would never come any near!

This seems like a very fast compression, done on the fly.  I think decompressing/compressing would be less expensive in time than the extra storage access time and bandwidth by far.

This compression might even be done on the sensor as a way to speed up frame reads.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 03:56:33 pm
Hi,

Jack discusses this here: http://www.strollswithmydog.com/how-many-bits-to-fully-encode-my-image/

My understanding is that Jack means that shot noise masks any possible banding in the sky. Shot noise is natural distribution of photons.

Let's assume 8000 levels (1 stop below saturation)

Shot noise will be 89 levels, meaning that 65% of the pixels will vary within 7911 and 8089. The remaining 35% of the pixels will have even wider variation. At 8000 levels the Sony ARW uses 1 digital number for 16 levels, so the 65% range from 7911 - 8089 will be coded with eleven number. That will give a good representation, and I don't think banding is possible. Sony is far from alone using this type of compression, I guess both Leica and Phase One uses it.

The probable cause of banding that can be observed is more likely colour management issues.

The artefacts below are caused by Colour Profile issues. They go away with original Adobe Standard DCP profile:
(http://echophoto.dnsalias.net/ekr/Articles/MFDLook/20150407-CF046278.jpg)
Just to say that the above image is coming from a P45+ back used with my usual DNG Colour Profile generated by DNG Profile Editor. Capture One 7.3 has similar issues with that image.

Probably explains why the posterisation looks different depending on which monitor/profile I'm viewing it on.

The shot noise issue probably also explains why posterisation seems to be more apparent when I add/average multiple frames together to reduce noise and improve DR (for an effective ISO 50, 25 or lower) - I suspect the lower shot noise from multiple added exposures brings the random noise below the 16-level limit.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 04:02:44 pm
Hi,

Photoshop is just 15 bits…

Regarding averaging exposures, we have seen a case on GetDPI where banding was caused by lens corrections. The photographer was combining several exposures and we assumed that using lens correction made the raw processor to apply dark frame subtraction in the wrong way. Complex banding was the result.

http://www.getdpi.com/forum/medium-format-systems-and-digital-backs/53149-strange-moire-like-pattern-assistance-requested.html

Best regards
Erik

Probably explains why the posterisation looks different depending on which monitor/profile I'm viewing it on.

The shot noise issue probably also explains why posterisation seems to be more apparent when I add/average multiple frames together to reduce noise and improve DR (for an effective ISO 50, 25 or lower) - I suspect the lower shot noise from multiple added exposures brings the random noise below the 16-level limit.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 22, 2015, 04:07:01 pm
Hi,

Photoshop is just 15 bits…

Regarding averaging exposures, we have seen a case on GetDPI where banding was caused by lens corrections. The photographer was combining several exposures and we assumed that using lens correction made the raw processor to apply dark frame subtraction in the wrong way. Complex banding was the result.

http://www.getdpi.com/forum/medium-format-systems-and-digital-backs/53149-strange-moire-like-pattern-assistance-requested.html

Best regards
Erik


What do you mean Photoshop is 15 bits?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 22, 2015, 04:10:17 pm


Jack discusses this here: http://www.strollswithmydog.com/how-many-bits-to-fully-encode-my-image/

My understanding is that Jack means that shot noise masks any possible banding in the sky. Shot noise is natural distribution of photons.

Let's assume 8000 levels (1 stop below saturation)

Shot noise will be 89 levels, meaning that 65% of the pixels will vary within 7911 and 8089. The remaining 35% of the pixels will have even wider variation. At 8000 levels the Sony ARW uses 1 digital number for 16 levels, so the 65% range from 7911 - 8089 will be coded with eleven number. That will give a good representation, and I don't think banding is possible. Sony is far from alone using this type of compression, I guess both Leica and Phase One uses it.



I dunno about Phase One, but neither the Leica M240 or M9 show the histogram depopulation that would result from applying an in-camera tone curve.

Nice job distinguishing between Poisson statistics and Gaussian ones, Erik. I bet you though nobody would notice.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 22, 2015, 04:13:11 pm
What do you mean Photoshop is 15 bits?

Photoshop's internel representation is [0,32768]. This gives a midpoint to the range and allows for faster math because they can use bit shifts instead of divides.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Bernard ODonovan on June 22, 2015, 05:27:14 pm
(with apologies to Franz Kafka:)

Sony: "We just made the finest 35mm image sensor in the world today"
Me: "Great!"
Sony: "Ok!  You can have images that look just like images from the finest 35mm image sensor in the world today"
Me: "I thought you were going to give me the finest 35mm image sensor in the world?"
Sony: "We totally are...well, we're giving you the next best thing, and you totally can't tell the difference!"
Me: "But...but..."
Sony: "Well, we can't exactly let you use the highest quality setting on the finest 35mm image sensor in the world..."
Me: "But...but..."
Sony: "then you might not think that there was something left that you hadn't tried"

(https://d.gr-assets.com/books/1364844956l/3015166.jpg)

 ;D
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: dreed on June 22, 2015, 05:40:04 pm
From:

http://www.dpreview.com/articles/0717419525/interview-kimio-maki-of-sony-the-customer-s-voice-is-the-most-important-data-for-me (http://www.dpreview.com/articles/0717419525/interview-kimio-maki-of-sony-the-customer-s-voice-is-the-most-important-data-for-me)

One of our main criticisms of the a7-series has been raw compression. Is the raw processing of the a7R II the same as previous cameras?

KM: Right now it is the same, yes. We’re still working on it. In the future we may change the software but that’s not completed yet. We have consumers who require 14-bit etc., and we’re considering [how to deal with it].
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 05:58:06 pm
Hi,

It would give missing bits, but the number of bits at high counts/digital numbers/collected electrons is enough to correctly describe the signal in presence of the shot noise. Low signals are densely sampled while high signals are sparsely sampled.

This posting by Emil Martinec explains it much better: http://www.openphotographyforums.com/forums/showthread.php?t=5499

Wouldn't the division from 14 bits to 11 bits and subsequent re-inflation to 14 bits will result in missing codes, even without consideration of the delta coding?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 06:02:29 pm
Hi Jim,

I corrected my posting regarding Leica using tone curve compression.

Regarding the Poisson stuff, it is something I read about back in 1978… I wouldn't mind if you choose to elaborate :-)

Best regards
Erik

I dunno about Phase One, but neither the Leica M240 or M9 show the histogram depopulation that would result from applying an in-camera tone curve.

Nice job distinguishing between Poisson statistics and Gaussian ones, Erik. I bet you though nobody would notice.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 22, 2015, 07:18:56 pm

I corrected my posting regarding Leica using tone curve compression.

Regarding the Poisson stuff, it is something I read about back in 1978… I wouldn't mind if you choose to elaborate :-)


The Poisson probability density function is a bit different from the Gaussian one until you get near zero where they can be wildly different. I usually ignore those differences, but you didn't, and quoted the correct value of 65% of the samples lying between +/- one sigma, where with the Gaussian distribution, it would be 68%.

I think that's right, anyway.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: labirdman on June 22, 2015, 09:42:51 pm
For those asking for a real world example of posterization caused by Sony's lossy RAW, dpreview made available an A7R raw file here (http://movies.dpreview.com.s3.amazonaws.com/sony_a7s/DSC00172.ARW). Raise exposure a little, and look at the arches and the outline of the needle itself. Apparently this wouldn't be an issue in a D810 file.

If you'd prefer to see 100% jpg crops showing same, check the penultimate section of their preview here (http://www.dpreview.com/articles/7945517371/opinion-did-sony-just-do-the-impossible/2).

I think Jim is right, though--the bigger problem is the drop from 13 to 12-bit precision when shooting in continuous, silent, and bulb modes. Nobody seems to know about that, and there's been very little discussion.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 22, 2015, 10:41:26 pm
Hi,

What we see in that sample may be an effect of the Delta compression. It is a bit similar to the star tracks image, but the signature is not that clear. The good thing is that it is my guess that Sony may be able to disable that easily. Older Sony's used to haw RAW and cRAW, I guess that cRAW may mean Delta compression.

Best regards
Erik

For those asking for a real world example of posterization caused by Sony's lossy RAW, dpreview made available an A7R raw file here (http://movies.dpreview.com.s3.amazonaws.com/sony_a7s/DSC00172.ARW). Raise exposure a little, and look at the arches and the outline of the needle itself. Apparently this wouldn't be an issue in a D810 file.

If you'd prefer to see 100% jpg crops showing same, check the penultimate section of their preview here (http://www.dpreview.com/articles/7945517371/opinion-did-sony-just-do-the-impossible/2).

I think Jim is right, though--the bigger problem is the drop from 13 to 12-bit precision when shooting in continuous, silent, and bulb modes. Nobody seems to know about that, and there's been very little discussion.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 23, 2015, 08:23:57 am
Photoshop's internel representation is [0,32768]. This gives a midpoint to the range and allows for faster math because they can use bit shifts instead of divides.

Jim

In the context of the 15-bit internal representation, are you stating this for just the Sony camera or for all cameras?

Photoshop CS2 has 32-bit channel mode, Photoshop 7.0 through 3.0 (in my direct experience) were limited to 16-bits per channel.

How do you know that Photoshop is using 15-bit internal representation? They should use 16-bits (or more) for 16-bits/channel and 32-bits for 32-bit/channel. It's easy enough to move pixel values to the lower 16-bits of a register, perform the operation on all 32-bits, then clip and store the lower 16-bit value. That's what I do. Speed- "not a problem" these days.

Also- 32768 is a 16-bit number, is '8000'x. 32767 is the largest 15-bit value, '7fff'x. EDIT- I have been working with 32-bit assembly for 25 years now. I default to 32-bit in HEX, then save off the lower word.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 23, 2015, 10:35:28 am
In the context of the 15-bit internal representation, are you stating this for just the Sony camera or for all cameras?

For all cameras.

How do you know that Photoshop is using 15-bit internal representation?

Adobe says so.

https://forums.adobe.com/thread/1790320

They should use 16-bits (or more) for 16-bits/channel and 32-bits for 32-bit/channel. It's easy enough to move pixel values to the lower 16-bits of a register, perform the operation on all 32-bits, then clip and store the lower 16-bit value. That's what I do. Speed- "not a problem" these days.

Tell that to Adobe.

Also- 32768 is a 16-bit number, is '80000000'x. 32767 is the largest 15-bit value, '7fffffff'x.

Precisely. That's why the Adobe encoding is sometimes called "15+1", as in 15 bit encoding with one extra value.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 23, 2015, 11:44:21 am
The way Adobe does it will lose resolution in the image as you apply processing- collisions will occur with the lost pixel depth. Too bad, with increased bit-depth from the cameras, most everything will require 32-bit integer or floating point to preserve resolution. I wrote code for image segmentation and pattern recognition- so artifacts were an issue. I've modified that software to process DNG files. I like the results.

I will have to see what Lightroom is doing with my 16-bit images, white set to 65535. If they do as they say, it should lose a bit and the white value should change in the new IFD that gets written in the file.

I read the discussion. Lightroom should look up the latest cycle times for the newer processors, 32-bit Integer operations are fast. They should do all of the operations in 32-bit integer, store back as 16-bit. The CPU registers are set up to handle this. It's been this was since the Pentium Pro was out. Which came after Photoshop 3.0. Maybe they have not bothered to rewrite that section of code.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 23, 2015, 11:56:46 am
Uncompressed files are faster to work with and faster to save and open, since there's no need to compress or decompress the file. Slightly larger file size, but storage space is cheap and high-resolution files already push performance envelopes.

Potentially, this could also translate to higher frame rates or larger buffers in camera, in addition to faster editing.

Actually the opposite is true as far as 'gamma' lossy compression is concerned: about 1/5th the data to move around, process and save.  The actual translation from 'lossy' compressed to uncompressed is just a look up table, an operation that can be performed very quickly by even simple DSPs, never mind the ultra sophisticated ones in our cameras.  On the other hand lossless compression/decompression is a memory and processor (power) intensive operation, which is why early DSLRs could not do it.  I don't think it's an issue with current cameras though.

In general the fastest FPS with Nikon DSLRs are typically obtained in 12-bit crop mode because there is less data to process (e.g. D7200 5 FPS in 14-bit mode, 6 FPS in 12-bit mode, 7 FPS in 12-bit 1.3x mode).

Jack
Title: My thoughts on "lossy" raw compression
Post by: ErikKaffehr on June 23, 2015, 12:04:42 pm
Hi,

I have not seen raw conversion artefacts with my Sony cameras. Now, this may depend on either my cameras or my subjects and shooting technique. Older Sony cameras have a choice of RAW vs. cRAW and I have almost exclusively used RAW. My guess is that RAW uses tone curve, which I basically regard to be a sound technique, while cRAW may use the Delta compression which may be loss free under many circumstances but can lead to demonstrable artefacts under some conditions.

It often happens that effects can be demonstrated, I have often issues with colour aliasing on my P45+, for instance. Others may have different subject, aperture usage, exposure times and such and don't have any problems. The Sony A7r shutter vibration is a similar issue. It is clearly measurable, but in most situations it may not be very observable.

As a parallell, there is a guy on GetDPI who shoots a lot of magnificent night pictures on a Pentax 645 and he gets some odd artefacts. Those are probably not coming from his Pentax but from processing. But, the case is that those things may not be a problem for a great majority of users but it is a real problem for Ed because the subjects he shoot take the processing pipeline to it's limits.

So my take is. Be aware there is a problem. If you don't see it on your Sony, be happy. If you shoot high contrast night subjects for living, the Sony may not be the platform of choice, there are others.

Would be nice if Sony made that Delta compression optional.

Personally, I have an A7rII on order, so that lossy compression doesn't scare me away. But I would never use it if it was an option.

Best regards
Erik




Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 23, 2015, 12:53:22 pm
The way Adobe does it will lose resolution in the image as you apply processing- collisions will occur with the lost pixel depth. Too bad, with increased bit-depth from the cameras, most everything will require 32-bit integer or floating point to preserve resolution. I wrote code for image segmentation and pattern recognition- so artifacts were an issue. I've modified that software to process DNG files. I like the results.


When I'm writing image processing code, I almost always use double-precision floating point with fs = 1.0, so I don't have to worry about rounding buildup, over and underflow, and negative intermediate values. But that code runs in batch mode.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 23, 2015, 02:33:28 pm
I just checked the DNG file as saved by Lightroom 6, after applying some processing on my 16-bit files. The White-Level is set to 'FFFF'x in the new SubIFD. So at least LR6 does not throw away the least significant bit when opening a linear-DNG file with 16-bit pixels.

I used a lot of double-precision on computers with code being ported from the 60-Bit single-precision CDC mainframes. So much that I wrote a FORTRAN-66 code to automatically do most of the source-code to source-code changes for me, including changing all of the constants to double-precision format, and changing Function names. Kind of nice, I get to tell people I earned my way through school be writing atomic structure programs. Some went back to 1957, on the IBM 704. To the 7090, CDC 6600, CDC-7600, then the TI-ASC vector supercomputer that I used. That machine was wonderful for image processing, could process sub-images of a 3-D image with one assembly language instruction. The computer used sets of registers to stream memory through the CPU, could use 3-deep loops to stride through memory. Nothing else ever built like it. Floating point multiply was 1 clock cycle, bit-shift took 1 cycle, floating Divide took 16.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: kers on June 23, 2015, 07:16:27 pm
From:

http://www.dpreview.com/articles/0717419525/interview-kimio-maki-of-sony-the-customer-s-voice-is-the-most-important-data-for-me (http://www.dpreview.com/articles/0717419525/interview-kimio-maki-of-sony-the-customer-s-voice-is-the-most-important-data-for-me)

One of our main criticisms of the a7-series has been raw compression. Is the raw processing of the a7R II the same as previous cameras?

KM: Right now it is the same, yes. We’re still working on it. In the future we may change the software but that’s not completed yet. We have consumers who require 14-bit etc., and we’re considering [how to deal with it].


+1 really on topic.
- i read the interview before and asked myself why is this discussion going on for 3 pages..? oh well..
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 23, 2015, 07:54:46 pm
Sony assumes that their customers are not going to apply a lot of post-processing to the images, hence they will be happy with the lossy/compressed file.

More sophisticated users know the difference, and cannot understand why any company would not offer lossless compression. Sony is not the only camera company to make this decision, Leica just made it with the M246- bringing it out as a 12-bit camera. They made the same mistake with the M8. At least with the M8 the firmware has a back-door to get lossless RAW out of it. I'm sure the Sony camera can be hacked in a similar fashion.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on June 23, 2015, 08:51:56 pm
Sony assumes that their customers are not going to apply a lot of post-processing to the images, hence they will be happy with the lossy/compressed file.

More sophisticated users know the difference, and cannot understand why any company would not offer lossless compression. Sony is not the only camera company to make this decision, Leica just made it with the M246- bringing it out as a 12-bit camera. They made the same mistake with the M8. At least with the M8 the firmware has a back-door to get lossless RAW out of it. I'm sure the Sony camera can be hacked in a similar fashion.

A stupid assumption, considering how many A7r adopters are ex-Canon refugees looking for a decent sensor to put behind their lenses, in order to shoot architecture, landscapes and other non-moving subjects. And Sony knows this, considering that they were including a Metabones adapter with every A7r body sold in Australia.

This isn't a stock-standard, low-cost, crop-sensor mirrorless system that appeals to point-and-shoot beginners wanting to 'step up'. It's a full-frame system with a top-tier sensor and professional users and needs the firmware to go along with the hardware.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Ray on June 23, 2015, 09:41:38 pm
Well, I'm not nearly as technically oriented as some of you guys, and I don't pretend to precisely understand these compression processes and all the circumstances where image flaws might become apparent during post processing, as a result of the lossy compression used.

However, what seems odd to me is that Sony should market an upgrade to the A7R which boasts the first BSI full-frame sensor, and yet still use the same old, lossy compression algorithms that the A7R uses.
Isn't the entire purpose of the BSI design to improve image quality, particularly dynamic range? Since the DR of 36mp full-frame sensors, as in the A7R, D800 and D810, is already close to 14 stops at the pixel level, wouldn't a BSI sensor push that limit even closer to a full 14 stops?

Out of curiosity, I just checked the DXOmark DR results for the A7R, D800E and so on. Whilst the D810 has a worthwhile advantage of around 2/3rds of a stop at its base ISO of 64, which is a lower ISO than the A7R's base ISO, the D800 and D800E have a fairly similar DR to the A7R, at their base ISOs of 100. The A7R is only a relatively insignificant 1/4th of a stop worse.

I wonder if Sony has lost a marketing opportunity here. In addition to announcing so many innovative features in the A7R2, the BSI design being one of the major features, surely Sony could have added to their marketing spiel, something along the following lines:

"In view of our breakthrough technology in being the first company to introduce a full-frame 35mm BSI sensor, we have considered it advantageous to our precious customers who have the ultimate, image-quality discernment, to also introduce the option for a full 14 bit uncompressed, RAW image pipeline, so that the full potential of our BSI design can be appreciated." (Or should I have written B ull S h I t design.)  ;D
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 24, 2015, 02:30:59 am
Sony assumes that their customers are not going to apply a lot of post-processing to the images, hence they will be happy with the lossy/compressed file.

More sophisticated users know the difference

Do you have an example image of 'lossy/gamma' raw compression showing artifacts when the uncompressed one doesn't, after a lot of PP?  I ask because I have looked (ignoring the delta thing) and I have yet to find one, no matter the PP.  Different data, yes; but with the same visual information.  Here's a question: how many bits of information (not data) do you need to fully encode scene information as captured by a specific current digital camera?  What would be the point of storing it in a larger data format, if scene information has been fully stored in both sets of raw data?  Information and rocket scientists have a pretty good idea.  Sony and Nikon speak to both.

Here  (http://www.strollswithmydog.com/iq-raising-iso-vs-pushing/)is an example where the same image information was apparently saved at 14 bits and 10 bits, with both subjected to aggressive adjustments (scroll down to the images in the 'Taking Video Path out of Equation' section).  No difference I can spot.  There is a huge stock of 'lossy' compressed captures PP'd by Nikon users going back at least 10 years, so I would hope Nikon has had plenty of time and hw iterations to fine tune what 'fully' means, with safety margins.  My D610 has an 11.5 bit lookup table.  Some think this is overkill (http://www.openphotographyforums.com/forums/showthread.php?t=5499).

I am genuinely interested in investigating this further so I would welcome some examples if anybody has any.  Thanks for yours, Erik, I agree with you that that artifact should not be caused by 'gamma encoding'.

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 24, 2015, 04:51:56 am
I can only answer this question for the Leica M8. I've stopped using the DNG-8 and (almost) always shoot it with the recently unlocked RAW mode. Button dance, then converted to full 14-bit DNG.

This is with RAW mode, full 14-bits:

(https://c2.staticflickr.com/8/7588/16592172458_4ec9bb6279_o.jpg) (https://flic.kr/p/rhcdNW)L1000439 (https://flic.kr/p/rhcdNW) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

This is the "Difference Frame", computed by taking the above image, compressed to DNG-8, expanded, then subtracted. Values scaled back to original white values.

(https://c2.staticflickr.com/8/7639/16572476567_f9b4c12e9b_o.jpg) (https://flic.kr/p/rfsgUB)DNG8DIF2 (https://flic.kr/p/rfsgUB) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

This is the portion of the image that gets thrown out.

Below v

This is the Leica M8 at ISO2500, DNG-8 as supported by the firmware:

(https://c2.staticflickr.com/8/7439/15898581794_d5eab81e71_o.jpg) (https://flic.kr/p/qdUotd)M8_3_DNG8_ISO2500 (https://flic.kr/p/qdUotd) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

100% crop:

(https://c2.staticflickr.com/8/7640/16751840270_37bdf2dd5a_o.jpg) (https://flic.kr/p/rwiyuf)L1015453_crop (https://flic.kr/p/rwiyuf) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

Same shutter-speed/F-Stop, ISO160, RAW mode, then pushed 4 Stops in Post.

(https://c1.staticflickr.com/9/8601/16334908409_bbea231553_o.jpg) (https://flic.kr/p/qTsFaT)M8_3_F15_ISO2500 (https://flic.kr/p/qTsFaT) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

100% crop:

(https://c2.staticflickr.com/8/7591/16751840400_fd494ebce3_o.jpg) (https://flic.kr/p/rwiywu)L1015454_Crop (https://flic.kr/p/rwiywu) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

No noise-reduction, No Sharpening

I took the M Monochrom and the M8 to the Marine Museum, set the M Monochrom to ISO5000 and set the same shutter-speed/F-Stop on the M8 with the M8 at ISO160 and RAW mode for this shot:

(https://c2.staticflickr.com/8/7388/16521819645_036749e86f_o.jpg) (https://flic.kr/p/raYDnp)M8_5_F4_ISO5000 (https://flic.kr/p/raYDnp) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

Then pushed in post.

This image is "unique", Orange filter to block Blue, leaving that channel sensitive to IR.

(https://c2.staticflickr.com/8/7594/17007670886_a5c91a81f9_o.jpg) (https://flic.kr/p/rUUL1W)I1015873 (https://flic.kr/p/rUUL1W) by fiftyonepointsix (https://www.flickr.com/photos/90768661@N02/), on Flickr

The Blue channel is then pushed to match the intensity of the other two channels, DNG channel assignment "rotated" as in Infrared Ektachrome. Custom FORTRAN code written for this.

So- I figure almost a 2-stop gain by having the uncompressed image from the 9-year old sensor. On the Nikon Df, did a test between compressed/uncompressed, also picked up ability to push in post. But- did not examine it in detail. Just never used compressed again.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 24, 2015, 09:36:07 am
I can only answer this question for the Leica M8. I've stopped using the DNG-8 and (almost) always shoot it with the recently unlocked RAW mode. Button dance, then converted to full 14-bit DNG.

This is with RAW mode, full 14-bits:

Thanks Brian,

A quick check on Leica M8's DNG-8 came up with this

Quote
Leica M8 raw files are saved as a 8-bit DNGs.

Here we are debating whether 11-bits is enough to contain scene information: we know that 8-bits is definitely not enough - as shown by the images converted to 8-bits in the earlier link.  One last comment: difference images are popular in this context but they typically do not prove or disprove loss of information: we all agree that the data is different (heck one set is typically 10-20% the size of the other).  The question is whether the visual information is different.

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 24, 2015, 11:51:29 am
The way to answer the question would be to take a 14-bit image that is lossless, and implement the Sony compression algorithm for it. Compress/Decompress the image, then subtract from the original. The difference frame is what you are losing. As I own an M8, and the file format is documented, was the easiest for me to work with.

Does anyone have a link to source code that implements the Sony compression/decompression scheme? I could incorporate "C" code into my DNG code, compress/uncompress then do the difference frame. I also have the IEEE Mixed-Radix FFT in the same code. I got what was needed from the difference frame- at 100% crop the edges stood out, but the frequency content is what most depend on.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on June 24, 2015, 02:05:50 pm
Hi,

Jim Kasson has made a lot of work on that about a year ago. This is at the end of those discussions: http://blog.kasson.com/?p=4854

Best regards
Erik

The way to answer the question would be to take a 14-bit image that is lossless, and implement the Sony compression algorithm for it. Compress/Decompress the image, then subtract from the original. The difference frame is what you are losing. As I own an M8, and the file format is documented, was the easiest for me to work with.

Does anyone have a link to source code that implements the Sony compression/decompression scheme? I could incorporate "C" code into my DNG code, compress/uncompress then do the difference frame. I also have the IEEE Mixed-Radix FFT in the same code. I got what was needed from the difference frame- at 100% crop the edges stood out, but the frequency content is what most depend on.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 24, 2015, 04:28:55 pm
The way to answer the question would be to take a 14-bit image that is lossless, and implement the Sony compression algorithm for it. Compress/Decompress the image, then subtract from the original.

Right Brian, this is what I was referring to. Imho the subtraction would show the difference in the data, not necessarily the difference in visual information contained within the data.  It may show you where compression occurred, that's it.

If you've followed the balance example (http://www.strollswithmydog.com/how-many-bits-to-fully-encode-my-image/), it would be like saying that the difference between the weight of the same tomato weighed and written down by person one (0.52428kg) vs that recorded by person two (0.52Kg) is 4.28 grams.  But if the best ever accuracy possible out of the balance is just two significant digits and no more - what value does the difference have?  It's just a random number which just shows you that in that particular spot compression was at work.  So next you need to look at the compressed vs uncompressed images there.  If you cannot see a difference, even after equal amounts of PP, there was no loss of visual information.  If there was, you'll be able to see it.  As was shown in the earlier link.

Data is the frame, visual information is the picture.  The picture does not change when one changes the frame as long as the new frame is big enough. 

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 24, 2015, 07:27:55 pm
I  have used difference frames for 35 years when doing image processing. It's the way to find out what has changed when performing an operation on an image. The problem with the Sony compression algorithm is that it depends on image content and the performance is chaotic.

I've used difference frames to examine image compression for JPEG, Wavelets, and other schemes over the decades. It shows the information lost, and makes artifacts that get introduced by the algorithm stand-out. My boss 35 years ago would ask "does it preserve the point response of the sensor?" Noise spikes that exceed the point-source response of the system, those are fair game. Diminish the point response, throw away information. I'd rather have the noise included in the image, makes it easier to get rid of.

There has been a lot of examples on the Internet showing the effects of the 11-bit compression scheme on images. If I owned one, I'd probably write code to test it myself- once the Raw mode had been unlocked. As it is, I am happy with the Nikon Df, Leica M9, M Monochrom, and M8. With Sony- too bad the first BSI Full-Frame sensor has been crippled.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on June 24, 2015, 07:32:53 pm
The question is really where you come from and what additional value you can expect to get.

If smaller size is important then the value of the a7rII probably far over shadows the concern with lossy raw.

If size isn't that important, then there are other excellent options.

As far as I am concerned, I'll wait until this is fixed.

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 25, 2015, 04:39:13 pm
I've used difference frames to examine image compression for JPEG, Wavelets, and other schemes over the decades. It shows the information data lost, and makes artifacts that get introduced by the algorithm stand-out. My boss 35 years ago would ask "does it preserve the point response of the sensor?" Noise spikes that exceed the point-source response of the system, those are fair game. Diminish the point response, throw away information. I'd rather have the noise included in the image, makes it easier to get rid of.

That sounds like good advice, Brian.  How would one apply it to a photographic image?

I'd rather have the noise included in the image, makes it easier to get rid of

If one is throwing away random digits one can always add them back at any time if needed.  I can't think of a situation where they would be, but one could.

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 25, 2015, 05:31:11 pm
I  have used difference frames for 35 years when doing image processing. It's the way to find out what has changed when performing an operation on an image. The problem with the Sony compression algorithm is that it depends on image content and the performance is chaotic.

I've used difference frames to examine image compression for JPEG, Wavelets, and other schemes over the decades. It shows the information lost, and makes artifacts that get introduced by the algorithm stand-out. My boss 35 years ago would ask "does it preserve the point response of the sensor?" Noise spikes that exceed the point-source response of the system, those are fair game. Diminish the point response, throw away information. I'd rather have the noise included in the image, makes it easier to get rid of.

There has been a lot of examples on the Internet showing the effects of the 11-bit compression scheme on images. If I owned one, I'd probably write code to test it myself- once the Raw mode had been unlocked. As it is, I am happy with the Nikon Df, Leica M9, M Monochrom, and M8. With Sony- too bad the first BSI Full-Frame sensor has been crippled.

Here are two different ways of looking at original and cRAW companded difference frames:

http://blog.kasson.com/?p=4838

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 25, 2015, 05:34:33 pm
That sounds like good advice, Brian.  How would one apply it to a photographic image?


WRT cRAW compression, the only way I've found it at all useful is with simulations, since the real original is not accessible (and my, in fact, never exist).

Even with simulations, it's of limited utility to me, since it doesn't inform on the issue of how visible are the artifacts.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 25, 2015, 06:44:12 pm
As the 14-bit raw images are not available from the Sony, the easiest way to test this would be to implement the compression/decompression algorithm and apply it to an uncompressed image from a different camera. The Leica files are easy to work with as the file format is documented. Unpack the image, apply the compression/decompression, subtract from the original.

http://www.rawdigger.com/howtouse/sony-craw-arw2-posterization-detection

A good description of the algorithm. Basically an 11-bit tone curve using further compression in 32-pixel chunks, that scales max-min to 7-bits. Chaos.

With noise reduction- it's easier to eliminate from the image when it has been resolved, rather than "folded" into the image. The noise is "too sharp" when it is resolved. The noise will always be there, it's analog. If you don't have enough bits to resolve the noise, it will still affect the image at contour boundaries- analog levels that are at the limit of a bit. The noise will cause levels at a contour to fluctuate between them, easy to see with the Leica DNG-8. Average noise may be reduced, peak-to-peak will be increased.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 25, 2015, 07:10:03 pm
As the 14-bit raw images are not available from the Sony, the easiest way to test this would be to implement the compression/decompression algorithm and apply it to an uncompressed image from a different camera. The Leica files are easy to work with as the file format is documented. Unpack the image, apply the compression/decompression, subtract from the original.

With noise reduction- it's easier to eliminate from the image when it has been resolved, rather than "folded" into the image. The noise is "too sharp" when it is resolved. The noise will always be there, it's analog. If you don't have enough bits to resolve the noise, it will still affect the image at contour boundaries- analog levels that are at the limit of a bit. The noise will cause levels at a contour to fluctuate between them, easy to see with the Leica DNG-8. Average noise may be reduced, peak-to-peak will be increased.

Here's what you asked for, but using a synthetic image plus simulated photon noise instead of a real image with the real photon noise:

http://blog.kasson.com/?p=4847

And here is the difference image if you leave out the delta mod piece of the algorithm, and just keep the tone curve:

http://blog.kasson.com/?p=4854

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 25, 2015, 08:05:51 pm
The algorithm is not much more difficult to code up than the Leica DNG-8, the 11-bit/7-bit algorithm is similar to the lossless running difference compression scheme that I coded up 30 or so years ago. For the latter, the first raw pixel value is stored for each row then differences between pixels coded as 8-bits, -127:127 with 'ff'x reserved as an escape to store a full value pixel when the difference exceeded 8-bits.

Sony's 11-bit tone curve has a maximum error of 32 counts, before the 7-bit lossy compression compounds it. The Leica DNG-8 has a maximum error count of 127 counts. Peak-to-Peak noise is increased by the max error count.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on June 25, 2015, 09:59:16 pm
The algorithm is not much more difficult to code up than the Leica DNG-8, the 11-bit/7-bit algorithm is similar to the lossless running difference compression scheme that I coded up 30 or so years ago. For the latter, the first raw pixel value is stored for each row then differences between pixels coded as 8-bits, -127:127 with 'ff'x reserved as an escape to store a full value pixel when the difference exceeded 8-bits.

Sony's 11-bit tone curve has a maximum error of 32 counts, before the 7-bit lossy compression compounds it. The Leica DNG-8 has a maximum error count of 127 counts. Peak-to-Peak noise is increased by the max error count.

The main advantage for Sony's lossy scheme is that it is performed on fixed parcels that can be implemented in simple hardware registers on the sensor. 

I would be very interested to take a D810 lossless raw, and a copy of the D810 raw converted to Sony lossy raw, and perform an extensive set of DSP filters/adjustments on both of them to see what artifacts one would find. 
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 26, 2015, 03:44:22 am
Here's what you asked for, but using a synthetic image plus simulated photon noise instead of a real image with the real photon noise:

http://blog.kasson.com/?p=4847

And here is the difference image if you leave out the delta mod piece of the algorithm, and just keep the tone curve:

http://blog.kasson.com/?p=4854

Jim

Very good analysis, Jim.  If those routines are still around it wouldn't be too hard to implement Luke's idea below with a natural image, would it? ;-)
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 26, 2015, 05:09:30 am
I had the misconception that Sony was somehow using a 14-bit base value rather than an 11-bit curve with a 7-bit difference spread across 16 pixels. think of it as "Bit Robbing". It compresses to 8-bits, acts like an 11-bit curve most of the time, and can go horribly bad in some situations. I'm thinking of some of the Vivitar ads with the girl flinging her hair up and the flash firing behind her. Artifacts will be much worse than those created by the DNG-8 scheme based on image content as the error will span 32 pixels. Why would any sane company develop a full-frame BSI sensor and use this algorithm?

I'll probably code up the algorithm in FORTRAN, plug it in where I have the DNG-8. I suspect the curve shown in the Link is approximate, with just a few data points on the curve shown and lines connecting them. The DNG-8 curve is generated by multiplying times 4 to get 16-bits, then take the square root. A similar DNG-11 curve would be computed by multiplying the 14-bit value by 256 (shift by 8 bits) to give a 22-bit value, then taking the square root to get to 11-bits. Nikon .NEF files are not as easy to work with as the Leica DNG files. The latter stores the entire image as one strip, just read it into a buffer and pass it to the subroutine as a 2-D array with Column and Row read in from the Sub-IFD.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 26, 2015, 11:10:03 am
Very good analysis, Jim.  If those routines are still around it wouldn't be too hard to implement Luke's idea below with a natural image, would it? ;-)


Not too hard. There are three obstacles, all probably surmountable.

The first is that I'm hard at work on my book and the new synthetic slit scan project.

The second is that it'll take me a while to remember how to set up the scripts to do the processing. You've seen my Matlab code. While I subscribe to the theory that a computer program is a communication with those who will come after and only incidentally with the computer that will run it, I never seem to be able to turn that maxim into practice.

The worst example I ever saw of this was thankfully not written by me. When I worked at hp, there was a bug in a 2116A ADC driver. I took a look at the code. It was written in assembler, which needs comments more than Matlab. There were pages of commentless code. Finally I saw a comment next the the instruction CMA,INA, which complements the A register and increments the result by one. The comment, repeated here in its entirety, was, "Make A negative."

The fact that the companding code is embedded in the camera simulator means there's more code to re-understand. It shouldn't take very long, but it's my least favorite programming task.

The third is finding a good candidate image. And that raises the Clintonian question, since which image would be good depends on the definition of good in this context. Is a good image one that will show a big difference after companding? Just what are we trying to discover/prove here?

I could use some help with that last one.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 26, 2015, 02:57:25 pm
 
Not too hard. There are three obstacles, all probably surmountable.

Didn't mean to put you in the spot, Jim, so don't feel that you need to do anything.

Quote
The comment, repeated here in its entirety, was, "Make A negative."

:)

Quote
The third is finding a good candidate image. And that raises the Clintonian question, since which image would be good depends on the definition of good in this context. Is a good image one that will show a big difference after companding? Just what are we trying to discover/prove here?

My own curiosity is whether Sony's 11-bit lossy compression can be made to give up the ghost in certain situations after aggressive PP.  By that I mean that processing both uncompressed and compressed images equally, the compressed one clearly shows loss of visual information compared to the other.  Based on past experience there will be a qualitative element to marginal differences ('if you add noise to those blocked blacks in image 1 they look no better or worse than those noisy blacks in image2') but if there are significant differences they should make themselves known rather easily.

If and when you may want to do it, perhaps it would be useful to put more than one image through the routines.  How about something with sharp corners and something landscapy with a smooth gradient, or even the classic slanted lamp on a wall?  Luke?  Everyone else?

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on June 26, 2015, 03:40:41 pm

My own curiosity is whether Sony's 11-bit lossy compression can be made to give up the ghost in certain situations after aggressive PP.  By that I mean that processing both uncompressed and compressed images equally, the compressed one clearly shows loss of visual information compared to the other.  Based on past experience there will be a qualitative element to marginal differences ('if you add noise to those blocked blacks in image 1 they look no better or worse than those noisy blacks in image2') but if there are significant differences they should make themselves known rather easily.

If and when you may want to do it, perhaps it would be useful to put more than one image through the routines.  How about something with sharp corners and something landscapy with a smooth gradient, or even the classic slanted lamp on a wall?  Luke?  Everyone else?


I've demonstrated that, with synthetic images, you can see the artifacts if you push hard enough in post, but it isn't easy to get the problems to show their ugly heads. I'm not sure what it would prove to do that with real images. We know there are a few out there.

Also, as I think about it, changing the code to work with real raw files isn't trivial; I need to provide a path into the camera simulator for real raw files or take the companding code and port it to its own class. In addition, I'd have to demosaic the images in Matlab. I still haven't figured out how to write raw files that can be decoded with just any raw developer.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 26, 2015, 05:16:20 pm
The last A/D driver I wrote was 15 years ago, I needed an optical feedback loop. Take 1 A/D sample, Do 1 D/A conversion. The software that came with the card expected an array for each call to the driver, did the full setup for the A/D and waited for the pre-amp to settle with each call. Mine ended up being 100x faster for the job required.

If I get some time- will look at implementing the algorithm in FORTRAN. It does not look hard, dimension each row as (2, 16, n/ 32) where n is the number of pixels in each row. Apply the gamma curve, find max/min, compute differences from the min and scale to (Max-Min) to store as 7-bits. Reverse and restore.  My code is setup to do conversions on batches of images, just tell it where the path to the DNG files. I've been using it to convert 14-bit linear DNG files from the M Monochrom to 16-bits using a Gamma curve. The idea is the extra 2-bits prevents collisions. I had fun digging out code that started in the 1980s and having it process DNG. The last time I used it- TIFF 6.0 was brand new and my Wife had a 1MPixel microscope camera that stored all of the images in 1 TIFF file. Nothing would process past the first image, so I wrote her a code to Unpack them. Also wrote code to implement her algorithm to identify and count cancer cells in the image.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 26, 2015, 09:45:39 pm
According to the Rawdigger explanation- the first 256 values of the 11-bit compressed table all get converted to output intensity 0. That seems very wasteful way to preserve a Black value. The DNG files store Black Level as a parameter in the SUB-IFD. The 14-bit to 11-bit compression table seems to be tabular with just a few bin sizes. Same with the decompression. This is not an optimal way of doing things, if this explanation is actually how the Sony scheme works. I'm starting with a 22-bit to 11-bit square root compression, then implement the 7-bit difference frame- which is how the artifacts spread across a larger 3x32 segment of the image, after color-demosaic.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: logeeker on June 26, 2015, 10:40:00 pm
SAR said Sony is going to make the a7R II have the feature of uncompressed RAW by a firmware update.
However, I'm bit worry of that it finally become a lossy uncompressed RAW file.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 27, 2015, 06:51:29 am
I've demonstrated that, with synthetic images, you can see the artifacts if you push hard enough in post, but it isn't easy to get the problems to show their ugly heads. I'm not sure what it would prove to do that with real images. We know there are a few out there.

Hi Jim,  I may have misinterpreted this  (http://[quote author=Jim Kasson link=topic=101380.msg832950#msg832950 date=1435347641)post of yours but I thought I had understood that as far as the 14 (or whatever) to 11 bit compression sans delta (what I am interested in) you had not gone as far as checking loss of visual information, i.e. two equally processed images side by side, one compressed one not, the latter showing more visual information than the former.  I know this can be easily done at 8-bits, I know I could not see any sign of it at 9.5 bits with my 12-bit D90 and 11.5 with my 14-bit D610, so I am curious about Sony's 11 bit implementation (no delta, I think they'll kill that).

Also, as I think about it, changing the code to work with real raw files isn't trivial; I need to provide a path into the camera simulator for real raw files or take the companding code and port it to its own class. In addition, I'd have to demosaic the images in Matlab. I still haven't figured out how to write raw files that can be decoded with just any raw developer.

Don't worry about it, Jim.  But in any case nothing wrong with demosaicing in Matlab imo.

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 27, 2015, 08:42:22 am
The Sony compression scheme does more than just compressing to 11-bits/decompressing to 14-bits. The 7-bit delta function is where things go horribly bad. The 7-bit scheme is scaled to cover the max-min range within the strip. It loses the 11-bit Bin number for the decompression, and can cause a much larger difference from the original.

This is more like an 8-bit compression scheme where the error is spread across a sub-image and you just hope there is not much going on to trigger it.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jack Hogan on June 27, 2015, 08:51:21 am
The Sony compression scheme does more than just compressing to 11-bits/decompressing to 14-bits. The 7-bit delta function is where things go horribly bad. The 7-bit scheme is scaled to cover the max-min range within the strip. It loses the 11-bit Bin number for the decompression, and can cause a much larger difference from the original.

This is more like an 8-bit compression scheme where the error is spread across a sub-image and you just hope there is not much going on to trigger it.

Got it Brian.  Perhaps I should try to understand Sony's raw better before switching to visual mode.

Jack
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on June 27, 2015, 10:36:42 am
To answer the original question-

Make a T-Shirt with a pathological case image designed to cause the Sony compression algorithm to fail. Give it away to new Sony camera owners family and friends.

Label it "Sony! Wear it in the Raw."
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: KAHA on July 13, 2015, 08:20:44 am
I'm not sure if any one's seen this post from Rishi Sanyal (technical editor at dpreview) regarding Sony's lossy Raw....

Quote
"Just to clarify - lossy Raw does not mean a 1-2 EV dynamic range cost. It means that b/c of the encoding, there can be artifacts around high contrast edges; it doesn't, to our knowledge, have any effect on dynamic range (other than that huge pushes can show some artifacts, but that's not the noise you traditionally associate with poor dynamic range).

Also, the 12-bit ADC mode is a totally separate issue. As long as you're not shooting continuous, or Bulb mode, or have Long Exposure NR turned on and the camera actually uses it (b/c you have a longer than 2s exposure), or you're shooting continuous mode, or bracketing... as long as you're not doing any of these things, you get full 14-bit capture.

But, it is kind of obnoxious that doing any of those things switches the camera into a mode that sacrifices a good deal of dynamic range. For example, how ironic is it that if you bracket your shot to deal with a high DR scene, your camera's DR drops?

That's pretty annoying, in my book."
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: AlterEgo on July 13, 2015, 11:20:25 am
I'm not sure if any one's seen this post from Rishi Sanyal (technical editor at dpreview) regarding Sony's lossy Raw....
seriously ? this thread has many people whom you shall read before anything "technical editor at dpreview" might want to say...
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 13, 2015, 01:36:09 pm
seriously ? this thread has many people whom you shall read before anything "technical editor at dpreview" might want to say...

True, but what Rishi said was accurate, IMHO, except for the part about being about being able to get 14-bit precision sometimes.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: AlterEgo on July 13, 2015, 02:07:06 pm
True, but what Rishi said was accurate, IMHO, except for the part about being about being able to get 14-bit precision sometimes.
and which camera __with new raw implementation__ from Sony does get truly full 14 bit precision then ?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on July 13, 2015, 02:24:18 pm
Dynamic range is a very different issue than bit depth. A sensor with high dynamic range and low bit-depth is "analogous" to a low-gamma film. A large change in intensity is required to change the output in the A/D. This is related to spatial resolution: if you cannot differentiate changes in intensity, details will become lost in the intensity contour. The Sony camera uses 11-bits a/d conversion and a non-linear scale to fake 14-bit values. Resolution is lost as the slope of the analog/digital conversion is reduced. why have 42MPixels if you are going to kill resolution by skimping on the A/D converter?

The KAF-1600 used in my 22 year old DCS200 has a linear dynamic range of 74dB. 8-Bit A/D. The first Digital Infrared Sensor that I worked on in 1981 used 12-bits, and the second one in 1984 used 15 bits. (They were expensive) Linear dynamic range has not changed that much since the KAF-1600, certainly not the same order of magnitude as megapixels. Prices have dropped. the DCS200 was $12,400- but mine is full-spectrum, was an extra $4K.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 13, 2015, 03:20:10 pm
and which camera __with new raw implementation__ from Sony does get truly full 14 bit precision then ?

None, AFAIK. The cRAW 13 -> 11 tone compression algorithm ignores a 14th bit, if present.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 13, 2015, 03:28:48 pm
Dynamic range is a very different issue than bit depth. A sensor with high dynamic range and low bit-depth is "analogous" to a low-gamma film.

I don't get the analogy. Please elucidate.

A large change in intensity is required to change the output in the A/D. This is related to spatial resolution: if you cannot differentiate changes in intensity, details will become lost in the intensity contour.

All that depends on how much Pre-ADC dither is present.

The Sony camera uses 11-bits a/d conversion and a non-linear scale to fake 14-bit values.

How do you know that the ADCs are 11 bits?

Resolution is lost as the slope of the analog/digital conversion is reduced. why have 42MPixels if you are going to kill resolution by skimping on the A/D converter?

In my testing and simulations, the cRAW compression/expansion does not affect resolution, as measured by MTF50 or MTF30.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: AlterEgo on July 13, 2015, 03:43:11 pm
None, AFAIK.
and technical editor has way higher obligation to be precise with his words vs a regular poster - that is the point of being an editor... that is about mr Rishi of DPReview.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on July 13, 2015, 05:11:02 pm
A low gamma film requires a large change of intensity of light to produce a change in density. A film with high gamma requires a smaller change of intensity to  for s similar change of density. A sensor with a high dynamic range and low bit-depth requires a large change of intensity to change the value of the pixel. The Leica M246 requires 8 times the change in level of light to increment the output of the a/d converter compared to a Leica M Monochrom.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 13, 2015, 06:02:52 pm
A low gamma film requires a large change of intensity of light to produce a change in density. A film with high gamma requires a smaller change of intensity to  for s similar change of density. A sensor with a high dynamic range and low bit-depth requires a large change of intensity to change the value of the pixel.

Not with proper dithering.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on July 13, 2015, 08:10:15 pm
Dithering reduces resolution.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 13, 2015, 11:25:48 pm
Dithering reduces resolution.

Dithering is inevitable with current sensor technology, by RN. Dithering is inevitable when photon detection is involved, with Poisson statistics thanks to physics.

I have tested very few sensors where the combination wasn't adequate to prevent posterization. The D810 at ISO 64 in 12 bit mode is an exception in deep shadows, but not in 14 bit mode.

http://blog.kasson.com/?p=8770

So are a few Sony sensors at base ISO in 12 bit mode in deep shadows, but not in 13 bit mode.

http://blog.kasson.com/?p=8586

Jim

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on July 14, 2015, 04:57:47 am
Dithering is an intentional introduction of noise. Better to have enough bit-depth to resolve the real noise of a sensor. Maybe unless the sensor has a lot of fixed-pattern noise in it, and some of them certainly do.

I don't want noise in an image. I prefer a clean signal with lots of bit-depth. Gives you more to work with.

If people are happy with a lossy compression scheme used to destroy the image of a 42MPixel sensor in a $3000+ camera- great. Sony has better marketing than they do engineers.

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on July 14, 2015, 07:17:21 am
Hi,

The noise that Mr. Kasson refers to is a combination of shot noise and redout noise. If you collect 100 pothons the shot noise will be around 10 photons. So some noise is also present. Some of the noise is coming from reading the sensor.

Best regards
Erik


Dithering is an intentional introduction of noise. Better to have enough bit-depth to resolve the real noise of a sensor. Maybe unless the sensor has a lot of fixed-pattern noise in it, and some of them certainly do.

I don't want noise in an image. I prefer a clean signal with lots of bit-depth. Gives you more to work with.

If people are happy with a lossy compression scheme used to destroy the image of a 42MPixel sensor in a $3000+ camera- great. Sony has better marketing than they do engineers.


Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 14, 2015, 12:17:32 pm
Dithering is an intentional introduction of noise.

I find that definition overly constrained in two aspects.

The first in the use of the word noise. Wholly deterministic waveforms can provide useful dither.  I myself have a patent on one such technique:

http://www.google.com/patents/US4187466

The second is the word intentional. The result is the same whether the non-signal component is natural or made for the use, and the electrons can't understand intention.


Better to have enough bit-depth to resolve the real noise of a sensor.

If shot noise is real noise, and I think it is, that means that you need to be able to resolve an individual electron. That would take a precision greater than (how much greater than is subject to some debate) log2(FWC).

Maybe unless the sensor has a lot of fixed-pattern noise in it, and some of them certainly do.

I don't want noise in an image. I prefer a clean signal with lots of bit-depth. Gives you more to work with.

You don't get a choice. Even with a ideal sensor, you'll have shot noise. With real sensors, you'll have PRML, and various flavors of RN.

If people are happy with a lossy compression scheme used to destroy the image of a 42MPixel sensor in a $3000+ camera- great. Sony has better marketing than they do engineers.

Your use of the word destroy is curious. I have done a great deal of testing and simulation of cRAM compansion. I think I know what it can do well, and where it has problems. I've never seen, in real images, or in simulations where photon noise is simulated, any artifacts that merit the use of the word destroy. I invite you to post examples.

Would I be happier if Sony didn't use cRAW compression? Sure. Will their use of that form of compression stop me from buying more cameras from them? Heck, no.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BrianVS on July 14, 2015, 05:56:14 pm
You need more than 11-bits in the Sony sensor to get down to the noise. Using a shallow bit depth and introducing artificial noise to obscure banding- a poor substitute for proper bit-depth to resolve the charge collected. I would rather have as close to the actual count collected then some 11-bit a/d using a non-linear scale and then destroying the bit-depth further compressing it to 7 bits. But, that's why I use cameras with 14-bits. Why Leica switched to 12-bits with the M246 is a mystery, the sensor is better than that. The M8 output was destroyed by the DNG-8; now having unlocked 14-bit pixels proved that.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 15, 2015, 11:16:51 am
You need more than 11-bits in the Sony sensor to get down to the noise.

You continue to make bold (and wrong,  IMHO) statements with no supporting material.

Here's Sony's latest shipping a7x camera modeled by moi:

http://blog.kasson.com/?p=8194

The relevant numbers are: FWC = 60,000 electrons, frame-to-frame variable read noise at base ISO of 6+ electrons, rms.

At base ISO, in the m=1 part of the cRAW tone curve, 1 LSB = 7 electrons. Therefore RN is about 1 LSB rms.

That’s enough dither even with no photon noise at all. Here’s why:

http://blog.kasson.com/?p=6513

Other researchers report perfect dither at higher values, up to 1.3 LSB. Jack Hogan is the local expert on that. In any event, 1 LSB is in the ballpark of even the most stringent criteria.

No signal is the worst case, since the photon noise helps as m changes higher on the tone curve.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 15, 2015, 11:20:07 am
Using a shallow bit depth and introducing artificial noise to obscure banding- a poor substitute for proper bit-depth to resolve the charge collected.


Are you asserting that Sony introduces "artificial" noise in the a7x cameras? If so, please tell us why you believe that to be the case.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 15, 2015, 11:44:56 am
...I would rather have as close to the actual count collected then some 11-bit a/d using a non-linear scale and then destroying the bit-depth further compressing it to 7 bits...


This is the second time you have asserted that the a7x ADCs are nonlinear, rather than linear with a tone curve applied digitally after conversion. That has been speculated, but, AFAIK, never proven. So, I ask you again, how do you know for sure?

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: AlterEgo on July 15, 2015, 12:32:56 pm
This is the second time you have asserted that the a7x ADCs are nonlinear, rather than linear with a tone curve applied digitally after conversion. That has been speculated, but, AFAIK, never proven. So, I ask you again, how do you know for sure?

Jim

what is cheaper to do - linear ADCs on die or non linear ADCs on die ... does it make sense for Sony Semi to manufacture for example 24mp sensor specifically for Sony Imaging with non linear ADCs and for Nikon with linear ADCs - hurts the economy of scale ... or they have ADCs which can operate in both modes (then more complex - why the need ?)
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on July 15, 2015, 02:24:48 pm
Hi,

I would assume that the kind of ACD-s Sony uses can be adjusted by firmware. I would assume they are digital ramp type devices.

I would assume that Jim is quite right about quantisation errors being masked by natural noise. AFAIK there are some references to Emil Martinec discussing this earlier on this thread.

I would say that the Sony solution should be pretty close to loss less except some extreme scenarios, like the star trail images published by Diglloyd and on the DPReview site. In both cases the problem is caused by delta compression rather than tone curve.

I have much less worry about Sony raw compression that may affect perhaps one in a million images than lack of OLP filtering that will affect 100% of image that have been shot in optimal conditions.

Best regards
Erik



what is cheaper to do - linear ADCs on die or non linear ADCs on die ... does it make sense for Sony Semi to manufacture for example 24mp sensor specifically for Sony Imaging with non linear ADCs and for Nikon with linear ADCs - hurts the economy of scale ... or they have ADCs which can operate in both modes (then more complex - why the need ?)
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: AlterEgo on July 15, 2015, 02:30:44 pm
I would assume that the kind of ACD-s Sony uses can be adjusted by firmware. I would assume they are digital ramp type devices.

so Sony is putting more complex ADC devices on die that can do non linear conversion or linear conversion on demand as the camera manufacturer wants, right ?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on July 15, 2015, 02:40:25 pm
Hi,

The ramp type converter is actually very simple and quite accurate. It should offer great flexibility in conversions. It may be slow, though but Sony uses several thousands of them in parallel.

Best regards
Erik


so Sony is putting more complex ADC devices on die that can do non linear conversion or linear conversion on demand as the camera manufacturer wants, right ?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 15, 2015, 02:56:55 pm
so Sony is putting more complex ADC devices on die that can do non linear conversion or linear conversion on demand as the camera manufacturer wants, right ?

There is no question that a single-slope ADC can be made nonlinear by controlling the ramp, which could be generated by a DAC, and thus flexible. The question is: "Do a7x ADCs work that way in fact?"

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on July 16, 2015, 04:56:59 pm
I find that definition overly constrained in two aspects.

The first in the use of the word noise. Wholly deterministic waveforms can provide useful dither.  I myself have a patent on one such technique:

http://www.google.com/patents/US4187466

The second is the word intentional. The result is the same whether the non-signal component is natural or made for the use, and the electrons can't understand intention.

If shot noise is real noise, and I think it is, that means that you need to be able to resolve an individual electron. That would take a precision greater than (how much greater than is subject to some debate) log2(FWC).

You don't get a choice. Even with a ideal sensor, you'll have shot noise. With real sensors, you'll have PRML, and various flavors of RN.

Your use of the word destroy is curious. I have done a great deal of testing and simulation of cRAM compansion. I think I know what it can do well, and where it has problems. I've never seen, in real images, or in simulations where photon noise is simulated, any artifacts that merit the use of the word destroy. I invite you to post examples.

Would I be happier if Sony didn't use cRAW compression? Sure. Will their use of that form of compression stop me from buying more cameras from them? Heck, no.

:-)

The word "intentional" is appropriate here actually.  Recall my claims on "noise" as being a kind of signal that is judged deleterious to the "proper function" of an information consumer.  A sensor coupled with a processing chain /may, by design, permit/ signal components, otherwise often considered as noise in other functions, to be recruited as a part of its proper function. 

Meanwhile.  As much as I can understand the claims that the missing codes do not make a practical difference, I remain incensed at the very idea that Sony throws away good bits before giving me the package.  I feel somehow, *ahem* sure, that one day I will find cases where this makes a difference.  But meanwhile, I just want the bits.  Give me the bits, all the bits, and nothing but the bits.  I'll decide what I need and what I don't.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 16, 2015, 07:32:01 pm
:-)

The word "intentional" is appropriate here actually.  Recall my claims on "noise" as being a kind of signal that is judged deleterious to the "proper function" of an information consumer.  A sensor coupled with a processing chain /may, by design, permit/ signal components, otherwise often considered as noise in other functions, to be recruited as a part of its proper function.  

I'm familiar with that approach. Pushed far enough, it can lead to telephone poles, acne, cigarette butts, and discarded beer cans being called noise. There was a long philosophical discussion on DPR about that. Were you part of it? I encouraged a much more narrow definition of noise. But I don't feel like rehashing the discussion here.

Meanwhile.  As much as I can understand the claims that the missing codes do not make a practical difference, I remain incensed at the very idea that Sony throws away good bits before giving me the package.  I feel somehow, *ahem* sure, that one day I will find cases where this makes a difference.  But meanwhile, I just want the bits.  Give me the bits, all the bits, and nothing but the bits.  I'll decide what I need and what I don't.

I'm with you. I want all those bits, too, but their absence won't keep me from buying and using the Sony a7x cameras. I've already gotten many excellent photographs from them that I'd have been unlikely to have made using other cameras.

(http://www.kasson.com/ll/_DSC4591-Edit.jpg)

(http://www.kasson.com/ll/_DSC5016-Edit-2.jpg)

(http://www.kasson.com/ll/_DSC0001-Editpartdehazed.jpg)

(http://www.kasson.com/ll/[Group 43]-_DSC6669__DSC6727-58 images_0000-Edit.jpg)

(http://www.kasson.com/ll/[Group 12]-_DSC8374 (3)__DSC8422 (3)-49 images_0001-Edit.jpg)

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on July 16, 2015, 08:00:09 pm
I'm familiar with that approach. Pushed far enough, it can lead to telephone poles, acne, cigarette butts, and discarded beer cans being called noise. There was a long philosophical discussion on DPR about that. Were you part of it? I encouraged a much more narrow definition of noise. But I don't feel like rehashing the discussion here.

Jim, here the teleological theory of noise came to your defense. 

Since you mentioned it, what would be wrong with the idea of phone poles, acne, cigarette buts, and discarded beer cans being called "noise"?  They can be considered noise at the appropriate level of functional architecture (human image-making), a level somewhat above the level of architecture you're thinking of (e.g., digital image sensors).
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 16, 2015, 08:03:41 pm
Jim, here the teleological theory of noise came to your defense. 

Since you mentioned it, what would be wrong with the idea of phone poles, acne, cigarette buts, and discarded beer cans being called "noise"?  They can be considered noise at the appropriate level of functional architecture (human image-making), a level somewhat above the level of architecture you're thinking of (e.g., digital image sensors).

They certainly can. A weed is a plant out of place.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: howgus on July 22, 2015, 04:35:35 pm
Try raising the levels in deep shadows.   You will see artifacts in Sony RAW images.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on July 22, 2015, 04:40:46 pm
Nope, that is BS. My Sony Alpha 99 runs circles around my P45+ back that has 16 bits data and no compression.

If you make a statement, please share a raw image showing it.

Best regards
Erik

Ps. Sony cameras may switch to 12-bit output on many obscure settings. That could cause some artefacts, may be…


Try raising the levels in deep shadows.   You will see artifacts in Sony RAW images.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: pegelli on July 22, 2015, 04:41:18 pm
Try raising the levels in deep shadows.   You will see artifacts in Sony RAW images.
Any examples to share?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on July 22, 2015, 07:02:56 pm
No update from Sony on their plan to deliver a fix for this limitation?

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on July 22, 2015, 07:56:13 pm
Try raising the levels in deep shadows.   You will see artifacts in Sony RAW images.

Deep shadows are unaffected by the tone curve in cRAW.

I've see a7x artifacts with really big shadow pushes, but they are not related to raw compression. Mostly column ADC effects, and heat from componentry around the edges of the light-sensitive area.

http://blog.kasson.com/?p=7681

http://blog.kasson.com/?p=7694

http://blog.kasson.com/?p=7709

Jim

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: rainer_v on July 25, 2015, 08:03:21 am
if i only would understand anything in this links is written..... ???
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: capital on August 06, 2015, 01:41:36 pm
I am trying to understand why electronic first curtain shutter release mode on the A7RII uses less bit depth than the focal plane shutter mode. Is there a technical issue, or ?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: ErikKaffehr on August 06, 2015, 02:15:29 pm
Hi,

Could be it may be related to speed of readout. A reduction in bit depth may allow for faster readout.

Keep in mind that this is all a compromise. CPU-power, heat generation, cooling and so on.

Best regards
Erik


I am trying to understand why electronic first curtain shutter release mode on the A7RII uses less bit depth than the focal plane shutter mode. Is there a technical issue, or ?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on August 06, 2015, 02:23:12 pm
Wouldn't the division from 14 bits to 11 bits and subsequent re-inflation to 14 bits will result in missing codes, even without consideration of the delta coding?

Yes it will. Even in the lowest part of the curve, half the codes are missing, and more as you go higher. The rationale is that shot noise will provide adequate dither. I buy that, for the most part.

http://blog.kasson.com/?p=6127

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on August 06, 2015, 02:29:13 pm
I am trying to understand why electronic first curtain shutter release mode on the A7RII uses less bit depth than the focal plane shutter mode. Is there a technical issue, or ?

I don't know that what you're saying is true. It's not true for the a7, a7S, and a7II. It is true for silent shutter in the a7S, and bulb can continuous modes in all three cameras.

How do you know that EFCS uses fewer bits of precision in the a7RII?

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: labirdman on August 06, 2015, 04:07:55 pm
RAW compression artifacts confirmed in A7RII shot with high contrast border and moderate exposure push, as expected. See http://www.dpreview.com/forums/thread/3883983, end of second page for analysis.

Bummer.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: capital on August 06, 2015, 05:44:22 pm
Is EFS the same as Silent Shutter? I guess I was thinking they were the same thing, one just being marketing speak.

I guess if they are different, okay, but why does Silent Shutter (not sure what the distinction is) lose bit depth?


Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on August 06, 2015, 05:46:23 pm
I am trying to understand why electronic first curtain shutter release mode on the A7RII uses less bit depth than the focal plane shutter mode. Is there a technical issue, or ?

That's just not true. I just ran a test and got identical precision, read noise, and EDR for EFCS on and off, in single shot mode.

http://blog.kasson.com/?p=11142

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on August 06, 2015, 06:08:26 pm
Is EFS the same as Silent Shutter? I guess I was thinking they were the same thing, one just being marketing speak.

I guess if they are different, okay, but why does Silent Shutter (not sure what the distinction is) lose bit depth?


In silent shutter mode, the reset signal occurs to start the exposure, but instead of ending the exposure with the second curtain and having all the time in the world to do the analog to digital conversion like in EFCS, the data is read out on the fly. It takes 1/30 second to read out the data with the a7S. I don't know about the a7RII. The Sony engineers do not habitually consult me, but my guess is that they are running the ramp twice as fast and the counters at the same speed in the single slope ADCs to that the conversions are twice as fast.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: capital on August 06, 2015, 06:28:07 pm
Thank you Jim for the nuts and volts discussion, regarding the increased frequency of the comparator's waveform. I am struck by the thought that the silent shutter operation could maintain the same precision, at the expense of reduced frames per second? Does Sony allow greater FPS in Silent Shutter mode as a consequence of this change in bit depth, or does this increased throughput go towards some other goal?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Jim Kasson on August 06, 2015, 06:30:46 pm
Thank you Jim for the nuts and volts discussion, regarding the increased frequency of the comparator's waveform. I am struck by the thought that the silent shutter operation could maintain the same precision, at the expense of reduced frames per second? Does Sony allow greater FPS in Silent Shutter mode as a consequence of this change in bit depth, or does this increased throughput go towards some other goal?

It would not be reduced fps that would be the casualty of slower readout, but a greater time to make one exposure. As it is, 1/30 second is too slow for many, who are used to, say, 1/250 second with mechanical focal plane shutters.

Jim
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: capital on August 06, 2015, 06:42:37 pm
The Sony's Silent Shutter is restricted to exposures of 1/30th of a second or slower?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: MatthewCromer on August 06, 2015, 06:57:44 pm
The Sony's Silent Shutter is restricted to exposures of 1/30th of a second or slower?

No, silent shutter can give shorter shutter speeds, but takes a total of 1/30 second to read the entire sensor.

This means that with faster shutter speeds it is exposing / reading only an increasingly narrow band of pixels at a time.

This is just like a focal plane shutter which, at speeds faster than the flash sync speed (1/160 is typical) is exposing a "band" of pixels at a time.

This can lead to visual artifacts, such as propeller warping.

https://www.google.com/search?q=focal+plane+shutter+distortion&espv=2&source=lnms&tbm=isch&sa=X
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 06, 2015, 08:51:25 pm
Obviously, everybody in this thread is smarter than me re. engineering, so as the designated Asperger may I ask the obvious question:

Has anybody actually tried engaging with Sony by walking up to one of their higher level japanese execs at a presentation?

My experience with japanese corporations is that the customer is God, and every bona-fide customer query that is registered will be listened to and relayed upstream, and there will be a careful response. However the query can only come in via a japanese employee, because a non-japanese employee does not (usually) participate in the collective decision process.

Edmund

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Chris Livsey on August 07, 2015, 03:25:10 am

My experience with japanese corporations is that the customer is God,


That recently, last 12-24 months at least, has not been the customer experience from Nikon. Thom Hogan has been collecting data on Nikon and their customer base and the conclusion is consistent: They are not listening or responding to us (main gripe is DX pro body, lack of (remember DX ? )
However one of the main Nikon customer complaints is "where is our mirror-less Sony/Fuji killer? " It would appear from the recent Nikon financials that the Nikon response, we don't care to get into that fight (at the moment), is fully justified despite the "noise" around Sony, and Fuji sometimes, here and elsewhere. 

For the full year Nikon expects to sell:

34% of all interchangeable lens cameras (slightly up from last year’s 33%)
30% of all lenses (slightly down from last year)
31% of all compact cameras


Against that Fuji results would seem to indicate they have 1% of the interchangeable lens camera market.

Source:
http://www.dslrbodies.com/newsviews/nikon-q1-financial-results.html
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: rdonson on August 07, 2015, 10:04:33 am
The larger the corporation the more likely it is that there will be several layers of MBAs making product and funding decisions.  If you'll recall Nikon's camera revival a few years ago it supposedly came as a result of firing a few layers of management.

My experience with Fuji is that they actively listen and seek out what their customers are saying about their cameras and lenses.  Its quite refreshing after 40+ years in the Canon camp.

Market share is of little value in this conversation.  For example, what market share does Leica have?  Does that influence how you perceive Leica?
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Wayne Fox on August 07, 2015, 01:23:39 pm
I am trying to understand why electronic first curtain shutter release mode on the A7RII uses less bit depth than the focal plane shutter mode. Is there a technical issue, or ?
I believe it’s silent shutter mode not front curtain mode that reduces the bit depth.  Silent shutter mode uses electronic first and second curtains.  According to Mark in another thread

Quote
Some shutter modes also make it revert to 12bit:

Silent Shooting
Long Exposure NR
Bulb (more than 30 sec. exposure)
Continuous Shooting (including Bracketing continuous shooting)
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: spidermike on August 07, 2015, 02:57:57 pm

However one of the main Nikon customer complaints is "where is our mirror-less Sony/Fuji killer? " [/i]

Source:
http://www.dslrbodies.com/newsviews/nikon-q1-financial-results.html

So precisely how many Nikon owners are actually calling for this? The internet is great at magnifying peoples' voices beyond all scale of their actual numbers and given the sales of DSLR and MFT (not the magazine bandwagons but actual sales) such a decision would be considered reasonable in any other area.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Chris Livsey on August 07, 2015, 05:32:35 pm
So precisely how many Nikon owners are actually calling for this? The internet is great at magnifying peoples' voices beyond all scale of their actual numbers and given the sales of DSLR and MFT (not the magazine bandwagons but actual sales) such a decision would be considered reasonable in any other area.

I refer you to the link I provided.
Thom Hogan has conducted a number of surveys of owners with large numbers participating and distilled the responses. The sample is of course skewed as this is web based but if you want numbers go there, or indeed some of the other Nikon-centric forums.
I did not say BTW that the decision was unreasonable, quite the contrary, given the sales figures of Nikon, and profits, they do not currently need to chase that small, currently, market segment.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on August 07, 2015, 05:32:59 pm
So precisely how many Nikon owners are actually calling for this? The internet is great at magnifying peoples' voices beyond all scale of their actual numbers and given the sales of DSLR and MFT (not the magazine bandwagons but actual sales) such a decision would be considered reasonable in any other area.

This is the downfall of any large company with an established product base to protect, and a huge opportunity for any up-and-coming company (or an old company having a revival).

Anyone remember Nokia? A huge company with a well-established product base, reduced to nothing because they missed the smartphone revolution while they were busy trying to sell people more and more of the same old thing (they made a few, feeble attempts - too little, too late). Or Kodak, reduced from being the big name in photography to a mere shell, subsisting mostly on a bunch of old patents? Or IBM, who missed the PC revolution because, in their myopia, they couldn't see a use for a computer in the home? Or, conversely, Apple's revival, from being a has-been in the 90s, with minimal market share and no-one buying their Macs (with all their software compatibility/availability issues), to becoming a dominant player firstly with the iPod (and, more importantly, iTunes), then the iPhone. And, more recently, Sony, whose digital imaging division has become the one bright spot in an otherwise-fading company?

There was no demand for cars before Henry Ford created one, and horses had a far larger share of the market than early cars...
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: capital on August 07, 2015, 06:42:22 pm
Thank you, Jim, Matthew and Wayne. I was reading some more about it, I found a short write up here:
http://caspegroup.com/How%20an%20electronic%20shutter%20works%20in%20a%20CMOS%20camera.pdf

So, it seems then that we can treat it like a scanning back, which trundles across the focal plane at a fixed pace, resetting, then reading out columns sequentially.

With that part clear, and knowing that they are prioritizing timeliness of column read outs, it seems like this is actually an artificial construct and not a hard limit. I don't see why they could not have this user assignable, as in allow prioritize for speed, versus prioritize for quantization.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: Chris Livsey on August 08, 2015, 03:54:23 am
There was no demand for cars before Henry Ford created one, and horses had a far larger share of the market than early cars...

Which is why Nikon customers are concerned and think the company is not listening. Do their numbers matter? Does it have to be a crowd or a minimum number who can see what is happening before Nikon wake up? ( they may be awake and working on it but don't want to say) They are realtime users not the "ambassadors" Nikon and other companies who have equivalents, who have already "made it" and form part of the PR, they have the best "horses" and also can't see why anyone would need a car, particularly a low "status" inexpensive one.
Those real users have seen the "car" and don't want another minor variant on the "horse". BTW I mentioned the D300 replacement, what a damp squib that will be now, unless they pull something out of the hat, maybe a mirror-less with pro build for DX glass?
Despite all this it is more likely the mass market will move even further away from "ILC cameras" as we know them to devices that take pictures and have connectivity  I think they call them mobile phones, tablets etc and the niche left for long glass and sports enabled high speed autofocus image capture will be filled by still frames from "video" cameras.

BTW I apologise for drifting too far from the thread topic, lets say: Why should Sony listen to a small number of customers ( and BTW "precisely how many Nikon " Sony "owners are actually calling for this? The internet is great at magnifying peoples' voices beyond all scale of their actual numbers." I think that is called a circular argument.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on August 11, 2015, 09:25:55 am
Which is why Nikon customers are concerned and think the company is not listening. Do their numbers matter? Does it have to be a crowd or a minimum number who can see what is happening before Nikon wake up? ( they may be awake and working on it but don't want to say) They are realtime users not the "ambassadors" Nikon and other companies who have equivalents, who have already "made it" and form part of the PR, they have the best "horses" and also can't see why anyone would need a car, particularly a low "status" inexpensive one.
Those real users have seen the "car" and don't want another minor variant on the "horse". BTW I mentioned the D300 replacement, what a damp squib that will be now, unless they pull something out of the hat, maybe a mirror-less with pro build for DX glass?
Despite all this it is more likely the mass market will move even further away from "ILC cameras" as we know them to devices that take pictures and have connectivity  I think they call them mobile phones, tablets etc and the niche left for long glass and sports enabled high speed autofocus image capture will be filled by still frames from "video" cameras.

BTW I apologise for drifting too far from the thread topic, lets say: Why should Sony listen to a small number of customers ( and BTW "precisely how many Nikon " Sony "owners are actually calling for this? The internet is great at magnifying peoples' voices beyond all scale of their actual numbers." I think that is called a circular argument.


That's exactly what I've been saying. Sony gave us the mirrorless 'car' and popularised it on a huge scale. Meanwhile, Canon and Nikon, instead of embracing the new technology and releasing their own competitive mirrorless products, continue to milk their current technology for all that it's worth and sell us more and more models of horse-and-buggy. Which will work for a time, until mirrorless matures and matches the D4s/1Dx in AF and the current crop of SLR lenses become due for replacement (or Sony finds a way to make them AF as fast on Sony bodies as they do on Canon/Nikon bodies, allowing people to switch systems at no cost).
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: LKaven on August 12, 2015, 06:35:33 pm
I'd hesitate to use the car <-> horse-and-buggy as an analogy to mirrorless ILCs and mirrorful ILCs.  The car analogy is a forced one in cameraland to begin with, and I don't feel that the mirrorless is so revolutionary as that.  They are different tool forms, each suited for a different style of working.  For the time being, they are each useful.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 12, 2015, 07:52:06 pm
I'd hesitate to use the car <-> horse-and-buggy as an analogy to mirrorless ILCs and mirrorful ILCs.  The car analogy is a forced one in cameraland to begin with, and I don't feel that the mirrorless is so revolutionary as that.  They are different tool forms, each suited for a different style of working.  For the time being, they are each useful.

Yes, yes, the SLR will be immortal like the rangefinder, Leica will continue to make them for another 50 years :)
Frankly, I prefer the SLR. But King Canute ...

Edmund
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on August 12, 2015, 08:02:26 pm
That's exactly what I've been saying. Sony gave us the mirrorless 'car' and popularised it on a huge scale. Meanwhile, Canon and Nikon, instead of embracing the new technology and releasing their own competitive mirrorless products, continue to milk their current technology for all that it's worth and sell us more and more models of horse-and-buggy. Which will work for a time, until mirrorless matures and matches the D4s/1Dx in AF and the current crop of SLR lenses become due for replacement (or Sony finds a way to make them AF as fast on Sony bodies as they do on Canon/Nikon bodies, allowing people to switch systems at no cost).

Well, Nikon does still have IMHO the best on sensor AF with the 1 series that I have first hand experience with, so they clearly have the core mirrorless technology figured out for quite come time. It is just a matter of deciding to apply it to larger sensors based cameras.

But they have also a very large user base that, to a certain extend, probably prefers OVF over EVF. Having tried the a7x, I find their finders usable but still offering a far worse viewing experience than OVF. I see the rush to those cameras, but to me the technology isn't mature quite yet.

So Nikon and Canon are facing a tough dilemma here. I think that's the reason why there isn't a D400 yet, Nikon keeps hesitating between delivering a boring, yet effective, 7DII killer or a more daring Samsung NX1 killer. They know that either way they choose, they will make hundreds of thousands of unhappy people. So they should probably release both versions and let photographers choose?

Now, would that make sense with existing lenses? Can they afford to develop a new mount and lenses? Probably not, which is why I still think that Nikon should adopt the FE mount for their mirrorless offering.

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 12, 2015, 08:13:49 pm
I think Nikon are going under. The market is deflating, year by year, there is no room for 3 companies at the top, and Nikon is the only one which has no video products of its own, and no large sensor design team, while Sony has a money pump with its sensor business, and Canon is a monster conglomerate that has cash and time.

Edmund

Well, Nikon does still have IMHO the best on sensor AF with the 1 series that I have first hand experience with, so they clearly have the core mirrorless technology figured out for quite come time. It is just a matter of deciding to apply it to larger sensors based cameras.

But they have also a very large user base that, to a certain extend, probably prefers OVF over EVF. Having tried the a7x, I find their finders usable but still offering a far worse viewing experience than OVF. I see the rush to those cameras, but to me the technology isn't mature quite yet.

So Nikon and Canon are facing a tough dilemma here. I think that's the reason why there isn't a D400 yet, Nikon keeps hesitating between delivering a boring, yet effective, 7DII killer or a more daring Samsung NX1 killer. They know that either way they choose, they will make hundreds of thousands of unhappy people. So they should probably release both versions and let photographers choose?

Now, would that make sense with existing lenses? Can they afford to develop a new mount and lenses? Probably not, which is why I still think that Nikon should adopt the FE mount for their mirrorless offering.

Cheers,
Bernard

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on August 12, 2015, 08:31:08 pm
I think Nikon are going under. The market is deflating, year by year, there is no room for 3 companies at the top, and Nikon is the only one which has no video products of its own, and no large sensor design team, while Sony has a money pump with its sensor business, and Canon is a monster conglomerate that has cash and time.

That is very possible. But for now Nikon still delivers the best solution on the market I feel. They have also been releasing for one year a constant stream of top level lenses, some offering unique combinations of features. Yes, all of those still work within the traditional DSLR framework.

I don't see why I should penalize myself by not using their products because of a technology trend that has not yet reached full maturity. Compactness is currently less important for me than absolute performance but I will see things differently when I'll turn 60 I am sure.

I'll switch to Sony the day they will help me capture images better than Nikon does. That will also mean they have lenses superior to those I am currently using.

This being said, I would buy a a7rII tomorrow as an addition to my main kit if it cost half it does, but no thank you at 430,000 yen. It is a great camera, but is over priced IMHO.

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 12, 2015, 09:24:45 pm
That is very possible. But for now Nikon still delivers the best solution on the market I feel. They have also been releasing for one year a constant stream of top level lenses, some offering unique combinations of features. Yes, all of those still work within the traditional DSLR framework.

I don't see why I should penalize myself by not using their products because of a technology trend that has not yet reached full maturity. Compactness is currently less important for me than absolute performance but I will see things differently when I'll turn 60 I am sure.

I'll switch to Sony the day they will help me capture images better than Nikon does. That will also mean they have lenses superior to those I am currently using.

This being said, I would buy a a7rII tomorrow as an addition to my main kit if it cost half it does, but no thank you at 430,000 yen. It is a great camera, but is over priced IMHO.

Cheers,
Bernard


Hugely overpriced in japan. It is at $3200 at B&H with the D810 at $3K.
Anyway, the kit you already own should probably see you through until you are 60 :)

Edmund
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on August 12, 2015, 10:04:00 pm
Hugely overpriced in japan. It is at $3200 at B&H with the D810 at $3K.

Yes, a new D810 goes for less than 280,000 in Tokyo compared to the Sony at 430,000... Don't know what they smoked at Sony Japan... the a7II is in stock everywhere btw.

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 12, 2015, 10:54:20 pm
Yes, a new D810 goes for less than 280,000 in Tokyo compared to the Sony at 430,000... Don't know what they smoked at Sony Japan... the a7II is in stock everywhere btw.

Cheers,
Bernard


Price for Sony gear always crashes when the new model comes out :)

Everything that is made in China is going to see its prices come down even sooner :)

Edmund

Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: stevesanacore on August 13, 2015, 12:38:55 am
That is very possible. But for now Nikon still delivers the best solution on the market I feel. They have also been releasing for one year a constant stream of top level lenses, some offering unique combinations of features. Yes, all of those still work within the traditional DSLR framework.

As long as you're not an architectural photographer. I was crazy about the Nikon D800E and loved the dynamic range, hoping they would come out with a new 24 and 17mm shift lenses to complement that fabulous sensor. But after waiting and waiting and waiting, Sony introduced a better solution. Now it's bye bye Nikon. A few years ago when digital really took over photography,  I told my assistants that one day we would be shooting with Sony and Panasonic cameras and they thought I was nuts..... Canon can compete with the big boys, but I'm not sure Nikon can.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: BernardLanguillier on August 13, 2015, 01:48:50 am
As long as you're not an architectural photographer. I was crazy about the Nikon D800E and loved the dynamic range, hoping they would come out with a new 24 and 17mm shift lenses to complement that fabulous sensor. But after waiting and waiting and waiting, Sony introduced a better solution. Now it's bye bye Nikon. A few years ago when digital really took over photography,  I told my assistants that one day we would be shooting with Sony and Panasonic cameras and they thought I was nuts..... Canon can compete with the big boys, but I'm not sure Nikon can.

Nikon has indeed been really slow reacting on the T/S lens front and I would probably have done the same had stitching not been my preferred solution for such images (I don't have productivity constraints).

This being said, I sincerely hope for you that they doesn't announce next gen T/S lenses in the coming months, but I am pretty sure they will... ;) Future will tell.

There are of course significant issue remaining in their lenses line up (macro lenses needing a refresh, 135mm f2.0 and the T/S of course), but they have done a good job recently at filling the gaps and most of their recent designs have been best in class. I am not too worried for them.

Cheers,
Bernard
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: hjulenissen on August 13, 2015, 01:50:00 am
That's exactly what I've been saying. Sony gave us the mirrorless 'car' and popularised it on a huge scale. Meanwhile, Canon and Nikon, instead of embracing the new technology and releasing their own competitive mirrorless products, continue to milk their current technology for all that it's worth and sell us more and more models of horse-and-buggy. Which will work for a time, until mirrorless matures and matches the D4s/1Dx in AF and the current crop of SLR lenses become due for replacement (or Sony finds a way to make them AF as fast on Sony bodies as they do on Canon/Nikon bodies, allowing people to switch systems at no cost).
For every horse vs car analogy, one can find counter analogies such as car vs hovering car, DVD vs Bluray, etc where the new technology did not catch on.

History does not support the idea that every new idea is "better" than older ideas. Mirrorless may well be the way to go today, or it may be the way to go in 10 years or it may never be. The answer to that lies in the decisions made by the manufacturers and the choices made by camera customers.

-h
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on August 13, 2015, 03:56:17 am
For every horse vs car analogy, one can find counter analogies such as car vs hovering car, DVD vs Bluray, etc where the new technology did not catch on.

History does not support the idea that every new idea is "better" than older ideas. Mirrorless may well be the way to go today, or it may be the way to go in 10 years or it may never be. The answer to that lies in the decisions made by the manufacturers and the choices made by camera customers.

-h

Blu-ray has caught on and largely replaced DVD for anyone who still buys physical copies of films.

Also, where's my hovercar? I want one...
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: shadowblade on August 13, 2015, 04:07:53 am
I'd hesitate to use the car <-> horse-and-buggy as an analogy to mirrorless ILCs and mirrorful ILCs.  The car analogy is a forced one in cameraland to begin with, and I don't feel that the mirrorless is so revolutionary as that.  They are different tool forms, each suited for a different style of working.  For the time being, they are each useful.

Early cars were slower than a horse.

Mirrorless is really quite revolutionary and opens up a lot of possibilities, once the technology becomes mature. High resolution and electronic shitters allow for simultaneous hi-res stills and 8k video shooting for action photography. Face recognition and fast processors also allow for much greater accuracy when shooting action. Constant recording and buffering allow for taking a shot after the event has actually taken place (2s buffer means you can press the shutter up to 2s after an action has taken place and still get the shot). It also allows for truly huge dynamic range - instead of taking a single 1s exposure, just use a firmware option to take 100 1/100s exposures and add up however many is needed to give correct exposure in each part of the image. Finally, the electronic nature of the display means that you are no longer physically tied to the camera, opening up remote tethering options for things like aerial and submersible drone photography.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: brianrybolt on August 13, 2015, 07:46:09 am
"electronic shitters"  That would be a revolutionary possibility!!
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: stevesanacore on August 13, 2015, 07:57:12 am
Nikon has indeed been really slow reacting on the T/S lens front and I would probably have done the same had stitching not been my preferred solution for such images (I don't have productivity constraints).

This being said, I sincerely hope for you that they doesn't announce next gen T/S lenses in the coming months, but I am pretty sure they will... ;) Future will tell.

There are of course significant issue remaining in their lenses line up (macro lenses needing a refresh, 135mm f2.0 and the T/S of course), but they have done a good job recently at filling the gaps and most of their recent designs have been best in class. I am not too worried for them.

Cheers,
Bernard

If they do, no problem, if they are better than the Canon versions, I'll just use them on my Sony along with all my other Canon, Leica and Nikon lenses I currently have. At this point after shooting with the A7R, I'm spoiled by the mirrorless design. The flawless live view is to me, a much better way to shoot with TS-E and any manual focus lenses. Having the digital viewfinder is amazing for focus in bright light outdoors and in no light at night.  I will miss the handling of the Nikon bodies, But the new A7R2 is a bit better than the previous version and having IBIS for all my lenses is a huge bonus for me. I do hope they update the firmware to eliminate the lossy compressed raw.... where this whole conversation started. Seems like a silly compromise to me.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: stevesanacore on August 13, 2015, 08:11:26 am
Early cars were slower than a horse.

Mirrorless is really quite revolutionary and opens up a lot of possibilities, once the technology becomes mature. High resolution and electronic shitters allow for simultaneous hi-res stills and 8k video shooting for action photography. Face recognition and fast processors also allow for much greater accuracy when shooting action. Constant recording and buffering allow for taking a shot after the event has actually taken place (2s buffer means you can press the shutter up to 2s after an action has taken place and still get the shot). It also allows for truly huge dynamic range - instead of taking a single 1s exposure, just use a firmware option to take 100 1/100s exposures and add up however many is needed to give correct exposure in each part of the image. Finally, the electronic nature of the display means that you are no longer physically tied to the camera, opening up remote tethering options for things like aerial and submersible drone photography.
.

You can shoot 8K raw motion now, or very soon, with the latest RED with some limitations, but it's not a practical substitute for a small quick handling DSLR for a still shooter. For my lifestyle work, the ability to roll 8k raw video and pick the perfect moment would be fantastic with a small camera like the Sony. I agree it's all coming one day as storage and energy issues resolve themselves. Saving 24 8k raw images for every second I shoot will require insane storage. And that's if we find 24 per second is enough ;-)



Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 13, 2015, 11:15:54 am
If I may be allowed to hijack the thread to a completely unrelated topic -


HAS ANYONE ACTUALLY APPROACHED SONY ABOUT UNCOMPRESSED RAW ?

Edmund
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: adias on August 13, 2015, 01:19:37 pm
If I may be allowed to hijack the thread to a completely unrelated topic -


HAS ANYONE ACTUALLY APPROACHED SONY ABOUT UNCOMPRESSED RAW ?

Edmund

I'm sure many did. I watched a B&H video with a factory rep stating that they are aware of the RAW compression objection. He stopped short of promising a change but he gave the impression that one would be offered. The question is when and for which models.
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 13, 2015, 01:42:51 pm
I'm sure many did. I watched a B&H video with a factory rep stating that they are aware of the RAW compression objection. He stopped short of promising a change but he gave the impression that one would be offered. The question is when and for which models.


OK - that gives us a good reason for waiting for the MODEL AFTER THE NEXT :)

The next model's electronics platform should be just about finalised about now, and it might not be possible to do any mods, but the model after the next is probably under development.

The same A7R2 sensor may though quite possibly have a Nikon mode, so you could get lossless from it in a Nikon, without the stabilisation and other addictive paraphernalia :)

Edmund
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: rdonson on August 13, 2015, 03:08:54 pm
If I may be allowed to hijack the thread to a completely unrelated topic -


HAS ANYONE ACTUALLY APPROACHED SONY ABOUT UNCOMPRESSED RAW ?

Edmund

http://www.dpreview.com/articles/0717419525/interview-kimio-maki-of-sony-the-customer-s-voice-is-the-most-important-data-for-me
Title: Re: How to convince Sony to do lossless raw on A7RII and others?
Post by: eronald on August 13, 2015, 03:54:09 pm
http://www.dpreview.com/articles/0717419525/interview-kimio-maki-of-sony-the-customer-s-voice-is-the-most-important-data-for-me

Keep whining until you win!

Edmund