Pages: [1] 2 3   Go Down

Author Topic: Proprietary RAW formats  (Read 8969 times)

Bryan Conner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 539
    • My Flickr page
Proprietary RAW formats
« on: May 01, 2012, 06:41:24 am »

I just finished watching the "Why Proprietary RAW?" video in the fabulous Introduction & Advanced Guide to Lightroom 4 video series.  Michael ranted about the fact that many camera manufacturers force the user to use their proprietary raw file format when using their camera.  I agree with Michael 100%.  I think that it is only a case of arrogance for camera manufacturers to do this.  There is no other good reason.  If you know of one, please educate me...I have an open mind.

I know that I do not use the disc that came with any of my cameras.  I think that it would make a statement to the Manufacturers if everyone mailed their software installation discs back to Canon, Nikon etc along with a not telling them that we are using dng and do not want their software.  Any opinions on this matter?
Logged

32BT

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3095
    • Pictures
Re: Proprietary RAW formats
« Reply #1 on: May 01, 2012, 07:37:30 am »

Arrogance? You know what is arrogant? To build a RAW converter based on illegal reverse engineering of files and a totally erroneous conversion process, and then trying to dictate the market into adopting some kind of bogus open raw format that fails to properly separate "data" from "processing".

So here are some counter arguments for discussion sake. In this case I will gladly play the devil's advocate.

1) Camera manufacturers first came out with camera's and a raw format, and third party raw converters only became available much later.

2) raw converters do not dictate raw format development.

3) DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure.

4) No need to exchange data with competitors

Logged
Regards,
~ O ~
If you can stomach it: pictures

32BT

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3095
    • Pictures
Re: Proprietary RAW formats
« Reply #2 on: May 01, 2012, 07:44:19 am »

5) obligation to manage and future proof intellectual property towards direct stakeholders, none of which include third-party raw developers…

6) DNG manages to be hopelessly incompatible even in between minor upgrades. (Direct result of not properly separating processing and data).

7) Very dumb evangelism tries to blame compatibility issues on camera manufacturers as opposed to for example the DNG development path.


Logged
Regards,
~ O ~
If you can stomach it: pictures

32BT

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3095
    • Pictures
Re: Proprietary RAW formats
« Reply #3 on: May 01, 2012, 07:49:35 am »

btw it reminds me very much of the whole ICC standardization.

If you know what kind of effort went into that, you may understand that just proposing a standard and submitting it to a standardization committee is not going to cut it. And as usual with standards involving too many stakeholders, it eventually bloats and stalls.
Logged
Regards,
~ O ~
If you can stomach it: pictures

Bryan Conner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 539
    • My Flickr page
Re: Proprietary RAW formats
« Reply #4 on: May 01, 2012, 07:57:59 am »

Arrogance? You know what is arrogant? To build a RAW converter based on illegal reverse engineering of files and a totally erroneous conversion process, and then trying to dictate the market into adopting some kind of bogus open raw format that fails to properly separate "data" from "processing".

So here are some counter arguments for discussion sake. In this case I will gladly play the devil's advocate.

1) Camera manufacturers first came out with camera's and a raw format, and third party raw converters only became available much later.

2) raw converters do not dictate raw format development.

3) DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure.

4) No need to exchange data with competitors



Michael had a good point that I had not thought of, and I will ask you the question:  Do you consider yourself to be the owner of your image once you press the shutter button?  If so, should you be able to use whatever tool you wish to use on your image?  Or, should you be restricted to the Camera Manufacturers software?

Your second statement was "raw converters do not dictate raw format development".  Who exactly is the raw format development dictator?

"DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure."  Could you please explain this statement?

"No need to exchange data with competitors"    What is there to exchange?  If it is so easy to reverse engineer in order to develop a top quality raw processor, then the author of the raw format has not done a very good job of hiding this "secret information".  AND, if the camera manufacturers really wanted to keep the customer using the proprietary software, then why do they not do a better job of keeping up with the technology that is currently available?  In my opinion, DPP is nowhere near the Raw Converter that Lightroom/ACR/ or C1.  

Do you shoot raw?  If so, which converter do you use? And why?  

Logged

32BT

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3095
    • Pictures
Re: Proprietary RAW formats
« Reply #5 on: May 01, 2012, 08:27:10 am »

Michael had a good point that I had not thought of, and I will ask you the question:  Do you consider yourself to be the owner of your image once you press the shutter button?  If so, should you be able to use whatever tool you wish to use on your image?  Or, should you be restricted to the Camera Manufacturers software?

Do you mean "am I able" or "should it be possible"?

Do you consider yourself the owner of your car? Do you go to a third-party car-workshop because the car manufacturer tells you their car is an open format, go wherever the h*ll you please and we still provide you with all the benefits and requirements of guarantees and obligations on our part?

Did you sign a mutual contract with the camera manufacturer that specifically states that they will make available RAW data to you that is yours and yours to do however you please? Or did you just purchase a camera which produces JPG which is a ubiquitous format and it can also do RAW?

(remember, I'm just playing the devil's advocate here, don't take it out on me).

Your second statement was "raw converters do not dictate raw format development".  Who exactly is the raw format development dictator?

I don't understand "dictator" in this context. But if I for example want to introduce a new Color Filter Array as did Fuji recently, should they somehow postpone introduction until all third-party converters have implemented a reasonable conversion and support?

"DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure."  Could you please explain this statement?

DNG fails to separate processing from data. An open RAW format should simply be designed as a container for the RAW data and meta data.
However, DNG seems to be developed from the processing paradigm at Adobe. One glaring example would be the introduction of dual dng profiles. First of all there was a perfectly good color profile standard available in the form of ICC profiles. Second there is absolutely no reason to introduce something like dual profiles where only one response will do fine from a color management point of view. And behind this idea is an entirely incorrect or false processing paradigm based on incorrect application of basic color science. (introduction of temperature and tint).

Another example would be the current lens profiling options. There is a significant difference between the idea of profiling a lens vs introducing formulas for distortion processing.


"No need to exchange data with competitors"    What is there to exchange?  If it is so easy to reverse engineer in order to develop a top quality raw processor, then the author of the raw format has not done a very good job of hiding this "secret information".  

How do you know whether it is easy to reverse engineer the data? How do you know they have reversed engineered *ALL* the data?

AND, if the camera manufacturers really wanted to keep the customer using the proprietary software, then why do they not do a better job of keeping up with the technology that is currently available?  In my opinion, DPP is nowhere near the Raw Converter that Lightroom/ACR/ or C1.

Do know if they want that, but I also wonder how prohibitive it is for a commercial company to have to license the Adobe RAW converter to sell with your cameras and also keep a timely product development cycle, plus the dependence on a third-party etc. etc..

It may also be that camera manufacturers generally tend to be hardware manufacturers, not software manufacturers, however, I do not read any arguments here why they should adopt an open RAW format and stop providing their own solution.

Do you shoot raw?  If so, which converter do you use? And why?  

Don't shoot the messenger.

Do you own a car? If so, what garage do you go to for periodic check up, and why?
Logged
Regards,
~ O ~
If you can stomach it: pictures

deejjjaaaa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1170
Re: Proprietary RAW formats
« Reply #6 on: May 01, 2012, 10:07:08 am »

Any opinions on this matter?

Should Panasonic wait with their software optics correction until Adobe modifies the DNG standard ? why ? for how long ?
Logged

BJL

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6600
Proprietary RAW formats are widely readable
« Reply #7 on: May 01, 2012, 10:40:05 am »

If so, should you be able to use whatever tool you wish to use on your image?  Or, should you be restricted to the Camera Manufacturers software?
I do not know of an camera whose raw format restricts to using the camera manufacturer's software. As fas as I know, Photoshop, Lightroom, Aperture,and numerous other software options handle almost all raw formats, with at worst a delay after the release of a new camera model. Heck, even the free Preview app in Mac OS can open raw files of almost every flavor.

Adobe sometimes has a twisted, self-serving idea of an "open standard": a new format devised by a single company without significant input from any other major stakeholders, and in which openness at one end goes with Adobe having control over developments in that format, and thus a natural advantage over competitors in developing software to work with the format. Like Adobe/Macromedia Flash, which is worshiped by some as "open" but is effectively controlled and dominated by Adobe at the money making end: have you seen any good commercial rivals to Adobe's software for creating Flash content?

If instead camera makers worked together to produce a raw format standard through some organization like JPEG or IEEE or  ANSI or ISO, or even if one company developed a draft format specification and then proposed it for revision and standardization by some such industry-wide organization (e.g. Apple with Firewire, which became IEEE1394) I would be more enthusiastic.
Logged

john beardsworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4755
    • My photography site
Re: Proprietary RAW formats
« Reply #8 on: May 01, 2012, 11:10:29 am »

I just finished watching the "Why Proprietary RAW?" video in the fabulous Introduction & Advanced Guide to Lightroom 4 video series.  Michael ranted about the fact that many camera manufacturers force the user to use their proprietary raw file format when using their camera.  I agree with Michael 100%.  I think that it is only a case of arrogance for camera manufacturers to do this.  There is no other good reason.  If you know of one, please educate me...I have an open mind.

I know that I do not use the disc that came with any of my cameras.  I think that it would make a statement to the Manufacturers if everyone mailed their software installation discs back to Canon, Nikon etc along with a not telling them that we are using dng and do not want their software.  Any opinions on this matter?

You make a good point. Almost every time a new camera comes out, you hear the moans directed at Adobe, Apple and others for not supporting a new proprietary format - it's [some of] the camera makers who should be on the receiving end.
« Last Edit: May 01, 2012, 11:24:40 am by johnbeardy »
Logged

Thomas Krüger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 447
    • http://thomaskrueger.eu
Re: Proprietary RAW formats
« Reply #9 on: May 01, 2012, 11:17:53 am »

Anybody remember OpenRAW at http://www.openraw.org/news/index.html
Logged

Bryan Conner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 539
    • My Flickr page
Re: Proprietary RAW formats are widely readable
« Reply #10 on: May 01, 2012, 11:36:59 am »

I do not know of an camera whose raw format restricts to using the camera manufacturer's software.

When a camera manufacturer introduces a new camera along with a "new", or "revised" raw file, Lightroom, ACR, C1 can not open it in the beginning.  In the beginning you are definitely restricted to the manufacturers software if you want to use raw.

I agree with you 100% about everyone working together.  But, as long as the bottom line is money, the happiness of the customer is going to always be second at best.
Logged

Bryan Conner

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 539
    • My Flickr page
Re: Proprietary RAW formats
« Reply #11 on: May 01, 2012, 12:22:33 pm »

Do you mean "am I able" or "should it be possible"?

If you are able, then it is possible.  Without 3rd party software, there are times immediately after a new camera release when a raw file can only be adjusted using the camera manufacturer's software.  In this case, you are not able because it is not possible.

Why would a camera manufacturer not want it's customers to have access to the means to produce the best image?

Do you consider yourself the owner of your car? Do you go to a third-party car-workshop because the car manufacturer tells you their car is an open format, go wherever the h*ll you please and we still provide you with all the benefits and requirements of guarantees and obligations on our part?

Car manufacturers will not void your warranty based on the brand of the tools used to work on the car.  We are talking about the digital file, not the camera.  If I sent my Canon camera to Nikon to be repaired, I would not expect my warranty to be honored.

Did you sign a mutual contract with the camera manufacturer that specifically states that they will make available RAW data to you that is yours and yours to do however you please? Or did you just purchase a camera which produces JPG which is a ubiquitous format and it can also do RAW?


No, I did not.  But, at the same time I think that there is a reason why Camera Manufacturers don't do this.  They are not stupid.  If Canon made it so where customers could not use 3rd party raw converters, the companies that allow their customers to use 3rd party converters would immediately have an advantage:  better looking images.  Pro and amateur photographers would abandon Canon in droves.  The average human knows the word Photoshop and associates that with digital images, how many know what you are talking about when you say DPP?  I purchase a camera based on the image quality that is possible to achieve from the particular camera.  If I am limited to a lower possible quality then I will look for an alternative.  Photoshop/Lightroom/C1 etc are that alternative.

(remember, I'm just playing the devil's advocate here, don't take it out on me).
  No worries.  I have an open mind and hope to learn something from this exchange.  So, give me a benefit to the customer for the camera manufacturers not to embrace a universal raw format?

I don't understand "dictator" in this context. But if I for example want to introduce a new Color Filter Array as did Fuji recently, should they somehow postpone introduction until all third-party converters have implemented a reasonable conversion and support?
  Why would there be a need to delay production?  The third party converter that is the first to respond with the goods will reap the benefits from the market.  You stated that "raw converters do not dictate raw format development". If raw converters do not dictate, then that means that another party is dictating...this means that there is a dictator.

DNG fails to separate processing from data. An open RAW format should simply be designed as a container for the RAW data and meta data.
  I was not aware that DNG fails in this aspect.  If it does, I am in agreement with you on this.   

However, DNG seems to be developed from the processing paradigm at Adobe. One glaring example would be the introduction of dual dng profiles. First of all there was a perfectly good color profile standard available in the form of ICC profiles. Second there is absolutely no reason to introduce something like dual profiles where only one response will do fine from a color management point of view. And behind this idea is an entirely incorrect or false processing paradigm based on incorrect application of basic color science. (introduction of temperature and tint).

Another example would be the current lens profiling options. There is a significant difference between the idea of profiling a lens vs introducing formulas for distortion processing.
I am not knowledgeable enough in this area to form a valid opinion.  I will have to research this first.  Can you provide me with a source of documentation?


How do you know whether it is easy to reverse engineer the data? How do you know they have reversed engineered *ALL* the data?
  I do not know that they have reversed engineered ALL of the data.  Apparently they have reversed engineered enough in order to produce an arguably higher quality output.

Do know if they want that, but I also wonder how prohibitive it is for a commercial company to have to license the Adobe RAW converter to sell with your cameras and also keep a timely product development cycle, plus the dependence on a third-party etc. etc..
  There is no need to depend on a 3rd party convertor, the manufacturer could also use an open raw format.  Note that I did not say dng.  I  dont care if it is dng or not.  Just a generic container for the info.  At the moment, dng is the most prominent.  Is there another alternative.  Heck, I don't care if we use .kdc format as it seems that kodak will not be using it..LOL.

It may also be that camera manufacturers generally tend to be hardware manufacturers, not software manufacturers, however, I do not read any arguments here why they should adopt an open RAW format and stop providing their own solution.
They do not have to stop providing their own solution.  But, how many people buy a Canon camera just so they can use DPP or how many Nikon user bought their Nikon just so they could use Capture NX?

Don't shoot the messenger.
  No danger.  My opinion is not changed as of yet.

Do you own a car? If so, what garage do you go to for periodic check up, and why?
  Yes, I own a car.  Here in Germany, we must (by law) take our cars to the local TÜV garage once every two years for a very thorough inspection.  But, I keep an eye on it myself.  My father was a mechanic by trade and I grew up helping him in his garage. 

Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Proprietary RAW formats
« Reply #12 on: May 01, 2012, 02:50:54 pm »

The camera manufactures allow us to capture into an open format, JPEG. So the idea that raw is somehow different doesn’t wash. And if they want to keep all the proprietary stuff, fine. All they have to do is build in a 2nd switch: save as DNG. Just as we have a switch to produce a JPEG. They, the manufacturers can have their proprietary raw data and build in all the secret sauce they want and keep what they feel is a competitive advantage. You want that, use their raw converter. You want JPEG, fine. But you want a non rendered, non demosaiced raw file that works the day the camera ships with 3rd party converters, you ask to save DNG. Simple.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Re: Proprietary RAW formats
« Reply #13 on: May 01, 2012, 02:54:12 pm »

+1

Erik

The camera manufactures allow us to capture into an open format, JPEG. So the idea that raw is somehow different doesn’t wash. And if they want to keep all the proprietary stuff, fine. All they have to do is build in a 2nd switch: save as DNG. Just as we have a switch to produce a JPEG. They, the manufacturers can have their proprietary raw data and build in all the secret sauce they want and keep what they feel is a competitive advantage. You want that, use their raw converter. You want JPEG, fine. But you want a non rendered, non demosaiced raw file that works the day the camera ships with 3rd party converters, you ask to save DNG. Simple.
Logged
Erik Kaffehr
 

alain

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 465
Re: Proprietary RAW formats
« Reply #14 on: May 01, 2012, 03:44:53 pm »

But you want a non rendered, non demosaiced raw file that works the day the camera ships with 3rd party converters, you ask to save DNG. Simple.

This is an illusion, it will work, but it won't be usable for most people.  For this to work it would mean that 3rd party converters would be able to give a pleasing image without having used the camera, aka shot a number of carefully chosen test charts or test images to "calibrate" the camera.   I quite sure this "calibrating" is mostly a lot more work that "decoding" the RAW file from model X from a known supplier.  It's also a lot easier to get some RAW files to decode before "launch" than a real camera.

I don't have any doubts that you can "calibrate" you're new camera, but the average photographer won't.   
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Proprietary RAW formats
« Reply #15 on: May 01, 2012, 03:54:45 pm »

This is an illusion, it will work, but it won't be usable for most people.  For this to work it would mean that 3rd party converters would be able to give a pleasing image without having used the camera, aka shot a number of carefully chosen test charts or test images to "calibrate" the camera.

I don’t see how it is an illusion or how it will not work.

If the DNG converter, LR, or any other method of converting proprietary raw to DNG can do this, so can the camera. The rendering afterwards is totally moot. And don’t a few Nikon and Canon competitions do exactly what I suggested, save off a DNG? Ricoh has been doing it since 2005. Pentax and Leica has been doing it since 2006!

There is absolutely nothing technology difficult here, it is solely a political issue from the likes of Nikon and Canon. They could provide a TIFF instead of a JPEG if they wanted to. The could provide a TIFF, JPEG, DNG or proprietary raw if they wanted to.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

alain

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 465
Re: Proprietary RAW formats
« Reply #16 on: May 01, 2012, 04:21:25 pm »

I don’t see how it is an illusion or how it will not work.

... The rendering afterwards is totally moot.
...

Well for most users it isn't.  As long as the rendering isn't good it useless for most users.  If I made a "good" RAW converter I even would block the new camera's until I could give it a good rendering, far better than temporally offering a bad rendering. 

The decoding of the RAW file isn't the big problem.  (Well unless a camera maker would block external raw converters on purpose with strong encryption, but that wouldn't be smart for a "normal" camera with RAW possibilities). 
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Proprietary RAW formats
« Reply #17 on: May 01, 2012, 04:30:05 pm »

Well for most users it isn't.  As long as the rendering isn't good it useless for most users. 

Why would the rendering be poor? It isn’t now with the current products that can render a DNG.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

alain

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 465
Re: Proprietary RAW formats
« Reply #18 on: May 01, 2012, 04:36:57 pm »

Why would the rendering be poor? It isn’t now with the current products that can render a DNG.

See my original message.  It can made to be good with either the own RAW format or DNG (if that's the camera's own RAW format), but it needs shooting with the camera. 
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Proprietary RAW formats
« Reply #19 on: May 01, 2012, 04:39:39 pm »

See my original message.  It can made to be good with either the own RAW format or DNG (if that's the camera's own RAW format), but it needs shooting with the camera. 

Makes no sense, doesn’t wash with the various camera systems that have been saving out DNG’s for 6+ years.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".
Pages: [1] 2 3   Go Up