Luminous Landscape Forum

The Art of Photography => The Coffee Corner => Topic started by: Bryan Conner on May 01, 2012, 06:41:24 am

Title: Proprietary RAW formats
Post by: Bryan Conner on May 01, 2012, 06:41:24 am
I just finished watching the "Why Proprietary RAW?" video in the fabulous Introduction & Advanced Guide to Lightroom 4 video series.  Michael ranted about the fact that many camera manufacturers force the user to use their proprietary raw file format when using their camera.  I agree with Michael 100%.  I think that it is only a case of arrogance for camera manufacturers to do this.  There is no other good reason.  If you know of one, please educate me...I have an open mind.

I know that I do not use the disc that came with any of my cameras.  I think that it would make a statement to the Manufacturers if everyone mailed their software installation discs back to Canon, Nikon etc along with a not telling them that we are using dng and do not want their software.  Any opinions on this matter?
Title: Re: Proprietary RAW formats
Post by: 32BT on May 01, 2012, 07:37:30 am
Arrogance? You know what is arrogant? To build a RAW converter based on illegal reverse engineering of files and a totally erroneous conversion process, and then trying to dictate the market into adopting some kind of bogus open raw format that fails to properly separate "data" from "processing".

So here are some counter arguments for discussion sake. In this case I will gladly play the devil's advocate.

1) Camera manufacturers first came out with camera's and a raw format, and third party raw converters only became available much later.

2) raw converters do not dictate raw format development.

3) DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure.

4) No need to exchange data with competitors

Title: Re: Proprietary RAW formats
Post by: 32BT on May 01, 2012, 07:44:19 am
5) obligation to manage and future proof intellectual property towards direct stakeholders, none of which include third-party raw developers…

6) DNG manages to be hopelessly incompatible even in between minor upgrades. (Direct result of not properly separating processing and data).

7) Very dumb evangelism tries to blame compatibility issues on camera manufacturers as opposed to for example the DNG development path.


Title: Re: Proprietary RAW formats
Post by: 32BT on May 01, 2012, 07:49:35 am
btw it reminds me very much of the whole ICC standardization.

If you know what kind of effort went into that, you may understand that just proposing a standard and submitting it to a standardization committee is not going to cut it. And as usual with standards involving too many stakeholders, it eventually bloats and stalls.
Title: Re: Proprietary RAW formats
Post by: Bryan Conner on May 01, 2012, 07:57:59 am
Arrogance? You know what is arrogant? To build a RAW converter based on illegal reverse engineering of files and a totally erroneous conversion process, and then trying to dictate the market into adopting some kind of bogus open raw format that fails to properly separate "data" from "processing".

So here are some counter arguments for discussion sake. In this case I will gladly play the devil's advocate.

1) Camera manufacturers first came out with camera's and a raw format, and third party raw converters only became available much later.

2) raw converters do not dictate raw format development.

3) DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure.

4) No need to exchange data with competitors



Michael had a good point that I had not thought of, and I will ask you the question:  Do you consider yourself to be the owner of your image once you press the shutter button?  If so, should you be able to use whatever tool you wish to use on your image?  Or, should you be restricted to the Camera Manufacturers software?

Your second statement was "raw converters do not dictate raw format development".  Who exactly is the raw format development dictator?

"DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure."  Could you please explain this statement?

"No need to exchange data with competitors"    What is there to exchange?  If it is so easy to reverse engineer in order to develop a top quality raw processor, then the author of the raw format has not done a very good job of hiding this "secret information".  AND, if the camera manufacturers really wanted to keep the customer using the proprietary software, then why do they not do a better job of keeping up with the technology that is currently available?  In my opinion, DPP is nowhere near the Raw Converter that Lightroom/ACR/ or C1.  

Do you shoot raw?  If so, which converter do you use? And why?  

Title: Re: Proprietary RAW formats
Post by: 32BT on May 01, 2012, 08:27:10 am
Michael had a good point that I had not thought of, and I will ask you the question:  Do you consider yourself to be the owner of your image once you press the shutter button?  If so, should you be able to use whatever tool you wish to use on your image?  Or, should you be restricted to the Camera Manufacturers software?

Do you mean "am I able" or "should it be possible"?

Do you consider yourself the owner of your car? Do you go to a third-party car-workshop because the car manufacturer tells you their car is an open format, go wherever the h*ll you please and we still provide you with all the benefits and requirements of guarantees and obligations on our part?

Did you sign a mutual contract with the camera manufacturer that specifically states that they will make available RAW data to you that is yours and yours to do however you please? Or did you just purchase a camera which produces JPG which is a ubiquitous format and it can also do RAW?

(remember, I'm just playing the devil's advocate here, don't take it out on me).

Your second statement was "raw converters do not dictate raw format development".  Who exactly is the raw format development dictator?

I don't understand "dictator" in this context. But if I for example want to introduce a new Color Filter Array as did Fuji recently, should they somehow postpone introduction until all third-party converters have implemented a reasonable conversion and support?

"DNG as a RAW format has an incorrect processing paradigm backed in which is an inherent failure."  Could you please explain this statement?

DNG fails to separate processing from data. An open RAW format should simply be designed as a container for the RAW data and meta data.
However, DNG seems to be developed from the processing paradigm at Adobe. One glaring example would be the introduction of dual dng profiles. First of all there was a perfectly good color profile standard available in the form of ICC profiles. Second there is absolutely no reason to introduce something like dual profiles where only one response will do fine from a color management point of view. And behind this idea is an entirely incorrect or false processing paradigm based on incorrect application of basic color science. (introduction of temperature and tint).

Another example would be the current lens profiling options. There is a significant difference between the idea of profiling a lens vs introducing formulas for distortion processing.


"No need to exchange data with competitors"    What is there to exchange?  If it is so easy to reverse engineer in order to develop a top quality raw processor, then the author of the raw format has not done a very good job of hiding this "secret information".  

How do you know whether it is easy to reverse engineer the data? How do you know they have reversed engineered *ALL* the data?

AND, if the camera manufacturers really wanted to keep the customer using the proprietary software, then why do they not do a better job of keeping up with the technology that is currently available?  In my opinion, DPP is nowhere near the Raw Converter that Lightroom/ACR/ or C1.

Do know if they want that, but I also wonder how prohibitive it is for a commercial company to have to license the Adobe RAW converter to sell with your cameras and also keep a timely product development cycle, plus the dependence on a third-party etc. etc..

It may also be that camera manufacturers generally tend to be hardware manufacturers, not software manufacturers, however, I do not read any arguments here why they should adopt an open RAW format and stop providing their own solution.

Do you shoot raw?  If so, which converter do you use? And why?  

Don't shoot the messenger.

Do you own a car? If so, what garage do you go to for periodic check up, and why?
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 01, 2012, 10:07:08 am
Any opinions on this matter?

Should Panasonic wait with their software optics correction until Adobe modifies the DNG standard ? why ? for how long ?
Title: Proprietary RAW formats are widely readable
Post by: BJL on May 01, 2012, 10:40:05 am
If so, should you be able to use whatever tool you wish to use on your image?  Or, should you be restricted to the Camera Manufacturers software?
I do not know of an camera whose raw format restricts to using the camera manufacturer's software. As fas as I know, Photoshop, Lightroom, Aperture,and numerous other software options handle almost all raw formats, with at worst a delay after the release of a new camera model. Heck, even the free Preview app in Mac OS can open raw files of almost every flavor.

Adobe sometimes has a twisted, self-serving idea of an "open standard": a new format devised by a single company without significant input from any other major stakeholders, and in which openness at one end goes with Adobe having control over developments in that format, and thus a natural advantage over competitors in developing software to work with the format. Like Adobe/Macromedia Flash, which is worshiped by some as "open" but is effectively controlled and dominated by Adobe at the money making end: have you seen any good commercial rivals to Adobe's software for creating Flash content?

If instead camera makers worked together to produce a raw format standard through some organization like JPEG or IEEE or  ANSI or ISO, or even if one company developed a draft format specification and then proposed it for revision and standardization by some such industry-wide organization (e.g. Apple with Firewire, which became IEEE1394) I would be more enthusiastic.
Title: Re: Proprietary RAW formats
Post by: john beardsworth on May 01, 2012, 11:10:29 am
I just finished watching the "Why Proprietary RAW?" video in the fabulous Introduction & Advanced Guide to Lightroom 4 video series.  Michael ranted about the fact that many camera manufacturers force the user to use their proprietary raw file format when using their camera.  I agree with Michael 100%.  I think that it is only a case of arrogance for camera manufacturers to do this.  There is no other good reason.  If you know of one, please educate me...I have an open mind.

I know that I do not use the disc that came with any of my cameras.  I think that it would make a statement to the Manufacturers if everyone mailed their software installation discs back to Canon, Nikon etc along with a not telling them that we are using dng and do not want their software.  Any opinions on this matter?

You make a good point. Almost every time a new camera comes out, you hear the moans directed at Adobe, Apple and others for not supporting a new proprietary format - it's [some of] the camera makers who should be on the receiving end.
Title: Re: Proprietary RAW formats
Post by: Thomas Krüger on May 01, 2012, 11:17:53 am
Anybody remember OpenRAW at http://www.openraw.org/news/index.html
Title: Re: Proprietary RAW formats are widely readable
Post by: Bryan Conner on May 01, 2012, 11:36:59 am
I do not know of an camera whose raw format restricts to using the camera manufacturer's software.

When a camera manufacturer introduces a new camera along with a "new", or "revised" raw file, Lightroom, ACR, C1 can not open it in the beginning.  In the beginning you are definitely restricted to the manufacturers software if you want to use raw.

I agree with you 100% about everyone working together.  But, as long as the bottom line is money, the happiness of the customer is going to always be second at best.
Title: Re: Proprietary RAW formats
Post by: Bryan Conner on May 01, 2012, 12:22:33 pm
Do you mean "am I able" or "should it be possible"?

If you are able, then it is possible.  Without 3rd party software, there are times immediately after a new camera release when a raw file can only be adjusted using the camera manufacturer's software.  In this case, you are not able because it is not possible.

Why would a camera manufacturer not want it's customers to have access to the means to produce the best image?

Do you consider yourself the owner of your car? Do you go to a third-party car-workshop because the car manufacturer tells you their car is an open format, go wherever the h*ll you please and we still provide you with all the benefits and requirements of guarantees and obligations on our part?

Car manufacturers will not void your warranty based on the brand of the tools used to work on the car.  We are talking about the digital file, not the camera.  If I sent my Canon camera to Nikon to be repaired, I would not expect my warranty to be honored.

Did you sign a mutual contract with the camera manufacturer that specifically states that they will make available RAW data to you that is yours and yours to do however you please? Or did you just purchase a camera which produces JPG which is a ubiquitous format and it can also do RAW?


No, I did not.  But, at the same time I think that there is a reason why Camera Manufacturers don't do this.  They are not stupid.  If Canon made it so where customers could not use 3rd party raw converters, the companies that allow their customers to use 3rd party converters would immediately have an advantage:  better looking images.  Pro and amateur photographers would abandon Canon in droves.  The average human knows the word Photoshop and associates that with digital images, how many know what you are talking about when you say DPP?  I purchase a camera based on the image quality that is possible to achieve from the particular camera.  If I am limited to a lower possible quality then I will look for an alternative.  Photoshop/Lightroom/C1 etc are that alternative.

(remember, I'm just playing the devil's advocate here, don't take it out on me).
  No worries.  I have an open mind and hope to learn something from this exchange.  So, give me a benefit to the customer for the camera manufacturers not to embrace a universal raw format?

I don't understand "dictator" in this context. But if I for example want to introduce a new Color Filter Array as did Fuji recently, should they somehow postpone introduction until all third-party converters have implemented a reasonable conversion and support?
  Why would there be a need to delay production?  The third party converter that is the first to respond with the goods will reap the benefits from the market.  You stated that "raw converters do not dictate raw format development". If raw converters do not dictate, then that means that another party is dictating...this means that there is a dictator.

DNG fails to separate processing from data. An open RAW format should simply be designed as a container for the RAW data and meta data.
  I was not aware that DNG fails in this aspect.  If it does, I am in agreement with you on this.   

However, DNG seems to be developed from the processing paradigm at Adobe. One glaring example would be the introduction of dual dng profiles. First of all there was a perfectly good color profile standard available in the form of ICC profiles. Second there is absolutely no reason to introduce something like dual profiles where only one response will do fine from a color management point of view. And behind this idea is an entirely incorrect or false processing paradigm based on incorrect application of basic color science. (introduction of temperature and tint).

Another example would be the current lens profiling options. There is a significant difference between the idea of profiling a lens vs introducing formulas for distortion processing.
I am not knowledgeable enough in this area to form a valid opinion.  I will have to research this first.  Can you provide me with a source of documentation?


How do you know whether it is easy to reverse engineer the data? How do you know they have reversed engineered *ALL* the data?
  I do not know that they have reversed engineered ALL of the data.  Apparently they have reversed engineered enough in order to produce an arguably higher quality output.

Do know if they want that, but I also wonder how prohibitive it is for a commercial company to have to license the Adobe RAW converter to sell with your cameras and also keep a timely product development cycle, plus the dependence on a third-party etc. etc..
  There is no need to depend on a 3rd party convertor, the manufacturer could also use an open raw format.  Note that I did not say dng.  I  dont care if it is dng or not.  Just a generic container for the info.  At the moment, dng is the most prominent.  Is there another alternative.  Heck, I don't care if we use .kdc format as it seems that kodak will not be using it..LOL.

It may also be that camera manufacturers generally tend to be hardware manufacturers, not software manufacturers, however, I do not read any arguments here why they should adopt an open RAW format and stop providing their own solution.
They do not have to stop providing their own solution.  But, how many people buy a Canon camera just so they can use DPP or how many Nikon user bought their Nikon just so they could use Capture NX?

Don't shoot the messenger.
  No danger.  My opinion is not changed as of yet.

Do you own a car? If so, what garage do you go to for periodic check up, and why?
  Yes, I own a car.  Here in Germany, we must (by law) take our cars to the local TÜV garage once every two years for a very thorough inspection.  But, I keep an eye on it myself.  My father was a mechanic by trade and I grew up helping him in his garage. 

Title: Re: Proprietary RAW formats
Post by: digitaldog on May 01, 2012, 02:50:54 pm
The camera manufactures allow us to capture into an open format, JPEG. So the idea that raw is somehow different doesn’t wash. And if they want to keep all the proprietary stuff, fine. All they have to do is build in a 2nd switch: save as DNG. Just as we have a switch to produce a JPEG. They, the manufacturers can have their proprietary raw data and build in all the secret sauce they want and keep what they feel is a competitive advantage. You want that, use their raw converter. You want JPEG, fine. But you want a non rendered, non demosaiced raw file that works the day the camera ships with 3rd party converters, you ask to save DNG. Simple.
Title: Re: Proprietary RAW formats
Post by: ErikKaffehr on May 01, 2012, 02:54:12 pm
+1

Erik

The camera manufactures allow us to capture into an open format, JPEG. So the idea that raw is somehow different doesn’t wash. And if they want to keep all the proprietary stuff, fine. All they have to do is build in a 2nd switch: save as DNG. Just as we have a switch to produce a JPEG. They, the manufacturers can have their proprietary raw data and build in all the secret sauce they want and keep what they feel is a competitive advantage. You want that, use their raw converter. You want JPEG, fine. But you want a non rendered, non demosaiced raw file that works the day the camera ships with 3rd party converters, you ask to save DNG. Simple.
Title: Re: Proprietary RAW formats
Post by: alain on May 01, 2012, 03:44:53 pm
But you want a non rendered, non demosaiced raw file that works the day the camera ships with 3rd party converters, you ask to save DNG. Simple.

This is an illusion, it will work, but it won't be usable for most people.  For this to work it would mean that 3rd party converters would be able to give a pleasing image without having used the camera, aka shot a number of carefully chosen test charts or test images to "calibrate" the camera.   I quite sure this "calibrating" is mostly a lot more work that "decoding" the RAW file from model X from a known supplier.  It's also a lot easier to get some RAW files to decode before "launch" than a real camera.

I don't have any doubts that you can "calibrate" you're new camera, but the average photographer won't.   
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 01, 2012, 03:54:45 pm
This is an illusion, it will work, but it won't be usable for most people.  For this to work it would mean that 3rd party converters would be able to give a pleasing image without having used the camera, aka shot a number of carefully chosen test charts or test images to "calibrate" the camera.

I don’t see how it is an illusion or how it will not work.

If the DNG converter, LR, or any other method of converting proprietary raw to DNG can do this, so can the camera. The rendering afterwards is totally moot. And don’t a few Nikon and Canon competitions do exactly what I suggested, save off a DNG? Ricoh has been doing it since 2005. Pentax and Leica has been doing it since 2006!

There is absolutely nothing technology difficult here, it is solely a political issue from the likes of Nikon and Canon. They could provide a TIFF instead of a JPEG if they wanted to. The could provide a TIFF, JPEG, DNG or proprietary raw if they wanted to.
Title: Re: Proprietary RAW formats
Post by: alain on May 01, 2012, 04:21:25 pm
I don’t see how it is an illusion or how it will not work.

... The rendering afterwards is totally moot.
...

Well for most users it isn't.  As long as the rendering isn't good it useless for most users.  If I made a "good" RAW converter I even would block the new camera's until I could give it a good rendering, far better than temporally offering a bad rendering. 

The decoding of the RAW file isn't the big problem.  (Well unless a camera maker would block external raw converters on purpose with strong encryption, but that wouldn't be smart for a "normal" camera with RAW possibilities). 
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 01, 2012, 04:30:05 pm
Well for most users it isn't.  As long as the rendering isn't good it useless for most users. 

Why would the rendering be poor? It isn’t now with the current products that can render a DNG.
Title: Re: Proprietary RAW formats
Post by: alain on May 01, 2012, 04:36:57 pm
Why would the rendering be poor? It isn’t now with the current products that can render a DNG.

See my original message.  It can made to be good with either the own RAW format or DNG (if that's the camera's own RAW format), but it needs shooting with the camera. 
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 01, 2012, 04:39:39 pm
See my original message.  It can made to be good with either the own RAW format or DNG (if that's the camera's own RAW format), but it needs shooting with the camera. 

Makes no sense, doesn’t wash with the various camera systems that have been saving out DNG’s for 6+ years.
Title: Re: Proprietary RAW formats
Post by: alain on May 01, 2012, 05:07:47 pm
Makes no sense, doesn’t wash with the various camera systems that have been saving out DNG’s for 6+ years.

Are you saying that there was nothing "needed" to get a nice rendering?  Just decoding the RAW file and it works and gives a nice rendering? 

If it's DNG it's already build in, if it's another RAW format only some extra decoding is needed.  A RAW file is just a container of RAW data, mostly "bayer" based.

No need to "calibrate" the new camera type, no need to shoot targets with it, just convert?

Title: Re: Proprietary RAW formats
Post by: digitaldog on May 01, 2012, 05:31:58 pm
Are you saying that there was nothing "needed" to get a nice rendering?  Just decoding the RAW file and it works and gives a nice rendering?

You need a converter, you need a user, you need controls. It makes zero difference if the raw is a DNG or proprietary in any such case. Your points thus far make no sense.

If you feed a NEF or CRW to Lightroom with default rendering you get one appearance. If you convert to DNG and feed that to the converter, you get exactly the same thing. The DNG is raw data. It is just data the converter can decipher from the get go.

Why should a NEF or an NEF converted to a DNG, either in-camera or 5 minutes after capture with a DNG converter be any different in terms of nice or not nice rendering?

The so called ‘issues’ you raise haven’t been anything I’ve heard of being an issue for the camera systems that save off a DNG from the get go. The main difference is, we don’t need to wait for the 3rd party raw converter folks to do anything.
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 01, 2012, 08:21:04 pm


If the DNG converter, LR, or any other method of converting proprietary raw to DNG can do this, so can the camera. The rendering afterwards is totally moot. And don’t a few Nikon and Canon competitions do exactly what I suggested, save off a DNG? Ricoh has been doing it since 2005. Pentax and Leica has been doing it since 2006!


And Samsung happily dropped DNG as soon as they stopped selling rebranded Pentax dSLRs... Ricoh purchased Pentax (even the brand name stayed)... so DNG adoption -= 2.
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 01, 2012, 08:23:09 pm
Why would the rendering be poor? It isn’t now with the current products that can render a DNG.
they can render DNG decently colorwise for example if the manufacturer supplied there a decent DNG profile... which may or may not be the case... proper raw converter manufacturers still profile the cameras.
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 01, 2012, 09:19:07 pm
And Samsung happily dropped DNG as soon as they stopped selling rebranded Pentax dSLRs... Ricoh purchased Pentax (even the brand name stayed)... so DNG adoption -= 2.

And you say happily based on what actual data points? And this in what way changes the facts about other’s who save DNG on the fly or how it would be useful to photographers?

Quote
they can render DNG decently colorwise for example if the manufacturer supplied there a decent DNG profile... which may or may not be the case... proper raw converter manufacturers still profile the cameras.

They don’t need a profile. It might be useful, it isn’t a requirement to start the rendering process. Or end users with a 24 patch ColorChecker could build their own quite easily. So the other manufacturers who save off a DNG on the fly are producing non decent color in your opinion (again based on what data points?). Or to put it another way, the camera manufacturers could produce a butt ugly DNG or they could produce a butt ugly JPEG but in the case of the later, they don’t try to do this.
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 03:02:56 am
And you say happily based on what actual data points?

yes, the data point is one and it is very simple ... no more DNG in Samsung cameras... gone...

And this in what way changes the facts about other’s who save DNG on the fly

the fact is - DNG adoption was reduced by 2 companies...


or how it would be useful to photographers?

DNG is as useful as TIFF - just another intermediate format... DNG does not prevent manufacturers to put (and encrypt) undocumented data inside, which may (or may not) be essential for a proper raw conversion... nothing wrong w/ using DNG as a container... yet another TIFF based one, like the others... yes, it is documented... so what ? take Fuji... how DNG may help Adobe to handle new bayer filter in their new camera ?


They don’t need a profile. It might be useful, it isn’t a requirement to start the rendering process.

the same is true for non DNG raw formats as well - we do not need a profile to start the rendering process...


Or end users with a 24 patch ColorChecker could build their own quite easily.

the same is true for non DNG raw formats

So the other manufacturers who save off a DNG on the fly are producing non decent color in your opinion (again based on what data points?).

I said - they may or may not... the mere fact that Adobe itself does profiling for cameras that have in camera DNG is the data point that you are asking for...

Title: Re: Proprietary RAW formats
Post by: digitaldog on May 02, 2012, 09:38:58 am
the fact is - DNG adoption was reduced by 2 companies...

Ah, so that is your argument that companies should not provide us a switch to have a DNG? We should continue to wait everyone but the manufacture’s raw converter to render the data we produce and own?

Quote
DNG is as useful as TIFF - just another intermediate format... DNG does not prevent manufacturers to put (and encrypt) undocumented data inside, which may (or may not) be essential for a proper raw conversion... nothing wrong w/ using DNG as a container... yet another TIFF based one, like the others... yes, it is documented... so what ? take Fuji... how DNG may help Adobe to handle new bayer filter in their new camera ?

You’re not telling many of us here anything we don’t already know. Expect a DNG is a lot more useful than a TIFF if we want to render the data in our raw converters. What you are doing is digressing into an area that has nothing to do with the idea that as customers, we should expect the camera manufacturers to provide is a non proprietary raw data file we can render the day the camera ships. Want to stay on topic? Can you explain why having this DNG option isn’t good for anyone but the manufacturer?
Title: Re: Proprietary RAW formats
Post by: Bryan Conner on May 02, 2012, 09:51:04 am
Can you explain why having this DNG option isn’t good for anyone but the manufacturer?

heck, I am interested in real tangible reasons why having an open source RAW format isn’t good period.  I have read a lot of info today in support of an open source RAW format and a lot against one. A lot of the people that right against dng, for example, seem to be a bit anti Adobe and not so much anti dng.  It seems to be an issue that Adobe has created this and also has a monetary interest in digital photography.  I think this is silly unless Adobe was planning to enter the camera market.

 I dont care if the open source format is dng or dog (create your own Andrew!), I just think that it is a logical step in the same sense that jpeg was a logical step in the past.  Heck...call it RPEG (rawpeg).
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 02, 2012, 09:57:00 am
heck, I am interested in real tangible reasons why having an open source RAW format isn’t good period.  I have read a lot of info today in support of an open source RAW format and a lot against one. A lot of the people that right against dng, for example, seem to be a bit anti Adobe and not so much anti dng. 

Sure seems that way when you attempt to find out their actual beef with having an open format. Like deejjjaaa (whoever that is, and whatever vested interest he/she may have against either DNG or an open raw format), they never answer simple questions like why it isn’t in our best interest to have an open format (raw or otherwise). JPEG out of the camera is fine but an open raw format? Not from the evil Adobe empire.

Quote
I dont care if the open source format is dng or dog (create your own Andrew!), I just think that it is a logical step in the same sense that jpeg was a logical step in the past.  Heck...call it RPEG (rawpeg).

I’m with you.
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 10:14:45 am
heck, I am interested in real tangible reasons why having an open source RAW format isn’t good period. 
it prevents manufacturer from implementing something new quickly and w/o letting a competition know well in advance for example.
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 02, 2012, 10:16:31 am
it prevents manufacturer from implementing something new quickly and w/o letting a competition know well in advance for example.

Quote
DNG does not prevent manufacturers to put (and encrypt) undocumented data inside, which may (or may not) be essential for a proper raw conversion

Make up your mind.
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 10:22:52 am
Ah, so that is your argument that companies should not provide us a switch to have a DNG?

no, it is not such an argument, I have nothing against DNG as a native camera raw format (except it ties manufacturers hands to certain extent) - not sure where did you get such an idea in my text... it is just a trend - DNG adoption by manufacturers is not growing both numberwise and markersharewise, but declining... when I owned Pentax cameras I did not use DNG though - just because its support by 3rd party raw converters was not as good as for PEF... plus I did not want to be in a situation when Adobe products might alter the files (even that was preventable by a proper setup - but the mere fact that Adobe products can treat raw files as not R/O only was enough).
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 10:29:16 am
Make up your mind.
everything is correct - those who use DNG now can (and do) still put undocumented data there (think Pentax/Ricoh for example) and that kills the whole idea (you know where the data is, but you do not know what it means... great, isn't it ?) and those who might want to use DNG in a proper way (= do not put undocumented data) can't innovate w/o waiting for Adobe to accomodate their needs (plus they might not want to let the world know in advance what they are up to) - think how much time it took Adobe to handle Panasonic' software optics correction in DNG
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 10:34:31 am
whatever vested interest he/she may have

well - you, for the record, clearly have vested interest in Adobe's products... you participate in PixelGenius, which is a commercial for profit company, selling products for Adobe products...
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 10:39:21 am
Can you explain why having this DNG option isn’t good for anyone but the manufacturer?

the issue is whether or not manufacturers want to fully document the data (even in their own container), not whether or not they want to use a documented container that can (and does) contain proprietary undocumented data and by nature of being a standard can tie their hands, etc.
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 02, 2012, 11:19:55 am
well - you, for the record, clearly have vested interest in Adobe's products... you participate in PixelGenius, which is a commercial for profit company, selling products for Adobe products...

Doing my best imitation of Samuel Jackson as Jules: Check out the big brain on Brad! <g>
You connect those dots yourself?

Of course I have a vested interest in Adobe products. Not that their design or the use of DNG has anything to do with a Pixel Genius product. Not a day goes by that I don’t use either Lightroom, Photoshop, InDesign, Acrobat. And yes, I make a living teaching people to use many of those tools. Vested interest? You bet. The difference between you and I is that this is all very transparent and obvious. And none of has any bearing on photographers needs, my needs, to have an open raw format be it DNG or DOG. Unlike you, I don’t use aliases here, there is a link in every post I make that takes you to a web site that spells out my involvements, including PG and Adobe.

Getting back to the topic (although you are doing a good job of digressing again)...

Quote
those who use DNG now can (and do) still put undocumented data there (think Pentax/Ricoh for example) and that kills the whole idea (you know where the data is, but you do not know what it means... great, isn't it ?)

Kills it, nope. Not even close (talk about the mind set of throwing the baby out with the bath water). The data is sufficient that should say Canon come out with the 5DMIV with a DNG switch, I can handle that data in my beloved Adobe raw converter the day it ships. If I want whatever you consider useful undocumented data, I move the switch to proprietary raw and use the Canon converter. Or I move the switch to JPEG.
Title: Re: Proprietary RAW formats
Post by: Bryan Conner on May 02, 2012, 03:30:03 pm
the issue is whether or not manufacturers want to fully document the data (even in their own container), not whether or not they want to use a documented container that can (and does) contain proprietary undocumented data and by nature of being a standard can tie their hands, etc.

How can you know that proprietary undocumented data exists if it is undocumented?
Title: Re: Proprietary RAW formats
Post by: deejjjaaaa on May 02, 2012, 07:15:58 pm
How can you know that proprietary undocumented data exists if it is undocumented?

how did we know that Adobe was removing masked to light pixels during non DNG to DNG conversion and then silently stopped removing starting from a certain version of Adobe DNG converter... we spend time reading forums and testings what was reported.
Title: Re: Proprietary RAW formats are widely readable
Post by: BJL on May 02, 2012, 09:43:08 pm
I agree with you 100% about everyone working together.  But, as long as the bottom line is money, the happiness of the customer is going to always be second at best.
I am not holding my breath, but there have been cases where competitors see the advantages of standardizing on some things that are not a key basis for competition (JPEG, MPEG/H.264) and I do not see the advantages of one raw format over another being used as marketing points, so standardization could be relatively painless.

But maybe raw formats are adapted to new hardware features, so that there would be the fear of having to share secrets in order to get the common format to work optimally with each new generation of sensors. After all, raw formats do seem to be refined for each new sensor.
Title: Re: Proprietary RAW formats
Post by: Bryan Conner on May 03, 2012, 01:03:05 am
how did we know that Adobe was removing masked to light pixels during non DNG to DNG conversion and then silently stopped removing starting from a certain version of Adobe DNG converter... we spend time reading forums and testings what was reported.

Then this is not undocumented data!  It had to be documented for anyone to know about it.  Also, you are talking about Adobe data, not a camera manufacturers raw file data.
Title: Re: Proprietary RAW formats
Post by: digitaldog on May 03, 2012, 10:12:53 am
"Reports that say something hasn't happened are interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don't know we don't know."
From: Donald Rumsfeld Wins 'Foot in Mouth Award'http://politicalhumor.about.com/b/a/047299.htm

Maybe this is what deejjjaaaa is talking about? <g>
Title: Re: Proprietary RAW formats
Post by: Bryan Conner on May 03, 2012, 01:58:44 pm
"Reports that say something hasn't happened are interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don't know we don't know."
From: Donald Rumsfeld Wins 'Foot in Mouth Award'http://politicalhumor.about.com/b/a/047299.htm

Maybe this is what deejjjaaaa is talking about? <g>

Probably, and like the great wordsmith Yogi Berra once said "It's like deja-vu, all over again.".
Title: Re: Proprietary RAW formats
Post by: Chris Pollock on May 20, 2012, 02:22:31 am
Arrogance? You know what is arrogant? To build a RAW converter based on illegal reverse engineering of files and a totally erroneous conversion process, and then trying to dictate the market into adopting some kind of bogus open raw format that fails to properly separate "data" from "processing".
Do you have any evidence to back up your assertion that reverse-engineering a file format is illegal? Anyone other than the camera manufacturer has to reverse engineer raw file formats to open them. Do you really think that the companies who write third party raw conversion software are all breaking the law? If so, why haven't they been succcessfully sued?

As you're probably aware, it's common practice for office software to be able to open files written by competitor's software. Is this illegal?
Title: Re: Proprietary RAW formats
Post by: Farmer on May 20, 2012, 02:40:53 am
Spot on.  Not only that, Chris, but does anyone really think that Adobe reverse engineers them before they are released so that often they have an update for a particular raw format at the same time the camera is released (or very shortly thereafter)?