RAF files

[teddillard]

yeah well...  

I know we've just had some more important stuff going on and all, but I'm beginning to see the camera manufacturers' positions.  After a few days, and after I've posted notice of this on several groups, there are a scant 50 sigs. If no one cares enough to simply go to a petition and support it, then they certainly aren't going to buy a camera because of it.  

(thanks to those that DID sign, by the way...)

The effort to post trashing OpenRaw and beat your chest about a "unified voice" is many times more that a simple couple of clicks signing the petition.  Even more, the effort to post supporting the petition is a constructive step, even better would have been some support on how to make a great petition, rather than a hollow rant on a forum.

Granted, I'm pretty new here, (as far as posting goes anyway), but this is the second time I've seen pretty, well, "spirited" opinions posted (read: opinionated, exclusionary and often mildly abusive...) and then when there's a suggestion that something actually be done, some consensus reached and some progress made in resolution of these issues, then, nothing.  

So, if that's what this noise amounted to, then fine, I'm just asking.  It makes it kind of hard to answer a simple question, like what the OP was here, but whatever, if that's the culture here... so be it.

If you do actually want to do something, however insignificant or misguided you feel it may be, the petition is here:
http://www.petitiononline.com/dng01/petition.html

[john beardsworth]

That is a rather silly post, Ted. OK, not many turned up to your petition, but try to think of it as your "I've been to the mountaintop" part in the long struggle. As for trashing OpenRaw, well, I think the response was more like Rosa Parks deciding it wasn't enough to just ask for better things. Even then, it took time, many other protests and wider developments, before the promised land was reached.

John

[teddillard]

That is a rather silly post, Ted. OK, not many turned up to your petition, but try to think of it as your "I've been to the mountaintop" part in the long struggle. As for trashing OpenRaw, well, I think the response was more like Rosa Parks deciding it wasn't enough to just ask for better things. Even then, it took time, many other protests and wider developments, before the promised land was reached.

John

Once again, John, missing my point.  Silly indeed.  One of the many words I decided to not use, however tempting, in describing the posts on this thread.

[john beardsworth]

Once again, John, missing my point.

Insofar as there was a clear point, Ted, I'm afraid it comes over as no-one cares enough to actually do something by joining your petition, so there's not enough support for DNG and what you call "noise" really amounts to nothing constructive. I doubt the reference would cross the pond, but it's all a bit like Wolfie Smith berating the proletariat for not hearing the Tooting Popular Front's call. Progress towards a universally-readable common image format comes in many small steps, from the O{ ticking the ballot box and converting those RAF files, to trashing OpenRaw or questioning camera makers' claims that they make the best software for our pictures, to people pointing out that DNG's workflow benefits shouldn't be narrowly-defined in processing terms, to Adobe making the DNG format an agreed standard, to Windows XP users being unhappy that Adobe failed to make a DNG codec for XP, to Apple including baseline support for the DNG format, and a thousand other moves in the right direction. Best not to get too hung up or imply too much from how few folk might sign a petition.

John

[teddillard]

Insofar as there was a clear point, Ted, I'm afraid it comes over as no-one cares enough to actually do something by joining your petition, so there's not enough support for DNG and what you call "noise" really amounts to nothing constructive. I doubt the reference would cross the pond, but it's all a bit like Wolfie Smith berating the proletariat for not hearing the Tooting Popular Front's call. Progress towards a universally-readable common image format comes in many small steps, from the O{ ticking the ballot box and converting those RAF files, to trashing OpenRaw or questioning camera makers' claims that they make the best software for our pictures, to people pointing out that DNG's workflow benefits shouldn't be narrowly-defined in processing terms, to Adobe making the DNG format an agreed standard, to Windows XP users being unhappy that Adobe failed to make a DNG codec for XP, to Apple including baseline support for the DNG format, and a thousand other moves in the right direction. Best not to get too hung up or imply too much from how few folk might sign a petition.

John

Thanks for the advice.  Like I've said before, I'm a man who needs an Editor.  

(My point was simply if nobody cares enough to take the 30 seconds it takes to sign this, then the manufacturers are justified in not putting the effort into offering it.)

Now sign the petition.  

[Schewe]

(My point was simply if nobody cares enough to take the 30 seconds it takes to sign this, then the manufacturers are justified in not putting the effort into offering it.)

No, you got it wrong...failure to place a vote on a poll has everything to do with the way you characterize and promote it. Maybe if you had asked Michael to post a note about it on his what's new page...or some other site that gets a ton of hits. Maybe if you hadn't posted the aborted thread where you asked people to just do the poll without engaging in discussion (yeah, like THAT'S gonna work–NOT).

The fact that you're all fired up and took to the streets only to find very few people out in the streets says nothing about the issue and everything about that fact everything takes time and promotion and a bit more of a long term view that 10 days. The fact is, 50 people saw the thread and voted...rather than feeling good about that, you are blaming everybody ELSE for not doing something.

Very short sighted in my view. I've been advocating DNG for about 5 years (encouraging Thomas Knoll even before the DNG format was announced) so pardon me if I don't get all excited over a poll.

[teddillard]

No, you got it wrong...failure to place a vote on a poll has everything to do with the way you characterize and promote it. Maybe if you had asked Michael to post a note about it on his what's new page...or some other site that gets a ton of hits. Maybe if you hadn't posted the aborted thread where you asked people to just do the poll without engaging in discussion (yeah, like THAT'S gonna work–NOT).

The fact that you're all fired up and took to the streets only to find very few people out in the streets says nothing about the issue and everything about that fact everything takes time and promotion and a bit more of a long term view that 10 days. The fact is, 50 people saw the thread and voted...rather than feeling good about that, you are blaming everybody ELSE for not doing something.

Very short sighted in my view. I've been advocating DNG for about 5 years (encouraging Thomas Knoll even before the DNG format was announced) so pardon me if I don't get all excited over a poll.

And THAT is exactly why I asked for your help...  you've been promoting this, you know the issues, you know this forum and how best to post something like this, yet all I've got is pot-shots and criticism.  I'm not asking anyone to sell their souls to satan here, I'm simply observing that the last thing done like this is apparently not current, from 2004, and had great support.  Now the issue is still there, Adobe has followed through and done what they've said they would do, I merely thought it would be a good time to give it another poke, and am amazed at how little interest there is in it beyond a willingness to yabber on endlessly.  

I'm done with it.  I'm going back to my lovely Smart Objects.

[Schewe]

And THAT is exactly why I asked for your help...  you've been promoting this, you know the issues, you know this forum and how best to post something like this, yet all I've got is pot-shots and criticism.

Uh huh...well, some of us HAVE been working on this for a while now and at the moment are otherwise engaged (two books to finish and the start of a 15 city tour for the Epson Print Academy), so if you want to do it, do it...I'll continue to work towards the goals as I've been doing (like working with Adobe to advance the agenda and the technology)

I'm done with it. I'm going back to my lovely Smart Objects.

Ok, bye now...

[teddillard]

Uh huh...well, some of us HAVE been working on this for a while now and at the moment are otherwise engaged (two books to finish and the start of a 15 city tour for the Epson Print Academy), so if you want to do it, do it...I'll continue to work towards the goals as I've been doing (like working with Adobe to advance the agenda and the technology)



Ok, bye now...

It took you more time to post the snipes at what I did do, than it would have taken to give some constructive suggestions, Jeff.  

...whatever.  I have my own 2 books to work on.

[Dan Wells]

How much is there in RAW files that DNG doesn't support? Whose RAW files contain secret sauce? The only folks I know of who'll talk openly about this are Hasselblad (who also support a DNG conversion option in their proprietary software, although not in-camera). Hasselblad folks claim (David or Paul from Hasselblad, please correct me if I have this wrong) that DNG will support everything they do, EXCEPT for their DAC lens corrections. There are apparently extra data in the FFF raw files that support the lens corrections, which there is no place to put in a DNG, and which only Flexcolor and Phocus can read anyway. I don't know of any other manufacturer doing similar lens corrections (or at least not describing them in the same way - Phase One and DXo (and maybe others) both seem to do similar things, but entirely after the fact, not relying on embedded data). Hasselblad's lens corrections reach the apex of complexity in their new HTS 1.5 tilt-shift adapter, where DAC corrects for side effects of camera movements! I know that nobody else is doing that right now. I'm pretty sure that both "odd sensor" RAW formats (Fuji and Foveon/Sigma) have some incompatibilities with DNG, which assumes a standard Bayer sensor. Is everything except these three formats (Hasselblad, Fuji and Foveon) basically compatible with DNG? Does Nikon, Canon or someone else have fields in their RAW files for future use in lens correction that DNG doesn't support? As far as I know, the Hasselblad trick with the tilt-shift adapter is completely unique, because nobody else's tilt-shift lenses electronically report the degree of tilt or shift to the camera? Other makers' lenses certainly do share data with the camera, and I'm not totally sure that other manufacturers are not using these data in their conversions (although, if they are, I'd be expecting to hear about it in advertising - it's a great argument to buy the camera manufacturer's lenses).


                      -Dan

[teddillard]

How much is there in RAW files that DNG doesn't support? Whose RAW files contain secret sauce? The only folks I know of who'll talk openly about this are Hasselblad (who also support a DNG conversion option in their proprietary software, although not in-camera). Hasselblad folks claim (David or Paul from Hasselblad, please correct me if I have this wrong) that DNG will support everything they do, EXCEPT for their DAC lens corrections. There are apparently extra data in the FFF raw files that support the lens corrections, which there is no place to put in a DNG, and which only Flexcolor and Phocus can read anyway. I don't know of any other manufacturer doing similar lens corrections (or at least not describing them in the same way - Phase One and DXo (and maybe others) both seem to do similar things, but entirely after the fact, not relying on embedded data). Hasselblad's lens corrections reach the apex of complexity in their new HTS 1.5 tilt-shift adapter, where DAC corrects for side effects of camera movements! I know that nobody else is doing that right now. I'm pretty sure that both "odd sensor" RAW formats (Fuji and Foveon/Sigma) have some incompatibilities with DNG, which assumes a standard Bayer sensor. Is everything except these three formats (Hasselblad, Fuji and Foveon) basically compatible with DNG? Does Nikon, Canon or someone else have fields in their RAW files for future use in lens correction that DNG doesn't support? As far as I know, the Hasselblad trick with the tilt-shift adapter is completely unique, because nobody else's tilt-shift lenses electronically report the degree of tilt or shift to the camera? Other makers' lenses certainly do share data with the camera, and I'm not totally sure that other manufacturers are not using these data in their conversions (although, if they are, I'd be expecting to hear about it in advertising - it's a great argument to buy the camera manufacturer's lenses).


                      -Dan

Dan, I think this is a really good question...  for the most part, my understanding is there is data in almost every, if not every, maker's RAW file that is considered proprietary.  The question of different sensor designs, like Foveon for example, is a good question I've seen brought up elsewhere...  in theory, if a maker designs a sensor that is not a Bayer Array then the DNG standard would need retooling.  I wasn't aware of data that couldn't be written to DNG, my impression from what Adobe has told me was that there was room for pretty much anything, it was more a case of the makers deciding they wouldn't.  

The issue, for me and what I recommend to people, comes down to the point made by the Fuji user above, Michael Bailey, who doesn't see a difference between a Camera RAW conversion from DNG or RAF.  IF you're happy with what Adobe does to the raw file from your camera then it makes perfect sense to convert and save the DNG as an archival source.  That is presuming that, although the maker does have special sauce, Adobe does fine without it.  In the case of the Hasselblad files you're bringing up, then I'm guessing you'd be happier using the Hasselblad software (have they got Phocus working yet?), since it sounds like it makes a better file.  My experience with Hasselblad has been that Flexcolor made a far, far superior conversion of the DNG than Adobe did, but that was a few years ago.

It all comes down to "show me the beef".  If the camera makers can prove to me they have a better solution than Camera RAW (or the processor of your choice) then fine, I'll use their processor.  If not, then I'm back to Camera RAW.  

That said I still just can't understand the maker's lack of adoption of DNG.  If their files and sauce are better, then adoption of DNG would prove that, as their raw files are superior to even their own in-camera JPG.  It would be an option for shooters, it would open their market more.  The one hint that I keep mulling is Tom Hogarty's cryptic comment about camera horsepower...  

In-camera support for the format is far from universal, though DNG-capable cameras from Casio, Pentax, Ricoh, Leica and Samsung have been launched. "A lot of cameras don't have the horsepower," said Hogarty, and confirmed discussions with manufacturers were continuing: "there is continual dialogue with manufacturers so we can support all the proprietary formats out there and make sure DNG has the features they need."

http://www.dpreview.com/news/0805/08051503adobeDNG.asp

Perhaps shooting DNG takes more resources, and slows things down?  I dunno, but I don't know of any makers who are telling...

[Dan Wells]

Dan, I think this is a really good question...  for the most part, my understanding is there is data in almost every, if not every, maker's RAW file that is considered proprietary.  The question of different sensor designs, like Foveon for example, is a good question I've seen brought up elsewhere...  in theory, if a maker designs a sensor that is not a Bayer Array then the DNG standard would need retooling.  I wasn't aware of data that couldn't be written to DNG, my impression from what Adobe has told me was that there was room for pretty much anything, it was more a case of the makers deciding they wouldn't.  

The issue, for me and what I recommend to people, comes down to the point made by the Fuji user above, Michael Bailey, who doesn't see a difference between a Camera RAW conversion from DNG or RAF.  IF you're happy with what Adobe does to the raw file from your camera then it makes perfect sense to convert and save the DNG as an archival source.  That is presuming that, although the maker does have special sauce, Adobe does fine without it.  In the case of the Hasselblad files you're bringing up, then I'm guessing you'd be happier using the Hasselblad software (have they got Phocus working yet?), since it sounds like it makes a better file.  My experience with Hasselblad has been that Flexcolor made a far, far superior conversion of the DNG than Adobe did, but that was a few years ago.

It all comes down to "show me the beef".  If the camera makers can prove to me they have a better solution than Camera RAW (or the processor of your choice) then fine, I'll use their processor.  If not, then I'm back to Camera RAW.  

That said I still just can't understand the maker's lack of adoption of DNG.  If their files and sauce are better, then adoption of DNG would prove that, as their raw files are superior to even their own in-camera JPG.  It would be an option for shooters, it would open their market more.  The one hint that I keep mulling is Tom Hogarty's cryptic comment about camera horsepower...  

In-camera support for the format is far from universal, though DNG-capable cameras from Casio, Pentax, Ricoh, Leica and Samsung have been launched. "A lot of cameras don't have the horsepower," said Hogarty, and confirmed discussions with manufacturers were continuing: "there is continual dialogue with manufacturers so we can support all the proprietary formats out there and make sure DNG has the features they need."

http://www.dpreview.com/news/0805/08051503adobeDNG.asp

Perhaps shooting DNG takes more resources, and slows things down?  I dunno, but I don't know of any makers who are telling...

If there's room for "pretty much anything" in DNG, then I see no reason for manufacturers not to write the standard stuff into DNG in a standard way, then add lens correction data (Hasselblad claims that they have 57,000 possible corrections in Phocus, and they use metadata from the lens to choose which one to apply) or other things that no non-manufacturer converter can understand into extra fields in the DNG. That way, a file exists that anything can convert, and that the manufacturer's software can convert better... The only place this might break down is in the (rare) case of non-Bayer sensors (and DNG seems to handle Fuji, although not Foveon, competently). What camera manufacturer is still trying to SELL RAW conversion software? Only Nikon? Anyone who is giving away the software with the purchase of a camera should have no interest in forcing its use... If CR2 (for example) is so close to DNG as to be convertible by firmware upgrade, what possible interest does Canon have in not providing that upgrade? If it's that close, it can't possibly require extra camera horsepower... Even if there's stuff in the file a standard converter won't understand, why not do what Hasselblad does and say "we're happy to go to DNG, here's what you get, and here's what's missing". If everybody else does it, Nikon will have to go along (what percentage of their revenue can they possibly make by selling Capture NX)? As far as I can see, the camera manufacturers don't lose (apart from Nikon, they haven't had a financial interest in their proprietary raw files for years), the converter manufacturers benefit by only having to understand one standard, and photographers benefit from knowing they can read the files years later. If there is something in the "extra" data they don't want competitors seeing, encrypt that, and that alone. Every manufacturer knows what a standard RAW file, a white balance, and the like look like anyway, so why bother keeping that secret? Even proprietary lens correction data shouldn't be an issue,  because most of the proprietary stuff is almost certainly in the lookup tables on the software side, anyway. Knowing that the lens was a 50 mm from production batch 11, maybe even serial number 11-09876, focused at 1.7 meters, an HTS was used, tilted 1.3 degrees, and 2 mm of front rise was used shouldn't help Hasselblad's competitors (I use Hasselblad as an example, because they've been unusually open about this), as long as what Phocus DOES with that information remains Hasselblad intellectual property. Can anyone think of truly proprietary information coming from the camera?

                                                                       -Dan

[john beardsworth]

It could merely be that they like to pack everything into a single lump of data rather than in easily-read paragraphs....

John

[teddillard]

I just found this, fairly incredible resource on DNG:
http://www.barrypearson.co.uk/articles/dng/

...cant' believe I hadn't heard about this, but then I've always been a little slow.  He goes into just about everything imaginable and I've only had a chance to skim the surface.  He even goes into some detail about OpenRaw as a post-mortem, which should make you guys happy, Jeff and John.    

[teddillard]

It could merely be that they like to pack everything into a single lump of data rather than in easily-read paragraphs....

John

...don't pick on him, I LOVE this guy!  

[teddillard]

You link to OpenRaw? Isn't that the site for people who don't see the point of DNG and believe the camera makers will give up their secret formats?

...not to beat a dead horse, but reading on Barry's (incredibly helpful) DNG resource site led me to this:
http://www.barrypearson.co.uk/articles/dng/origins.htm

I'm feeling vindicated in suggesting reading up on OpenRaw.

Barry was, early on, involved with the OpenRaw site.  He also, early on, saw the mistakes that were being made there, as did, eventually I see, the guy that started OpenRaw.  Barry went on to create his own site in support of DNG.  30-some pages.

I'm not trying to stir up OpenRaw debates here again, only to point out the value of complete research for a balanced understanding of the issues.  You learn as much from the mis-steps as you do from the successes.

That said, if I'd known about Barry's site at the time, I probably would have foregone the OpenRaw link for his.