Yet some DNG comments (from a raw software developer)

[digitaldog]

Dave is amazing. Hopefully, if he ever gets tired of doing DCRaw single handed, DCRaw could be formed up into a proper open source project, or perhaps e.g., LibRaw could take a more active role in developing new camera support.

This begs the question, why should he have to do all this hard work in the first place?

[torger]

which DNG does not solve either... solution is not DNG, but documenting of whatever format is being used, that's it... and it is already shown that (dSLR/dSLM) manufacturers do not want or do not have compelling reasons to document...

PS: Ricoh/Pentax - what about "DNGPrivateData" tag in their DNG ?

My background is in network communications and telecom. The internet today is founded on open well-documented standards. I know this is politically harder to achieve in the photography world as there are not the same commercial driving forces to actually be able to exchange data. In any case I'm not going to adopt the idea that hundreds of slightly different proprietary raw formats, documented or not, is a good solution. Although not perfect, DNG is the only attempt to actually get closer to something standardized, and there are already a few adopters.

To make it a real standard we can't have DNG managed by one person on Adobe, it must of course get into a standards body. But we can work from where we are now. Adobe stuff has ended up as standards before, such as the relative colorimetric black point compensation algorithm.

I have not been much involved in the "flame wars" that usually seems to arise around DNG vs proprietary raw, and they're still a mystery to me. How can you not want a standard as a photographer? I think it's fair to argue that DNG specifically may not be up to it in it's current state, but striving to get to a standard is a noble cause and noone which cares about photography should think that is a bad idea. It seems to me that some people don't like the idea of DNG is more about that they don't like Adobe as a company or their licensing policies or whatever rather than they hate standards for real. Those that do hate standards for real should be forced to work as an engineer and integrate various systems without any standard protocols. I'm sure they will change their minds after a while.

It's not that hard for the camera manufacturers to strive for a standard, even help out Adobe in refining DNG. The overhead of managing a standard format should be minimal compared to the cost of developing a new camera, and in the longer term you could actually save money. The only way to make it happen is if photographers start asking for it though, for the camera makers are in many ways really really conservative.

[digitaldog]

To make it a real standard we can't have DNG managed by one person on Adobe, it must of course get into a standards body.

In the works I'm told. But that takes far too long but again, in the works.

I have not been much involved in the "flame wars" that usually seems to arise around DNG vs proprietary raw, and they're still a mystery to me. How can you not want a standard as a photographer?

I've asked that question numerous times in the "flame wars" and as yet haven't received an answer from the photographer’s side, only the manufacturers side through someone who is presumably a 'photographer'.

I think it's fair to argue that DNG specifically may not be up to it in it's current state, but striving to get to a standard is a noble cause and noone which cares about photography should think that is a bad idea

Many of us agree. Some don't. Which makes it more difficult to move this all forward. What's the old saying? You're either part of the solution or part of the problem.

It seems to me that some people don't like the idea of DNG is more about that they don't like Adobe as a company or their licensing policies or whatever rather than they hate standards for real. Those that do hate standards for real should be forced to work as an engineer and integrate various systems without any standard protocols. I'm sure they will change their minds after a while.

I can't see how it would be otherwise so I violently agree with you.

It's not that hard for the camera manufacturers to strive for a standard, even help out Adobe in refining DNG.

As I've said countless times, this isn't a technology issue, it's a political issue. Which makes it more difficult to accept so called photographers auguring for the manufacturers and not those of us that simply want control over OUR data.

[torger]

This begs the question, why should he have to do all this hard work in the first place?

He doesn't do all hard work today, I'm one of those that sends him patches once in a while and I'm sure others help out too.

A very much respect Dave's work and very grateful about it, but it's not all good that he has the "central power". A key problem is that he has a very special coding style as seen in DCRaw, and I mean very special, which means that the code is very hard to get into and maintain and expand except for Dave himself. He thinks himself the exact opposite that his coding style is ideal, and it probably is if you're some kind of programming genius, but most of us aren't.

Although I thoroughly document my patches document what I have figured out and what is still open issues, Dave removes all comments when he includes it in a release (because if you're a programming genius you don't need comments ).

So having it all concentrated in DCRaw is not an ideal situation for collaborative work.

To make it work as a broader collaborative open-source project I think the code should be restructured from scratch, and the code should be richly commented etc so it's clear what's supported and not, what is sure and what is guessed etc, and it should exist in the open at github or similar.

Maybe the libraw guys could step in at some point, I don't know. I think few can reverse-engineer camera formats as good as Dave can, and as long he is number one, DCRaw will be the reference.

When I reverse engineer a format it's with feelings of anger and frustration and my driving force is the obsession with "no f*****g manufacturer is going to lock in my image data". I do hope that Dave is one that actually enjoys the challenge though, otherwise he has a living hell

[AlterEgo]

In any case I'm not going to adopt the idea that hundreds of slightly different proprietary raw formats, documented or not, is a good solution.

you are certainly entitled to your opinion

there are already a few adopters.

the sad fact is that those few adopters in dSLR/dSLM market are in place for ages (so "already" is not a proper word here - unless for a purely rhetorical purpose) and they are still - few... and for a good reason, the manufacturers either simply do not have a reason or do not want to have the hands tied/keep competition aware/etc.

How can you not want a standard as a photographer?

as a photographer (which I am not - being an amateur does not count) I do not care actually... otherwise I simply understand the reason why manufacturers do not want to use DNG or do not want to publish documentation publicly - but may as well share the info off the record/under NDA with for example Adobe itself or others... and I am hoping for a simple (for manufacurers) solution that is they will eventually get to the point where documenting their own formats will be beneficial for them, at least post factum (after release of a camera)... that's more realistic to expect... but for as long as the likes of pro-DNG people here will continue to buy cameras w/o DNG, for as long as the likes of Adobe will continue to support prop. formats and for as long as the masses will be using Adobe converters despite the fact that Adobe supports prop. formats (yes, that is actually the reason) then camera manufacturers are simply correct in that they do not need to bother.... I might not like it, but at least I am not hypocrite (hello guys).

[AlterEgo]

It's not that hard for the camera manufacturers to strive for a standard, even help out Adobe in refining DNG.

not hard - but no reasons, more expenses, need to share developments with competition, hands are tied to implement changes fast... so ?

The overhead of managing a standard format should be minimal

there is no overhead not to do that... so ?

[AlterEgo]

I've asked that question numerous times in the "flame wars"

and you were asked numerous times in the "flame wars" what Panasonic was supposed to do when they introduced additions related to software optics corrections in their raws... disclose the move to competition in advance ? lose money waiting (with the release of their new cameras) for Adobe to implement changes in DNG - do you recall how many month did it take for Adobe to support that in DNG ? and that was when changes in DNG are basically one Knoll-man decision.. now imagine changes when there is ISO deciding ? you never answered... for a reason - you simply do not have an answer

[AlterEgo]

so I violently agree with you.

how about not buying a camera w/o DNG or not buying Adobe or any other raw converter that dares to support prop. formats ? put your money where your violent mouth is 

[AlterEgo]

When I reverse engineer a format it's with feelings of anger and frustration and my driving force is the obsession with "no f*****g manufacturer is going to lock in my image data". I do hope that Dave is one that actually enjoys the challenge though, otherwise he has a living hell

good feelings... I only can add that users of ACR/LR/C1/etc apparently are OK that "f.......g" Adobe/PhaseOne/erc lock their hard work on parametric raw conversion adjustments... so I 'd assume you only use open converters like rawtherapee, which does not do that... OR ?

actually for me that "f.......g" Adobe's (or PhaseOne's) policy matters much more than prop. raw formats  ... yet I understand the reasons why they want to not to have any open standard on access to parametric raw conversion adjustments (that is how to interpret them)... what's your stand on that ?

[sandymc]

I do hope that Dave is one that actually enjoys the challenge though, otherwise he has a living hell

Yes, well, I think he would have to enjoy it in order to have done what he does for so long. But it would be better if DCRaw was a least broken out into a library and a main program. Even better if it was modularized.

But it's always going to be problematic. E.g., the raw engine I use for PhotoRaw/Accuraw is multithreaded, and optimized to minimize memory usage, both of which add a lot of complexity. If I were to be honest, I doubt that DCRaw (or LibRaw/any open source venture) would be willing to deal with that kind of complexity - for open source purposes, it's just not necessary.

Sandy

[AlterEgo]

Yes, well, I think he would have to enjoy it in order to have done what he does for so long. But it would be better if DCRaw was a least broken out into a library and a main program. Even better if it was modularized.

but libraw, for example, already does that kind of refactoring...

I doubt that DCRaw (or LibRaw/any open source venture) would be willing to deal with that kind of complexity

even libraw code-man said a few days ago "А остальное - параллелим (потому FastRawViewer - fast), но не в опенсорсе."

[sandymc]

but libraw, for example, already does that kind of refactoring...

Indeed. But the libraw guys are even slower to put out updates than Dave is! ( No insult to them - open source libraries are really hard)

Which is to say, I'd be very happy if there were a raw library that worked to commercial standards - clean, modular code, multithreaded, frequent updates. And I'd contribute code. But the incentives aren't really there for that happen.

BTW, on the topic of DNG, it would be entirely possible to integrate the DNG SDK with the front end of DCRaw. A major undertaking to pick apart DCRaw, certainly, but entirely doable. And the DNG SDK has multithreading hooks built in. But of course, licensing would be a problem.

Sandy

[AlterEgo]

But the libraw guys are even slower to put out updates than Dave is!

public speed (or pace, rate) of updates : https://github.com/LibRaw/LibRaw/commits/master

[torger]

AlterEgo, I think you're on the border of trolling.

As much as I enjoy the politics side of it, I don't want the technical aspect to get lost here. Even if DNG won't ever be a standard a see gains in my coding projects if the format and/or documentation is improved, as I must support both DNG and DCP in them.

[AlterEgo]

I don't want the technical aspect to get lost here.

OK, let us stay on technical aspects - we will see how next will post anything non technical here 

[digitaldog]

AlterEgo, I think you're on the border of trolling.

Par for the course and the reason he's on my very short ignore list.
Please continue! Lots of excellent information thus far.

[AlterEgo]

Par for the course and the reason he's on my very short ignore list.
Please continue! Lots of excellent information thus far.

so who is trolling here  ?

[BrianVS]

TIFF-EP is set by the ISO body; no one seems to use it as you have to pay $300 or so to get a PDF of the standard.

Many companies spend a lot of money putting representatives into these bodies to gear the "international standard" towards their products. Telcos standards are infamous for this. Not all adopted standards are without problems, and some should have been thought out better. The "Ones-Complement 16-bit checksum" used in IPV4 should have been explained in the standard, it caused a lot of problems when V4 came online. Ones complement arithmetic to a CDC programmer has a different meaning than the one used by the IPV4 author. Fun Days. I implemented it both ways and let the error analyzer sort out which was the correct interpretation of the standard.

It would be nice to see DNG used in more cameras. But- I have to ask "How does this benefit me". Being able to look at a Hex dump and write code to pull images out of it- pays well.

[Schewe]

To make it a real standard we can't have DNG managed by one person on Adobe, it must of course get into a standards body. But we can work from where we are now. Adobe stuff has ended up as standards before, such as the relative colorimetric black point compensation algorithm.

As far as I know, Adobe has already offered DNG to the ISO to be incorporated into an upcoming TIFF-EP update. How long that takes is anybody's guess.

In terms of background, Thomas originally created DNG for the purpose handling backwards compatibility to older software–a new camera file today can be can be converted to DNG and be opened as far back as Photoshop CS2. Thomas also wrote DNG for the purposes of educating the camera companies on how to create a well formed raw file format. While Nikon and Canon have not adopted DNG, they have evaluated DNG and learned from it.

The more modern raw file formats are really close to DNG in many respects compared to the early raw file formats like Canon's original .tif and .crw. I'm not sure about Nikon–although they caused a major problem when they encrypted the white balance metadata of the D2x). Dave Coffin decrypted the metadata. Ironically, the reason why Nikon encrypted the metadata was they couldn't figure out how to pass the data from the camera to Nikon Capture so they encrypted it and used a key that the camera and software could read. To solve the problem and to allow Adobe to use the metadata, Nikon released a mini SDK to decrypt the metadata after Dave had broken it :~)

[jrp]

Here is why one photographer no longer converts to DNG https://photographylife.com/why-i-no-longer-convert-raw-files-to-dng#more-116618