Luminous Landscape Forum
The Art of Photography => Rantatorials [ARCHIVED] => Topic started by: ThomasH_normally on September 17, 2015, 10:59:35 am
-
I hope that you saw the great technical analysis of DNG details by 'torger'. If not, I would recommend to lookup the thread "Yet some DNG comments (from a raw software developer)". Probably none of us is better qualified to assess usability of DNG as a universal and standard RAW format.
Still, I believe that Michael's rantatorial hits the spot. We need prominent photographers to voice this opinion more often. If not DNG, than another format should be taken, but please the same universal format for all. Personally though I have the impression that software manufacturers also like the situation, because the constantly arriving new cameras force everybody to upgrade the software. With a format like DNG in place, many users would not be compelled to grab every one of the new image processor versions.
The problem of standard fragmentation itself is wider than the RAW file format. Think of the battery-o-mania. We have meanwhile hundreds of differing batteries, for each we need a dedicated charger. On the road I carry with me a set of power cord spliters, and a dedicated bag of chargers. Once I made a mistake, and plugged a wrong charger cable into my portable drive with card reader: Same plug, different voltage. What a smell of burning electronics... Once I took both the Canon and Nikon popular battery, put these on a scale and assessed the charge to compare to 4 low discharge AA's, such as Sanyo Eneloops. The difference in weight was minimal, but the 4 AA's provided way more charge, justifying the few more grams. And they are everywhere, offered in a fierce competition for charge and quality. We are spending fortune on proprietary batteries, and often get 3rd party replacements, which prove subpar.
By analogy to this raw-file and battery chaos, imagine every county or state would have its own power plug, maybe even differing voltage. we would need in every hotel a dedicated cable and/or converter! We have this situation across countries, and we clearly suffer from it.
At least at a national level, we enjoy the building code, forcing everybody to use identical standard for plugs and sockets, approved through a certified underwriter. Sometimes industry standards establish themselves and prevail. Think of the now forgotten IBM-Floppy, or the cat-5 internet plug and cable. Sometimes a stringent control and laws rule, like in case of radio frequencies.
My opinion is, that such an action is necessary for the portable power sources as well, and also for the raw file format. Imagine a law would be issued, that every camera has to use a battery from a mandatory set of alternatives, and has to use a mandatory raw file. I am all for it.
-
> Think of the battery-o-mania. We have meanwhile hundreds of differing batteries, for each we need a dedicated charger.
classical FUD, as in most cases the it is the same format with just a different camera model in the same tag and that still does not address the need of raw converter manufacturers to make their own camera profiles...
-
We need prominent photographers to voice this opinion more often. If not DNG, than another format should be taken, but please the same universal format for all.
Actually, if you want something like this done, the shortest path is to simply do it. I see lots of people standing around saying "someone ought to do a bunch of work here" but nobody actually stepping up to, you know, do it.
Just write a standard. Pull together a committee of experts, of which there seem to be a tremendous number. Promote it to standards bodies.
-
Once I took both the Canon and Nikon popular battery, put these on a scale and assessed the charge to compare to 4 low discharge AA's, such as Sanyo Eneloops. The difference in weight was minimal, but the 4 AA's provided way more charge, justifying the few more grams.
When? Because most modern batteries are Li-Ion, and have much greater charge density than anything that comes in AA-format. Moreover, since a single "classic" dry-cell is a 1.5V cell and a single Li-Ion is 2.4V, there will not be true Li-ion AA's... too much risk of burning electronics.
Yes, standard batteries would be wonderful. So would standard headlamps on cars. It's not going to happen, too much money to be made selling brand-specific batteries.
Non-standard RAW is a different story, it does nothing to make anyone any money that I can see...
There is a solution of course: buy a Pentax. You get DNG, you get in-body shake-reduction, two features that should be standard, but aren't. If it's important, vote for it with your credit card.
-
When? Because most modern batteries are Li-Ion, and have much greater charge density than anything that comes in AA-format. Moreover, since a single "classic" dry-cell is a 1.5V cell and a single Li-Ion is 2.4V, there will not be true Li-ion AA's... too much risk of burning electronics.
Yes, standard batteries would be wonderful. So would standard headlamps on cars. It's not going to happen, too much money to be made selling brand-specific batteries.
Non-standard RAW is a different story, it does nothing to make anyone any money that I can see...
There is a solution of course: buy a Pentax. You get DNG, you get in-body shake-reduction, two features that should be standard, but aren't. If it's important, vote for it with your credit card.
Great if you begin from scratch, but should you already own an armoury of very expensive glass and no way to write it off against business...
Rob C
-
DNG is a proprietary format and adds nothing of value in terms of workflow much less archival image data. The standard format that has been around for decades and remains as the only universal public domain format is TIFF. Any photo editing or graphics or press print application can work with a TIFF file and that cannot be said for any other format where version levels are not even compatible as with Adobe's other formats for Illustrator, et al.
-
DNG is a proprietary format and adds nothing of value in terms of workflow much less archival image data.
on the contrary - as an intermediate workflow format it is very useful actually, as an archival format of course not unless it is the original raw format of the camera or unless you also save the original raw file as it was written by cameras firmware... but to argue against using it in between the original raw (which may as well be DNG) and final output is useless - how about using optics correction in one software and passing otherwise un-WB/non-color transformed data further down the pipeline... DNG is a better solution for that vs a generic tiff (DNG is tiff, yes)
-
DNG is a proprietary format and adds nothing of value in terms of workflow much less archival image data.
How is it proprietary when the spec is openly documented? There are provisions for private tags yes, to hold actual proprietary data that DNG doesn't itself 'need' because, well it's some camera proprietary data. DNG add's huge value to my workflow. If it doesn’t for you, that's fine. But both your comments above hold zero value and are inaccurate to those of us who understand how this all operates. DNG is based on TIFF! Both are owned and controlled by Adobe. Consider that...
-
Hi,
Several good points.
DNG is owned by Adobe but so is TIFF, and as stated before DFG is a variant of TIFF. DNG is well specified and the specification is open.
AlterEgo makes a good point that DNG is a good vessel for handling lens corrections. Adobe offers a well thought out platform for handling metadata, lens corrections, camera profiling and other things. As pointed out by Anders Torger, it is far from perfect. But it is well defined. Would vendors use similar functionality on proprietary and undocumented raw formats we would have a great mess.
Now, proprietary raw formats are mostly based TIFF-PE (another Adobe controlled standard) without a lot of advanced features, so they can be read by many programs, although all programs probably cannot use all proprietary information in the files.
Best regards
Erik
on the contrary - as an intermediate workflow format it is very useful actually, as an archival format of course not unless it is the original raw format of the camera or unless you also save the original raw file as it was written by cameras firmware... but to argue against using it in between the original raw (which may as well be DNG) and final output is useless - how about using optics correction in one software and passing otherwise un-WB/non-color transformed data further down the pipeline... DNG is a better solution for that vs a generic tiff (DNG is tiff, yes)
-
DNG is a proprietary format and adds nothing of value in terms of workflow much less archival image data.
Well ya might wanna double check your facts...then get back to us.
-
Actually, if you want something like this done, the shortest path is to simply do it. I see lots of people standing around saying "someone ought to do a bunch of work here" but nobody actually stepping up to, you know, do it.
Just write a standard. Pull together a committee of experts, of which there seem to be a tremendous number. Promote it to standards bodies.
(http://imgs.xkcd.com/comics/standards.png)
Jeremy
-
DNG is a proprietary format and adds nothing of value in terms of workflow much less archival image data.
+1
Most if not all camera manufacturers won't adopt it because it severely restricts the ability to produce new features.
Hasselblad start using it in 2005 and dropped it within a year or so.
However if you want to do it go right ahead. Waste of time and money. My Mac OS reads all the raw files and has all the support I need for them, thanks.
-
Well ya might wanna double check your facts...then get back to us.
you mean the part that DNG used with a particular software from a particular vendor for conversion from non DNG to DNG is well known historically to drop either part of the data read off sensor or part of metadata or both, no ;D ?
-
+1
Most if not all camera manufacturers won't adopt it because it severely restricts the ability to produce new features.
Hasselblad start using it in 2005 and dropped it within a year or so.
However if you want to do it go right ahead. Waste of time and money. My Mac OS reads all the raw files and has all the support I need for them, thanks.
Got very little to do with an OS or anything that: reads all the raw files and has all the support I need for them! There are a good dozen or more capabilities of the format that support some people's workflow that has absolutely NOTHING to do with access to the raw data. Like data verification of the raw data and container! Perhaps you don't care about bit rot of your negatives, some of us do. Or the ability to embed a pretty large and useable JPEG of the rendered image 'just in case'. Or the ability to embed DNG camera profiles we build because we strive for better color and what that critical profile to be transportable. Or better performance thanks to fast load preview data.
+1 all you like, that's fine. But get the facts about the potential advantages to some of us straight before you and other's suggest DNG is a solution solely to aid accessibility of proprietary raw data; that's the tip of the iceberg.
-
+1
Most if not all camera manufacturers won't adopt it because it severely restricts the ability to produce new features.
Hasselblad start using it in 2005 and dropped it within a year or so.
That's little better than a factoid.
Haven't Leica gone downhill since they started using DNG? And Pentax?
-
There is a solution of course: buy a Pentax. You get DNG, you get in-body shake-reduction, two features that should be standard, but aren't. If it's important, vote for it with your credit card.
Yup! I did! Pentax K-30, then K-3. Every single lens has shake reduction as well! If they offered good raw-capable super compacts, I'd get those in a heartbeat. But they don't, so I had to go with Canon S100, then later a G7X.
And I agree about hearing people say "I will wait until something other than DNG, because it's 'controlled' by Adobe".. There seems to be nothing else developing though. Hmm..
The big reason why I throw away camera originals (Canon), even with the Pentax too.. Is 1) Image data verification! (saved me several times!) 2) I get to rid the raws of their useless (to me) full-size jpg previews - can save ~3-6MB per file with having no previews 3) Holds xmp inside for belt-and-suspenders backup of metadata & develop settings/snapshots (although could be considered bad by some because of backup reasons) 4) Can use lossless (EDIT: I meant lossy) DNG for those shots that you just can't quite seem to throw away. ;)
For me, there is absolutely no reason I'd even want to save camera originals anymore.. The verification part alone is valuable beyond belief!
P.S. Don't use a credit card - horrifically bad... Use a debit card instead. ;)
--
Die-Hard DNG Fanatic 8)
-
P.S. Don't use a credit card - horrifically bad... Use a debit card instead. ;)
Bad advice in the UK. You get a moderately comprehensive degree of protection against fraudsters when you pay by credit card and none if you use a debit card.
Jeremy
-
The big reason why I throw away camera originals (Canon), even with the Pentax too.. Is 1) Image data verification! (saved me several times!)
Hi,
I wonder, why did your files get corrupted? And if they were detected as being corrupt, how did you repair them, without the originals still available? And do you realise that the repeated writing of the DNGs, every time the file is edited and new XMP data has to be written, increases the risk of something going wrong ...
2) I get to rid the raws of their useless (to me) full-size jpg previews - can save ~3-6MB per file with having no previews
Useless to you, not useless to others. Even if the Raw data would get corrupted, one can often still extract the Preview. But I agree that it's nice to be able and write one's personally preferred preview size.
3) Holds xmp inside for belt-and-suspenders backup of metadata & develop settings/snapshots (although could be considered bad by some because of backup reasons)
Besides that that XMP data is only useful inside an Adobe application like LR/ACR, the required additional backups, even if the Raw data itself didn't change, but the XMP data might have been, the repeated writing of changed XMP data causes the entire file to be rewritten to disk. Besides causing potential disk fragmentation (and wear on SSDs), each write holds an inherent possibility of data corruption. Since the verification data is held inside the file, there is no way of verifying with absolute certainty that the file as such (including verification data) was written correctly. The verification data is calculated from the Raw data which was perhaps changed by memory module error during editing. Only when reading the file next time, is the verification data compared (assuming the file as such was readable), and since it was calculated from potentially corrupted data, it will report that everything is fine. Even rewriting the verification data itself seperately over and over again is bad for security. Once calculated and verified, one should not keep recalculating and writing it, it only adds to the risk of something going wrong. Maybe that's what you've experienced?
4) Can use lossless DNG for those shots that you just can't quite seem to throw away. ;)
You mean that you normally write 'lossy' compressed DNGs? Or do you mean that it is possible to encapsulate the original Raw inside a DNG wrapper? The latter seems a bit of a convoluted cludge, why not just keep the camera original? It will get converted on the fly anyway when it's read by LR/ACR.
Camera original Raws have other benefits, such as for some Canon files they can hold dust removal data, and they can be lens corrected with the DPP application very well, and the colors are often reported as superior. With the advent of sensor shift technologies to improve the Bayer CFA color accuracy per pixel, and resolution enhancing half pixel shifts, it gets increasingly difficult to store such solutions in a DNG wrapper. We've seen it with Foveon file formats, and also for Digital camera backs that store a lot of additional (blackframe and other calibration data, and need LCC corrections), we'll see it with the Lytro kind of solutions, stereo file formats, etc.
For me, there is absolutely no reason I'd even want to save camera originals anymore.. The verification part alone is valuable beyond belief!
It seems you might need to give that some more thought. While the verification feature is not entirely without merit, it does give a false sense of confidence. But then that happens more often with (DNG) fanatics ...
Cheers,
Bart
-
It seems you might need to give that some more thought. While the verification feature is not entirely without merit, it does give a false sense of confidence. But then that happens more often with (DNG) fanatics ...
Silly name calling.... But you expect that from DNG phobics, don't you?
-
Silly name calling.... But you expect that from DNG phobics, don't you?
John, you are mistaken. Diehard DNG fanatic is "Hoggy's" signature. No name calling on my part.
DNG phobic? I rest my case.
Cheers,
Bart
-
John, you are mistaken. Diehard DNG fanatic is "Hoggy's" signature. No name calling on my part.
DNG phobic? I rest my case.
Cheers,
Bart
OK, Bart, I accept that it wasn't name calling on your part! But I do think you approach his post from a phobic position. Without going through every point you made, XMP metadata isn't only useful inside Adobe apps - it's a standard that many other apps read. They can even read Adobe's adjustment data (eg Mylio, I think, does it). Repeated writing of metadata doesn't increase backup demands if you use Lightroom - it just means you should fine tune your backup scheme. The DNGs to which metadata is repeatedly-written don't really need backup when you have a catalogue backup and the virgin DNGs. And so on. Verification balances the risks of repeatedly writing to the DNGs, if you even bother doing that. Would anyone seriously argue for a proprietary format like PSD over TIF? Same with raw data.
By lossy DNG, he means that he saves some less-valuable files in the lossy DNG format. Personally I don't do it - space is cheap - but I do use lossy DNG for photos that I may have captured as JPEG since it helps identify the files as originals and avoids overwriting errors. Lossy DNG is totally different from encapsulating the raw files in the DNG wrapper, which I agree isn't ideal. It may be convenient if you ever want the raw file again, but I feel it's better to archive the raw files separately.
John
-
OK, Bart, I accept that it wasn't name calling on your part! But I do think you approach his post from a phobic position.
John, I'm just being critical, push back, when people tend to believe everything they get spoonfed, not phobic (I don't get scared that easily). I try to always keep an eye open for better solutions, and I can tell you, there are no perfect solutions. There is always something that can be improved, or we simply need to accept that perfection is just not possible. Good enough may be acceptable if near perfection becomes a prohibitive effort.
Without going through every point you made, XMP metadata isn't only useful inside Adobe apps - it's a standard that many other apps read. They can even read Adobe's adjustment data (eg Mylio, I think, does it).
While true, the XMP format standard (https://en.wikipedia.org/wiki/Extensible_Metadata_Platform) is open enough to allow describing one's own metadata properties, e.g. only meaningful for Adobe LR/ACR, not necessarily relevant (and thus ignored) by other applications.
Repeated writing of metadata doesn't increase backup demands if you use Lightroom - it just means you should fine tune your backup scheme. The DNGs to which metadata is repeatedly-written don't really need backup when you have a catalogue backup and the virgin DNGs. And so on.
The issue is that a Camera original Raw, as long as it is not altered one bit, will not be backed up again and again, if intelligent backup software is used. This will reduce the amount of data (and time) that needs to be transferred to sometimes off-site (e.g. cloud) storage. When a DNG with embedded XMP is edited, not only the small XMP section, but the entire DNG file (preview/Raw data/Makernotes/XMP) needs to be backed up. By separating the XMP data from the DNG and storing it in a catalogue (backup) creates other issues, and solves nothing for non-Adobe software.
Verification balances the risks of repeatedly writing to the DNGs, if you even bother doing that.
In fact, it makes it almost a necessity. Reading and rewriting the Raw data increases the risk of corruption, so some form of verification becomes mandatory.
Would anyone seriously argue for a proprietary format like PSD over TIF? Same with raw data.
I don't follow the analogy. DNG is merely a TIFF type of wrapper. PSD is even discouraged (unless mandatory for a few very specific uses, like displacement maps) by Adobe because TIFF is much more standardized and the library is maintained openly, mainly by others than the formal owner Adobe. And with the advent of the Big-TIFF standard, PSB's will become irrelevant as well.
Cheers,
Bart
-
While true, the XMP format standard (https://en.wikipedia.org/wiki/Extensible_Metadata_Platform) is open enough to allow describing one's own metadata properties, e.g. only meaningful for Adobe LR/ACR, not necessarily relevant (and thus ignored) by other applications.
Not sure what you are trying to say here. Yes, xmp permits each vendor, even you or I, to add metadata that is meaningful to only ourselves and make it readable by others if they choose to do so. If your "that that XMP data" means Adobe's adjustment metadata, any app can read and use it, and I think Mylio does, PM may do so (at least for crops), maybe C1 too....
Using DNG and updating the embedded metadata doesn't mean you run into problems with off-site storage - only if you fail to fine tune your backup scheme, which is surely what one should do for every situation. You get 100% backup coverage if you backup virgin DNGs once, plus the Lightroom catalogue routinely, and writing metadata into the catalogued DNGs and backing up those rewritten files to the cloud is purely a matter of choice. That choice can be because someone wants more than 100% backup coverage, but even if it results from their failure to review backup needs to take advantage of a DNG workflow, the cost of such excessive backup is lower and lower.
Verification just balances any risk of rewriting files that don't even need to be rewritten.
I would have thought the analogy is pretty obvious without going down the rabbit holes of ultra-large files. What sensible person would prefer a proprietary format over a non-proprietary one?
-
Well, John has already gone over most of what I would've replied with.. And yes, I meant 'lossy'. I've only done lossy on a few so far, seeing as sometimes there isn't a whole lot of space savings there anyways - the amount can vary widely I've discovered.
The 'sig' was supposed to be joke.. It's not even my actual sig - I have none.
As to why some files got corrupted, I don't know how. I used to keep them on an external, but have since moved them to this laptop's spinner HD - the SSD also on here is not really meant for storing image files, as is probably the case for most people (besides, my understanding is that not the whole file would need to be rewritten anyways, just some blocks - otherwise we'd be in a sorry state for our LR/C1 catalogs & previews for sure). I never had the 'automatically write changes into XMP' checked. I do however write it out to the DNG after I change any amount I don't want to lose (for belt&suspenders). However for me that could be after even one picture as I have issues with a disability. The catalog is still the main priority though. As to how I recovered those files, my library is small enough that I store multiple complete backups - so I just went back and checked for a good copy.
There is a way in LR, to validate files - which doesn't take very long at all. So I now always do a verification before doing a full backup. I've also converted my old jpgs to DNG for the file verification bit as well.
As to not being able to use the manufacturer's software, I don't care anymore. My disability also affects cognition & memory, so I don't really have it in me to learn the software of every single maunfaturer's camera I've owned or will come to own. If there would've been one, it would've been the Pentax as I initially liked the colors of the original jpgs on it. However I much prefer to concentrate on the software that does many - like LR & C1. I've found I can get much better results with those options - maybe not the same, but that's a rather pointless endeavor as one can do far better with the likes of LR/C1.
The whole point behind DNG is it's made to be extensible. And hopefully it will become less Adobe-centric in time. As to that end, there was a very good conversation on ways to improve it just below, even if most of it was over my head. ???
-
The issue is that a Camera original Raw, as long as it is not altered one bit, will not be backed up again and again, if intelligent backup software is used. This will reduce the amount of data (and time) that needs to be transferred to sometimes off-site (e.g. cloud) storage. When a DNG with embedded XMP is edited, not only the small XMP section, but the entire DNG file (preview/Raw data/Makernotes/XMP) needs to be backed up. By separating the XMP data from the DNG and storing it in a catalogue (backup) creates other issues, and solves nothing for non-Adobe software.
In fact, it makes it almost a necessity. Reading and rewriting the Raw data increases the risk of corruption, so some form of verification becomes mandatory.
It's actually because C1 doesn't save into a DNG that prevents me from fully exploring the C1 app. That belt & suspenders backup is very important to me. While the catalog version is still the most important up-to-date version of keywords & develop settings/snapshots, it's nice to know that it's also stored in the files themselves should there be some catastrophic catalog loss requiring re-reading the files. Unlikely, but...
And one does not need whatsoever to continually backup the DNG's if they don't wish to, or if it's impractical. Only if one wishes to have the belt-and-suspenders reassurance.. And I do, as my library isn't that big right now. There is nothing preventing anyone to rely on solely the catalog for any and all future metadata updates (I'm including develop settings here, as it's still metadata).
It should also be possible to store, for example, Capture One edits alongside of LR edits & snapshots. As far as I know there is nothing preventing that. And in fact I wish that would actually happen. There is also a provision to allow multiple jpg previews if one wishes that, as well.
Let's also not forget about plain ol' bitrot - there's simply no easy way to double-check whether or not a camera original is wholly intact, save for files from eg. Pentax. I suppose one could use something like Quickpar to double-check and create repair blocks for originals should they become bad, but that route is kind of a PITA. I have thought about doing that though..
... Different strokes for different folks ...
-
Bad advice in the UK. You get a moderately comprehensive degree of protection against fraudsters when you pay by credit card and none if you use a debit card.
FWIW, same in the US. Little if any protection with debit cards.
-
All I know is that when I switched to Capture One Pro (8), my DNG files were not very well received. I wish I had left them all in the original RAW format (NEF). I get much better processing with C1 when using NEF as opposed to DNG.
-
C1's business decision, Mike.
-
... And exactly why I take issue with Capture One. Plus they're behind the times on several things. Of course, not the least of which is the DAM features compared to LR - although I understand they're working on that.
However DNG is still workable in C1.. I still don't regret conversion.
And from what I understand, they're finally getting into the DNG era, even if they're kicking, screaming, and dragging their feet about it.
-
C1's business decision, Mike.
That's a bit too simple, John, but expected from a Lightroom fan like yourself. There are also technical reasons (e.g. different color and distortion correction models) that make it less obvious with some cameras to convert from the Adobe color model encoded in the DNG to the (considered by many as better) Capture One color model.
Despite that, Phase One's Lionel Kuhlmann has stated (in this thread (http://forum.luminous-landscape.com/index.php?topic=102186.msg841495#msg841495)) that they do intend to improve the DNG conversion quality. It's appreciated that they do spend their limited resources to work at that improvement, even though it's not really needed if one uses the (non-DNG) Camera original Raws.
Anyway, it has already been discussed in this thread (http://forum.luminous-landscape.com/index.php?topic=104627.msg859809#msg859809).
Cheers,
Bart
-
That's a bit too simple, John, but expected from a Lightroom fan like yourself. There are also technical reasons (e.g. different color and distortion correction models) that make it less obvious with some cameras to convert from the Adobe color model encoded in the DNG to the (considered by many as better) Capture One color model.
Wait, if someone desires a DNG workflow and a company can't provide a proper and competitive handling of that data, that company shouldn't handle that data at all or in the first place. The customer should not buy a product from that company.
Or are you suggesting that the technical issues are shared by every and all raw converters that accept the processing of DNG data? Do you have to be a Lightroom 'fan' to expect a serious answer or do you have to be a C1 'fan' to accept flaws in their processing of some data?
-
Wait, if someone desires a DNG workflow and a company can't provide a proper and competitive handling of that data, that company shouldn't handle that data at all or in the first place. The customer should not buy a product from that company.
Or are you suggesting that the technical issues are shared by every and all raw converters that accept the processing of DNG data? Do you have to be a Lightroom 'fan' to expect a serious answer or do you have to be a C1 'fan' to accept flaws in their processing of some data?
Andrew, We've discussed this before (also see the threads mentioned earlier). I'm not going to rehash that discussion, again. Don't know about you, but I have better things to do.
Cheers,
Bart
-
Andrew, We've discussed this before (also see the threads mentioned earlier). I'm not going to rehash that discussion, again. Don't know about you, but I have better things to do.
Exactly what I'd expect from a C1 'fan'. Good luck John and Mike, best thing to do is view Bart's last post and his inability to prove his points about DNG (again).
-
Hi Bart,
Reading DNG doesn't mean that they would need to utilize the colour rendition algorithms in DNG, just threat it as another raw format.
I would think the distortion model in C1 is not using parameters supplied in the raw data. If say Nikon would supply distortion correction parameters in NEF it may be different from what Canon would supply in CR2.
But the main issue for me is that I feel that having hundreds of slightly different image file formats instead of one is not very sane. Also, all code supporting those file formats needs to supported over a long time.
At this time I think dcraw support 647 different cameras. Now, it is just 9000 lines of code. That is maintainable, I guess, but I would say that coding effort would be smaller and more future safe would a standard coding be used.
I don't think DNG is the perfect solution, but I think it is a decent one.
Just to say, I have some reservations about storing xmp-data in the DNG file, as it causes frequent rewrites of the file. On the other side I feel that embedding processing data in the DNG-files is a great idea. Obviously, it is not much help for programs not using Adobes processing pipelines. But some information is usable regardless, say crop,star ratings, keywording etc. So, I see both risk and damages. The integrity algorithm in DNG is obviously a good thing, but it would need to be used any time the DNG file would be updated.
Best regards
Erik
That's a bit too simple, John, but expected from a Lightroom fan like yourself. There are also technical reasons (e.g. different color and distortion correction models) that make it less obvious with some cameras to convert from the Adobe color model encoded in the DNG to the (considered by many as better) Capture One color model.
Despite that, Phase One's Lionel Kuhlmann has stated (in this thread (http://forum.luminous-landscape.com/index.php?topic=102186.msg841495#msg841495)) that they do intend to improve the DNG conversion quality. It's appreciated that they do spend their limited resources to work at that improvement, even though it's not really needed if one uses the (non-DNG) Camera original Raws.
Anyway, it has already been discussed in this thread (http://forum.luminous-landscape.com/index.php?topic=104627.msg859809#msg859809).
Cheers,
Bart
-
Hi Bart,
Reading DNG doesn't mean that they would need to utilize the colour rendition algorithms in DNG, just threat it as another raw format.
Hi Erik,
Yes, and that's what I suggested in one of the other threads (http://forum.luminous-landscape.com/index.php?topic=102186.msg841935#msg841935) earlier. They could have just used the actual Raw date part of the DNG wrapper. But I think that they took another approach early on (presumably to conform to the DNG SDK), which now becomes much harder to translate towards the C1 color model.
When relying on what the DNG metadata suggests, e.g. an exposure bias, they now have to know that that is being enforced (but not documented how that bias was calculated/assumed/derived), then reverse that unknown exposure bias, and only then think of applying their own color model and demosaicing on the original Raw data. Not that it would be impossible to do, but it takes a lot of work/resources to figure out for all the cameras that are supported.
I would think the distortion model in C1 is not using parameters supplied in the raw data. If say Nikon would supply distortion correction parameters in NEF it may be different from what Canon would supply in CR2.
Most likely they would use a different system to characterize the distortion, indeed. But that doesn't mean that Phase One has not been in contact with the bigger manufacturers. So maybe they do know (some) of the secrets, or they may have reverse engineered some, who knows?
But the main issue for me is that I feel that having hundreds of slightly different image file formats instead of one is not very sane. Also, all code supporting those file formats needs to supported over a long time.
For an enduser it should not be an issue. For a software developer maybe, but once the Raw format for a given camera is decoded, not much will change unless one starts decoding in a completely different/novel way. The problem with DNG is that it is yet another version of the Raw data that was already decoded in its original form and, since Adobe controls the DNG format, with any newer version of the DNG spec it needs to be verified to see if it changes anything that affects earlier work. So the DNG may be less static than the old Raw file format for a given camera (which will stay the same for 'eternity'). Newer Cameras with newer features will get their own permanent format. No backward compatibility issues will be created, the older formats stay unchanged.
Just to say, I have some reservations about storing xmp-data in the DNG file, as it causes frequent rewrites of the file. On the other side I feel that embedding processing data in the DNG-files is a great idea. Obviously, it is not much help for programs not using Adobes processing pipelines.
That's what I was pointing out, and it makes DNG a too much Adobe centric solution for it to become a universal format. In addition, as Anders Torger commented on with regards to his DCP profiling solution, there are parts of the DNG spec that are not documented. That doesn't help it getting universal adoption either.
Cheers,
Bart
-
Just to say, I have some reservations about storing xmp-data in the DNG file, as it causes frequent rewrites of the file. On the other side I feel that embedding processing data in the DNG-files is a great idea. Obviously, it is not much help for programs not using Adobes processing pipelines. But some information is usable regardless, say crop,star ratings, keywording etc. So, I see both risk and damages. The integrity algorithm in DNG is obviously a good thing, but it would need to be used any time the DNG file would be updated.
But isn't it possible that other converters could add their instructions alongside of LR adjustments (or any other converter, for that matter)? I thought it was, but so far I don't know of any that do.
Cause that's one thing that I hope Capture One could start doing soon.. Storing it's adjustments (& variants) alongside of LR adjustments, in my case. Of course I realize they wont understand each other's adjustments - but that's a good thing IMO.
I know some people may not want to write into the DNG, and they absolutely do NOT have to do so - but I like that ability, personally. That's an extra reason the DNG hash can come in handy. And that's also why I think the option in LR to "automatically write changes into XMP" is a Bad Idea - cause that does cause frequent rewrites. Personally, I prefer to do it in one fell swoop after a 'session'.
-
But isn't it possible that other converters could add their instructions alongside of LR adjustments (or any other converter, for that matter)? I thought it was, but so far I don't know of any that do.
Correct, they could, and they could even encrypt it if they wanted. They can also add their own embedded previews and thumbnails.
Frequent rewrites of the file don't matter. The "virgin" DNGs and the catalogue are your backup, not these "working" DNGs.
John
-
Frequent rewrites of the file don't matter. The "virgin" DNGs and the catalogue are your backup, not these "working" DNGs.
Hi John,
I'm not sure I understand that. Do you mean keeping an original DNG (conversion in the case of a different virgin Camera Raw format), and then do the actual work on a separate DNG? How is that initial DNG conversion a benefit then, over the Camera Raw original?
Besides, a DNG conversion is not really virgin Raw data anymore, it got somewhat impregnated (transformed or even added) with Adobe style tags. From a data integrity/security point of view, every (additional) conversion/rewrite of data is subject to adding potential errors/glitches. It usually goes well, but then airplanes also usually don't fall from the sky, until they do.
Accidents happen, especially if the opportunity is created or frequency is increased. People probably do not really understand that our hard disks use lots of error correction to restore corrupted data reads/writes without us knowing it. The massive amount of error correction is there for a reason, and gets more relied upon as miniaturization progresses.
Cheers,
Bart
-
I'm not sure I understand that. Do you mean keeping an original DNG (conversion in the case of a different virgin Camera Raw format), and then do the actual work on a separate DNG? How is that initial DNG conversion a benefit then, over the Camera Raw original?
Yes, I do mean backing up the original or "virgin" DNG upon its creation from the raw file and treating that backup as write once (like optical discs). The file that is in your catalogue (ie the file from which you had made the backup, so no big effort) just doesn't require repeated backup, no matter how much you write to it, because you have the virgin DNGs and your routine catalogue backups, giving you 100% coverage of the images and the work on them. Nothing wrong with backing up the "working DNGs" if you want, but if you don't want to do so, it's no reason to avoid DNG.
As for advantages, some would say file size - true, usually, though in my view it's more like a happy side effect. Another minor advantage (minor since we're really talking about raw files) is when you use DNG to contain non-raw originals like TIF scans or mobile JPEGs and offer them protection from being overwritten or being mistaken for derivative files. For me the bigger gains come from working initially on the raw files, culling them, making general or initial adjustments and adding initial metadata - and then converting the keepers to DNG. So my virgin DNG archive consists of files with embedded previews which are usable outside raw converters and, unlike adjustments to raw files, display a corrected image. Adobe adjustment metadata, which some 3rd party apps can read, and IPTC metadata are embedded in the files and therefore more likely to be read by more 3rd party apps than if they had been in a sidecar. And that's putting aside the long term archival benefits of DNG being a publicly-documented format, which you may not believe anyway.
John
-
The road to DNG is two-way. Or, to put it differently, most of you guys (including the OP) are barking at the wrong tree.
First off: I am all for standards, and yes, I would love to see DNG universally excepted. But let me explain what needs to happen before you can even start asking Sony et. al. to adopt it.
The responsibility for maintaining the standard has to be fully transferred to an international non-profit standards organization, such as e.g. ECMA (http://www.ecma-international.org). A good test of how "full" is the transfer should be: if Adobe ever wants to make a change to the standard, they would have to go through exactly the same procedure as any other ECMA's (or other organization's) DNG committee member. THEN we can call it an open standard.
Those who call DNG non-proprietary today have no idea what they are talking about. Yeah, tell Google, who are currently being sued by Oracle for $1Bn, how open and non-proprietary Java standard is. BTW, I directed development of a JVM variant for a proprietary OS in very early 2000th, so I do have some idea what that all is about.
As was pointed out in this thread, there was a very useful post by an active RAW developer who pointed out several deficiencies in DNG documentation. An esteemed forum member (whom I fully respect for his photography-related work) answered something like "You can always ask Thomas Knoll". Are we going to also tell that to Sony's CTO? You know what, I can't even make fun of that. For anyone having a slightest idea of what we're talking about here, it already sounds as funny as it gets.
So, if you are really, really interested in wide adoption of DNG: put pressure on Adobe to let go of DNG before you even start talking to camera manufacturers.
Now, suppose that miracle did happen, and Adobe did pass DNG standard into the little hot hands of ECMA (or similar organization). Will it then make sense for Canon/Nikon/Sony/etc. to adopt it?
I do not know. What is Sony going to do with its lossy RAW compression (which I believe is simply brilliant; your mileage may vary, please do not derail this thread...)? Also submit it to DNG? What if they invent new compression, how the necessity of jumping through ECMA hoops will affect their time-to-market?
These, and many other questions need to be answered before any groundbreaking DNG-related decisions benefiting us photographers can really be taken... So far, I saw that many forum members just dismiss these questions as unimportant. Well, you shouldn't -- if you are really, really interested in wide acceptance of DNG, and not just in talking about it.
-
You needn't pry it loose from Adobe, you merely need to write down what you know of DNG, write down some more material clarifying that, and run with the result.
This is something which I have pointed out several times, and which none of the loudest complainers seem willing to actually do. The great thing about standards is that anyone can write one. Just write it and pull together a coalition of influential people to get behind it. Everyone necessary for this is already here in this thread on this web site. And yet, oddly enough, they seem to prefer that someone else do the work.
Well, me too, to be honest. I hate work.