Pages: 1 [2] 3   Go Down

Author Topic: To DNG or not to DNG  (Read 20182 times)

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
To DNG or not to DNG
« Reply #20 on: February 23, 2009, 12:06:00 am »

Quote from: budjames
I do burn the original RAW files to DVDs before converting to DNG and deleting them from my LR 2.2 database.

I just converted a bunch more CR2 files and the resulting DNGs had the "?" in the upper right corner indicating that the sidecar file is missing. Since they are DNG files, I don't understand why LR is looking for a sidecar file in the first place.  

With this batch of converted files, clicking on the "?" made it disappear from the thumbnail on about 50% of the images, but it's a pain to click on each file one at a time. Is there a way to accomplish this in a batch mode?

Also, the original files that started me on this thread, the "?" doesn't disappear when I click on the DNG frame in grid mode. Any more suggestions there to get ride of the erroneous "sidecar file missing" message on my converted DNG files?

Thanks.
Bud James
Not sure of the precise answer but have you tried deleting the RAW files using Finder/Explorer (as opposed to in LR), and then Synching the folder in LR to update metadata and file listing?
Logged
Nick Rains
Australian Photographer Leica

budjames

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 956
    • http://www.budjamesphotography.com
To DNG or not to DNG
« Reply #21 on: February 23, 2009, 06:30:30 am »

Quote from: Nick Rains
Not sure of the precise answer but have you tried deleting the RAW files using Finder/Explorer (as opposed to in LR), and then Synching the folder in LR to update metadata and file listing?

Nick, what I tried before your post was to copy the folder out to another location, delete the folder in LR and the reimport. That made the "?" mark disappear. However, I have thousands of images inside dozens of folders, to I was hoping for an easier and faster solution.

Cheers.
Bud James
Logged
Bud James
North Wales, PA [url=http://ww

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
To DNG or not to DNG
« Reply #22 on: February 23, 2009, 07:51:40 am »

Quote from: budjames
Nick, what I tried before your post was to copy the folder out to another location, delete the folder in LR and the reimport. That made the "?" mark disappear. However, I have thousands of images inside dozens of folders, to I was hoping for an easier and faster solution.

Cheers.
Bud James
OK, but try the synch feature anyway, you never know...
Logged
Nick Rains
Australian Photographer Leica

James R

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 364
To DNG or not to DNG
« Reply #23 on: February 23, 2009, 12:02:35 pm »

My understand is converting a raw file into a dng will ensure that Adobe products will be able to open the file?  Beyond that, you have no certainty.  Also, any file adjustments will not be read by Adobe if they are made by a non-Adobe program.  I have been saving my worked files as tif's and I also have the raw version converted as a dng.

I never worried too much about the file format used to save the original file.  My concern was always saving the final version, which has all of my final adjustments, including PS layers.  I decided tif files were more likely to be supported by future programs than any of the newer formats.


Logged

Farkled

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 97
To DNG or not to DNG
« Reply #24 on: February 25, 2009, 04:31:38 am »

I started out life in the digital world thinking that DNG would be the "salvation."  I don't use the format now, as much as I would like to, because:

1. LR & ACR seem to be indifferent to RAW file format
2. Some other converters I use cannot take DNG as inputs
3. Using DNG (today) is an unnecessary burning of bridges.
4. I don't have a sidecar issue with CR2 files because I store the metadata in the database (backed up daily)
5. Adobe is no more or less likely to survive than are either Canon or Nikon
6. If it becomes needful, one can always convert to DNG down the road
7. With billions of CR2 files out there, converters will always be there.
8. In 10 years (maybe more, maybe less), I believe that some new technology will render all current file formats obsolete and that we   will be migrating all existing files to that new technology and this discussion will be meaningless.

Nonetheless, I do wish that Nikon and Canon would give up their proprietary formats and go with DNG.  It would make their customer's lives so much easier to say nothing of richer.  Software publishers can provide more features for the same dollar if they don't have to decode new file formats every 30 days.
Logged

Josh-H

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2079
    • Wild Nature Photo Travel
To DNG or not to DNG
« Reply #25 on: February 25, 2009, 05:00:38 am »

Quote from: Farkled
I started out life in the digital world thinking that DNG would be the "salvation."  I don't use the format now, as much as I would like to, because:

1. LR & ACR seem to be indifferent to RAW file format
2. Some other converters I use cannot take DNG as inputs
3. Using DNG (today) is an unnecessary burning of bridges.
4. I don't have a sidecar issue with CR2 files because I store the metadata in the database (backed up daily)
5. Adobe is no more or less likely to survive than are either Canon or Nikon
6. If it becomes needful, one can always convert to DNG down the road
7. With billions of CR2 files out there, converters will always be there.
8. In 10 years (maybe more, maybe less), I believe that some new technology will render all current file formats obsolete and that we   will be migrating all existing files to that new technology and this discussion will be meaningless.

Nonetheless, I do wish that Nikon and Canon would give up their proprietary formats and go with DNG.  It would make their customer's lives so much easier to say nothing of richer.  Software publishers can provide more features for the same dollar if they don't have to decode new file formats every 30 days.

Well said - +1 for me to the above.
Logged
Wild Nature Photo Travel

Pete Ferling

  • Guest
To DNG or not to DNG
« Reply #26 on: February 26, 2009, 09:01:39 pm »

The beauty and true usefulness of DNG reveals itself when you share the same files amongst other members of a production team.  DNG is also recognized in many familiar DAM applications, such as cumulus which my employer uses.   Because cumulus works with DNGs, our clients and marketing managers have full access to the raw files, which cumulus faithful converts to jpegs or other formats as they request.  All done repeated from the same master file.  A few managers also have PS CS3 and they love the ability of having a virtual minilab on their desktop to develop and tweak their own versions.  The benefit is a huge time savings for our designers.
Logged

milt

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 70
    • Striking & Distinctive Custom Photographic Prints
To DNG or not to DNG
« Reply #27 on: March 10, 2009, 08:03:01 pm »

This thread has been interesting reading.

I've been wondering whether I should be using DNG myself, but there is a sort of fundamental problem about DNG that has been bothering me.  Maybe somebody here knows the answer.

The "meaning" of the numeric pixel values in a RAW file (i.e. the correspondence between the captured photons and the numeric values) are relative to the engineering properties of the particular sensor (and its supporting electronics).  These engineering properties are not only different for each different camera, they are regarded as proprietary by (I think all) camera manufacturers.  Anybody that makes any kind of a RAW converter has know a lot about these sensor properties for each camera.  (If you are the camera manufacturer, you presumably know everything you need to know and can put that knowledge into either the in-camera firmware RAW converter or your own proprietary software RAW converter.  Everybody else has to do some kind of reverse engineering in order to write a RAW converter.)

Now, lets look at the image data in a DNG file (lets say one where the RAW data is not embedded). I can think of two possibilities:

A) The numbers in the image data pixels are copied exactly from the raw file.

B) The numbers in the image data pixels have undergone a conversion to some kind of a universal grey-scale space.

If A), then DNG files still need to undergo RAW conversion in a camera-specific way, and are thus not really "universal".  If B), then the DNG files are no longer camera-relative, but ARE relative to the quality of the knowledge that the convert-to-DNG software had about the camera (and, some might argue, are not really "RAW files" anymore).

Anybody know whether its "A" or "B" (or something else)?

Thanks.

--Milt--
Logged
Los Gatos, California | http://miltonbarber.com

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
To DNG or not to DNG
« Reply #28 on: March 10, 2009, 08:55:57 pm »

Quote from: milt
The "meaning" of the numeric pixel values in a RAW file (i.e. the correspondence between the captured photons and the numeric values) are relative to the engineering properties of the particular sensor (and its supporting electronics).  These engineering properties are not only different for each different camera, they are regarded as proprietary by (I think all) camera manufacturers.  Anybody that makes any kind of a RAW converter has know a lot about these sensor properties for each camera
What you mean to be so important is the spectral responses of the color filters. The Adobe DNG converter adds some metadata to the raw file, which describe these characteristics. It is disputed if this approach is equivalent to the camera makers' own raw conversion, but *now, with the customizable profiles*, Adobe's conversion became better re the colors.

The vast majority of users I have read from are using ACR for the integrated workflow and for the adjustment and color space options; some are occasionally turning to the "home" raw converter in color critical cases, but I guess that will become much less with the new profiles.
Logged
Gabor

sandymc

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 350
To DNG or not to DNG
« Reply #29 on: March 11, 2009, 11:46:09 am »

Quote from: milt
This thread has been interesting reading.

I've been wondering whether I should be using DNG myself, but there is a sort of fundamental problem about DNG that has been bothering me.  Maybe somebody here knows the answer.

The "meaning" of the numeric pixel values in a RAW file (i.e. the correspondence between the captured photons and the numeric values) are relative to the engineering properties of the particular sensor (and its supporting electronics).  These engineering properties are not only different for each different camera, they are regarded as proprietary by (I think all) camera manufacturers.  Anybody that makes any kind of a RAW converter has know a lot about these sensor properties for each camera.  (If you are the camera manufacturer, you presumably know everything you need to know and can put that knowledge into either the in-camera firmware RAW converter or your own proprietary software RAW converter.  Everybody else has to do some kind of reverse engineering in order to write a RAW converter.)

Now, lets look at the image data in a DNG file (lets say one where the RAW data is not embedded). I can think of two possibilities:

A) The numbers in the image data pixels are copied exactly from the raw file.

 The numbers in the image data pixels have undergone a conversion to some kind of a universal grey-scale space.

If A), then DNG files still need to undergo RAW conversion in a camera-specific way, and are thus not really "universal".  If , then the DNG files are no longer camera-relative, but ARE relative to the quality of the knowledge that the convert-to-DNG software had about the camera (and, some might argue, are not really "RAW files" anymore).

Anybody know whether its "A" or "B" (or something else)?

Thanks.

--Milt--

The answer is (A), the pixels are copied exactly. What is added (at least in DNG V1.1) are a pair of color conversion matrixes that describe how to get from "sensor space" to "image space" (plus some other bits and pieces).  Those matrixes are Adobe developed, although usually with input from the camera manufacturers (with some exceptions!). But there isn't that much secret or complicated about them; pretty much every raw developer does their own color conversion firstly to get their own look-and-feel, but secondly because the color conversion is intimately tied to other things, e.g., color temperature adjustments.

In DNG V1.2 things are little more complicated - you can have embedded profiles that allow a lot more flexibility in manipulating color reproduction, but the principle remains the same.

Sandy
Logged

paulbk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 528
To DNG or not to DNG
« Reply #30 on: March 11, 2009, 08:51:56 pm »

I speak only for me.. Canon 1D mark 2, mostly landscape
Currently the ‘perceived benefits’ of DNG do not justify the problems created. I do not convert to DNG.

For serious shots that may make a decent large print I use DPP (v3.5.1). I think it works best for getting the most out of the raw file. After raw convert I use photoshop for post and print.

I use lightroom for processing event shoots with hundreds of shots. Use: web, email, or small prints (8x10 max).

The benefit of DNG goes to 3ird party software developers. They don’t have to keep up with the growing list of OEM raw formats.

The problem with DNG is the extra time and space, with no benefit to me. I can convert to DNG later on an as-needed basis. I keep the native raw files for the same reasons others have provided above.

DNG is good idea if and when it’s adopted universally by OEMs. Not yet.
Logged
paul b.k.
New England, USA

Bill Caulfeild-Browne

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 313
    • http://www.billcaulfeild-browne.com
To DNG or not to DNG
« Reply #31 on: March 11, 2009, 09:00:44 pm »

Phase One users beware - if you convert your Phase files to DNG, Capture One will not be able to "recognize" the lens you used and apply its automatic lens corrections!
I no longer convert my P45+ files to DNG.
Bill
« Last Edit: March 11, 2009, 09:01:31 pm by billcb »
Logged

milt

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 70
    • Striking & Distinctive Custom Photographic Prints
To DNG or not to DNG
« Reply #32 on: March 12, 2009, 02:52:09 pm »

Thanks to Panopeeper and sandymc for the replies on the nature of DNG files.

Keeping the original bits plus a camera-specific "conversion rule" is better than either (A) or (B).  The files have the advantage of (A) of being universal, even if (like both (A) and (B)) they are also dependent on the quality of the knowledge that the convert-to-DNG software had about the camera.  However, this scheme means one can presumably later improve things if better camera-specific knowledge becomes available.

--Milt--
« Last Edit: March 12, 2009, 02:52:50 pm by milt »
Logged
Los Gatos, California | http://miltonbarber.com

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
To DNG or not to DNG
« Reply #33 on: March 12, 2009, 06:22:57 pm »

For a production environment there are a couple of useful features of DNG files over RAWs like CR2 or NEF.

1. They are slightly smaller if you don't include the original RAW data - not necessary if you back up the original RAWs 'just in case'.

2. The embedded preview properly reflects any adjustments made which is essential if you need to build web galleries using any of the DAMs. It's also faster since the corrected jpegs are already created and available to 3rd party apps.

3. For fast lo-res file generation ( eg submission to client for approval) using DNGs and RAW Extractor or Photomechanic is way faster than fully processing the original RAWs to jpegs in ACR, LR or C1Pro. The proccessing has already been done, all you need to do is pull out the jpeg preview.

Total time taken in these tasks is the same for DNG or RAW, the difference is when you spend the time. There;s nothing worse than having to process out a bunch of RAWs to a tight deadline. Better to convert to DNG right after the shoot, file in a DAM and then you can produce files quickly down the track since the time consuming RAW processing step has already been done. It obviously has to be done at some point, I just prefer to do it once at the beginning rather than as needed later when I may be in a hurry.

The geeky side of DNG does not interest me, it's the practical aspects that make me a user.
« Last Edit: March 12, 2009, 06:24:16 pm by Nick Rains »
Logged
Nick Rains
Australian Photographer Leica

Pete Ferling

  • Guest
To DNG or not to DNG
« Reply #34 on: March 13, 2009, 10:25:44 pm »

Crash or corrupt your LR database.  Just start a new one and re-import the DNGs.  All edits preserved.  Just came back from a week in Vegas, my companies National Sales Meeting.  2000 shots cut to 475 keepers, all with edits, key words and names with captions.  Copied the DNGs to a backup drive.  Original drive went south on me during my return travels.  Just started a new one and re-imported the DNGs.  Three days work preserved in five minutes.  I can put up with a little more noise in the pixel peeping department.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
To DNG or not to DNG
« Reply #35 on: March 13, 2009, 11:09:19 pm »

Quote from: Pete Ferling
Original drive went south on me during my return travels.  Just started a new one and re-imported the DNGs.  Three days work preserved in five minutes.
This has nothing to do with DNG. ACR is creating XMP files, which are MUCH easier to save (7 KB instead of many megabytes).

Furthermore, it is *absolutely brainless* to save the *original raw data* in order to save the adjustments. Add to this, that I can keep several XMP files for the same raw file, which is not possible if storing the XMP data in the DNG file, at least this is so with ACR 4. It is a horrendeous solution (instead of packeging several sets of adjustments in one XMP file), but it is a solution.
Logged
Gabor

john beardsworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4755
    • My photography site
To DNG or not to DNG
« Reply #36 on: March 14, 2009, 05:11:45 am »

Quote from: Panopeeper
....Add to this, that I can keep several XMP files for the same raw file, which is not possible if storing the XMP data in the DNG file , at least this is so with ACR 4.

That "objection" is obsolete. You can save "snapshots" in the DNG which ACR5 or LR can access (as can ACR4 via a Bridge script).
Logged

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
To DNG or not to DNG
« Reply #37 on: March 14, 2009, 06:59:44 am »

Quote from: Panopeeper
This has nothing to do with DNG. ACR is creating XMP files, which are MUCH easier to save (7 KB instead of many megabytes).

"Much easier to save"? How so? What "many megabytes"?

A basic DNG conversion comes in slightly smaller than the raw file it was made from, including the adjustment data, and as John says, including versions (snapshots) of adjustment data too.

And one file is always easier to backup and otherwise manage than two. Sidecar files always were a kludge, time to move on.

What part of the usefulness of a fully corrected embedded preview image do people not understand? Without it you can only do any meaningful exports in a RAW converter. With it you can use any app that can access the preview. Try generating a web gallery from a bunch of RAWs without using Bridge, LR, C1 Pro etc. All the extra power of Expressions Media or Photo Mechanic is lost to you with basic RAWs since the images they make look awful - 'cos they are not RAW convertors. Even the catalog thumbs made by DAMs look bad when made directly from basic RAW files.

It's OK not to use DNG if you don't choose to, it's your prerogative, but please don't go into denial about the format's numerous and genuinely useful aspects.
Logged
Nick Rains
Australian Photographer Leica

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
To DNG or not to DNG
« Reply #38 on: March 14, 2009, 07:35:16 am »

Quote from: Nick Rains
And one file is always easier to backup and otherwise manage than two. Sidecar files always were a kludge, time to move on.

I agree, but did have a similar conversation last night with "the big guy" who also suggested that one convert to DNG after doing much of the work in LR/ACR so you don't have to back up the entire DNG to in essence back up the tiny metadata edits. I still like the DNG route for the reasons you and others expressed and the back up happens while I'm sleeping automatically. I've also been a lot happier with auto metadata update preferences in LR since it was fixed many releases ago. Doesn't seem to bog down. Plus, if you're working with rendered images, the idea that you have to back up, or spend time saving is and always has been necessary.

Would it be cool if the metadata could somehow be saved or backed up without the rest of the Raw? Sure. But I don't know I'd ask the Adobe team to take precious time away from other functionality to do so.

The question I have is, would leaving the Raw data alone, only backing up the side car be more or less "reliable" over time in terms of the data handling than saving often, the entire DNG?
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
To DNG or not to DNG
« Reply #39 on: March 14, 2009, 06:25:17 pm »

Quote from: digitaldog
I agree, but did have a similar conversation last night with "the big guy" who also suggested that one convert to DNG after doing much of the work in LR/ACR so you don't have to back up the entire DNG to in essence back up the tiny metadata edits.

Good point - one potential disadvantage of DNGs is that (if set) the jpeg preview will be rebuilt everytime you made a change to the RAW settings. This is time consuming.

Best practice - fully adjust the RAW file first and then convert to DNG once you are happy. Converting to DNG on import does not really make sense and is quite inefficient.
Logged
Nick Rains
Australian Photographer Leica
Pages: 1 [2] 3   Go Up