Pages: [1]   Go Down

Author Topic: DNG filesize confusion  (Read 6192 times)

viahorizon

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
DNG filesize confusion
« on: January 17, 2009, 03:43:03 pm »

I use Adobe DNG converter to convert all raw files to DNG. Most of them are around 10-15MB in size. They are compressed. And then there is this one file which is 42MB. And a very similar, a reframe is ~10MB. Do you have any clue why could it be?

And I believe it's not a matter of compression as a much more 'complicated', detailed and harder to compress file is also ~10MB.

What do you think?


40M:

10M:

10M:


viahorizon.com
Logged
gallery@viahorizon

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG filesize confusion
« Reply #1 on: January 17, 2009, 03:53:00 pm »

If you are using Windows, or Mac with some Windows simulator, then download this program, start it and enter the file name. It will produce a list, named xxxx.DNG.taglist.txt; that contains the entire structure of the file. If you upload or post that file, I can say what is so large there.
Logged
Gabor

viahorizon

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
DNG filesize confusion
« Reply #2 on: January 17, 2009, 04:10:39 pm »

Wow! Panopeeper, thank you for such a prompt and informative answer.

I run the tool and uploaded the log: http://viahorizon.com/tmp/log.txt

I've had a look and it and couldn't fine any clues, why it's so big. It's Canon 5D file...
Logged
gallery@viahorizon

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG filesize confusion
« Reply #3 on: January 17, 2009, 04:28:51 pm »

This file is in demosaiced, "linearized" form, in a quasy-RGB format. Instead of a single value per pixel, there are three values stored per pixel, that's the reason of the large file. The data is in a transitional state between raw and RGB.

Perhaps you clicked on the wrong radio check in the dialog.

Added: you may want to take a look at Rawnalyze (the taglist is a spin-off)
« Last Edit: January 17, 2009, 04:35:10 pm by Panopeeper »
Logged
Gabor

viahorizon

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
DNG filesize confusion
« Reply #4 on: January 18, 2009, 03:27:03 am »

I'm sure I never demosaic my raw files (and all 3 files were converted in one batch). So it seems there is a bug in DNG Converter 5.2.0.65. Damn it. I guess there is no way to go back to RAW file configuration now?

The problem is it really made me distrustful to the converter app, so my workflow just got expanded to a step of verifying if DNG Converter didn't fail
« Last Edit: January 18, 2009, 03:40:47 am by viahorizon »
Logged
gallery@viahorizon

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
DNG filesize confusion
« Reply #5 on: January 18, 2009, 04:01:31 am »

Quote from: viahorizon
So it seems there is a bug in DNG Converter 5.2.0.65. Damn it. I guess there is no way to go back to RAW file configuration now?

What did you do with the original raw?
Logged
Nikos

viahorizon

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
DNG filesize confusion
« Reply #6 on: January 18, 2009, 05:07:16 am »

I convert all CR2 files to DNG, and delete CR2 (originals) afterwards. I just checked some couple thousand files I've got here and only 3 are demosaiced. Strange thing.
Logged
gallery@viahorizon

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
DNG filesize confusion
« Reply #7 on: January 18, 2009, 05:17:36 am »

Quote from: viahorizon
I convert all CR2 files to DNG, and delete CR2 (originals) afterwards. I just checked some couple thousand files I've got here and only 3 are demosaiced. Strange thing.


Please don't take this as an insult and I don't want to start another pointless discussion about manufacturer vs Adobe raw formats but, with the current availability of cheap storage I believe deleting the originals is a very dumb thing to do. At the very  least this ties you up with a more limited selection of raw conversion tools.
Logged
Nikos

viahorizon

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
DNG filesize confusion
« Reply #8 on: January 18, 2009, 06:53:37 am »

I don't take this as an insult. I just believe your point does not add to this thread.
Logged
gallery@viahorizon

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG filesize confusion
« Reply #9 on: January 18, 2009, 11:20:48 am »

Quote from: viahorizon
I convert all CR2 files to DNG, and delete CR2 (originals) afterwards
Your unlimited trust in the DNG conversion is not justified. Through the conversion you are freezing Adobe's current interpretation of the raw data, which may or may not be ideal, sometimes not even correct.

I am just preparing a small program to correct DNG files, which have been incorrectly converted from 5D2 raw. In this case the correction is possible, but who guarantees, that this is always so?

Btw, I do not believe, that the DNG converted created a linear, demosaiced file "accidentally". That kind of error is very improbable.
Logged
Gabor

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
DNG filesize confusion
« Reply #10 on: January 18, 2009, 11:59:46 am »

Quote from: viahorizon
I don't take this as an insult. I just believe your point does not add to this thread.

Well, I believe it does. Maybe not for you since you seem to have made your mind about this, but maybe for other people. BTW To answer your question, no there's no way to get a true to original undemosaiced DNG from a linear one
Logged
Nikos

DarkPenguin

  • Guest
DNG filesize confusion
« Reply #11 on: January 18, 2009, 01:10:02 pm »

Any commonalities between the demosaiced files?

edit: Wait,  you didn't convert to DNG files some already converted files, did you?  Like put a jpg from a raw+jpg into a dng?
« Last Edit: January 18, 2009, 01:17:05 pm by DarkPenguin »
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
DNG filesize confusion
« Reply #12 on: January 18, 2009, 02:55:51 pm »

Quote from: Panopeeper
Your unlimited trust in the DNG conversion is not justified. Through the conversion you are freezing Adobe's current interpretation of the raw data, which may or may not be ideal, sometimes not even correct.

Personally, I have no issues converting to DNG and throwing away the original CR2.s in my case. This is like the discussions of saving PSD or TIFF (TIFF for me, I don't need another proprietary format). DNG/TIFF are both open, fully documented formats. But I understand the desire of some to keep both (or IMHO better, embed the original camera Raw file INTO the DNG).

I've been burned far too many times with proprietary formats (PhotoYCC PCD, all my old DCS Kodak files).
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

viahorizon

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
DNG filesize confusion
« Reply #13 on: January 19, 2009, 02:22:55 pm »

@DarkPenguin: I couldn't find any shared variables -- it looks like a random thing. And sure both files were converted from RAW (CR2) files, as I said in one batch operation. I'm pretty sure there is a reason behind this madness -- I just can't figure it yet  I will have a look at this issue while converting from know on.

@digitaldog: There is always that point of preserving some extremely proprietary information that might be left behind in the conversion. But for me, personally, keeping originals would rather serve as a way of additional redundancy/backup. I'm not a strong believer in that hidden, precious data embedded in OEM RAWs.
« Last Edit: January 19, 2009, 02:27:46 pm by viahorizon »
Logged
gallery@viahorizon

DarkPenguin

  • Guest
DNG filesize confusion
« Reply #14 on: January 19, 2009, 02:35:48 pm »

Quote from: viahorizon
@DarkPenguin: I couldn't find any shared variables -- it looks like a random thing. And sure both files were converted from RAW (CR2) files, as I said in one batch operation. I'm pretty sure there is a reason behind this madness -- I just can't figure it yet  I will have a look at this issue while converting from know on.

Let us know what you find.
Logged

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
DNG filesize confusion
« Reply #15 on: January 19, 2009, 05:13:41 pm »

Quote from: digitaldog
Personally, I have no issues converting to DNG and throwing away the original CR2.s in my case. This is like the discussions of saving PSD or TIFF (TIFF for me, I don't need another proprietary format). DNG/TIFF are both open, fully documented formats. But I understand the desire of some to keep both (or IMHO better, embed the original camera Raw file INTO the DNG).

I've been burned far too many times with proprietary formats (PhotoYCC PCD, all my old DCS Kodak files).

I agree with you here - but you can have your cake and eat it too. I use an off-site backup strategy and simply burn the original CR2 files to DVD to be stored off site. The on-site images, stored on two mirrored HDs are converted to DNG and thus offer all the conveniences of the DNG format. If you ever need the originals, just retrieve the offsite DVDs. Works for me!
Logged
Nick Rains
Australian Photographer Leica
Pages: [1]   Go Up