Pages: 1 [2] 3   Go Down

Author Topic: How to run DNG validation  (Read 1126 times)

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #20 on: June 25, 2018, 08:50:27 AM »

Francisco,

Thanks for the warning. Just checked the external drive for a subfolder. No sub folder but disaster. Seems it replaced all the files and moved originals to the trash. They are now less than half in MB size. Probably stripped out the inferred channel and more.

Input profile ARGB, should be scanner profile. 8 bit tif instead of 16 bit. 20 x 29 inch @ 72 ppi. Should be about 5 x 7 @ 300 ppi. All 117 files with the same creation date of the same minute.

I was just going to start running all the folders until I saw your warning. You saved me.

Now what to do? Think I restore originals from trash or a backup and give up on the dng validation. There was no "put back" option from the trash.
Logged

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #21 on: June 25, 2018, 09:24:23 AM »

Clarification,

I opened, edit in PS, from LR. PS shows the correct size, ppi, but opened as tif and not in ACR. Maybe that's normal?

I opened the new dng from the external drive in SilverFast HDR Studio. For some reason it used a for web preset that gave the above mentioned sizes. File itself has correct sizes. You can not export as dng from HDR Studio. Must be a rendered file format.

The inferred iSRD tool was not available. Same with 2 others I checked. I always include IR in the scan so it must have gotten striped out. That is a deal breaker. And the scanner profile should be showing but is not.
Logged

FranciscoDisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1191
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #22 on: June 25, 2018, 12:27:44 PM »

Saiguy,

I though you converted the images to DNG when importing, but now I realize you used the LR function "Convert to DNG" after your images were imported. In the dialog that appears, there is an option to "Delete orginals after succesfull conversion" which might have been the cause of your original images being deleted. The difference in size may be due to compression.

If all your files are on the trash, you should be able to copy or drag them to their original location.

Having said that, I checked a few things regarding the DNGs generated by Silverfast. First they are linear DNG or rendered files (not RAW), similar to tif, and it is very likely that the ifrared channel is not kept when you do the conversion. 

I did not realize before that you would need to process these files in HDR studio. It is very likely that the Silverfast suite prevents processing DNGs not created by their scanning software or modified by another app, so it is better to stay with the DNG generated in the scanner software.

In this scenario, the DNG validation by LR/ACR does not look as viable option, unfortunately

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13917
    • http://www.digitaldog.net/
Re: How to run DNG validation
« Reply #23 on: June 25, 2018, 12:36:20 PM »

Having said that, I checked a few things regarding the DNGs generated by Silverfast. First they are linear DNG or rendered files (not RAW), similar to tif, and it is very likely that the ifrared channel is not kept when you do the conversion. 
Which may be a bit OT, but I still do not understand why some companies feel the need to 'suggest' that a rendered image, linear or otherwise, stored into the DNG container is useful, it's not raw data.
I suppose if the scanning software were limited, one could argue for saving a 'rawer' rendered image to 'fix' later in say Photoshop. But SliverFast is quite good for a global editor of input scans. Call my old fashion, someone who was scanning back when the Leaf 45 first arrived with software that isn't anywhere as robust then, as SilverFast is today. We did all the heavy lifting at the scan stage. That includes working with really lovely and powerful scanning software later like LinoColor, ScanView etc.

Logged
Andrew Rodney
Author “Color Management for Photographers"

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #24 on: June 25, 2018, 03:51:02 PM »

Francisco,

I imported the HDR dng files to LR via add method. I do not remember seeing any dialog. Think they were just added. Then I used the LR function Convert to DNG.

I did restore the originals from the trash and they look to be OK.

Given your last reply and Andrew's, it seems the SF implementation of dng is useless if I can't run a validation. In future I will export the HDR raws in the tif file format. And yes, the work flow is to later develop them using their scanning tools in the HDR Studio.

Thank you for sharing & caring,
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13917
    • http://www.digitaldog.net/
Re: How to run DNG validation
« Reply #25 on: June 25, 2018, 03:54:42 PM »

Given your last reply and Andrew's, it seems the SF implementation of dng is useless if I can't run a validation.
Don't know about useless but questionable as to why one would do all this and save off as a DNG. If you have something that would validate a TIFF instead, well that's another compelling reason to save as TIFF rather than DNG which is a very close cousin of TIFF.
Logged
Andrew Rodney
Author “Color Management for Photographers"

kirkt

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 450
Re: How to run DNG validation
« Reply #26 on: June 25, 2018, 04:21:43 PM »

You can download the DNG SDK from Adobe.  Inside of the included project folders, there are target builds of the command line tool for dng_validate.  For example, in the Mac SDK folders:

... dng_sdk_1_4/dng_sdk/targets/mac/release64/dng_validate

and Windows:

... dng_sdk_1_4\dng_sdk\targets\win\release64\dng_validate.exe

kirk
Logged

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #27 on: June 25, 2018, 05:15:44 PM »

Andrew,  Is there a tool to verify TIFF files?
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13917
    • http://www.digitaldog.net/
Re: How to run DNG validation
« Reply #28 on: June 25, 2018, 05:40:12 PM »

Andrew,  Is there a tool to verify TIFF files?
Yes posted what I’m using earlier.
Logged
Andrew Rodney
Author “Color Management for Photographers"

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #29 on: June 25, 2018, 06:40:17 PM »

Andrew, Can you please point to that post. Am very interested. Guess I missed it.

Sent a forum query to LSI asking if validation can be run on there HDR dng's. Will report here if they respond.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 13917
    • http://www.digitaldog.net/
Re: How to run DNG validation
« Reply #30 on: June 25, 2018, 07:27:48 PM »

Andrew, Can you please point to that post. Am very interested. Guess I missed it.

Sent a forum query to LSI asking if validation can be run on there HDR dng's. Will report here if they respond.

Re: How to run DNG validation
« Reply #11 on: June 24, 2018, 04:33:50 PM »

What I’ve used in the past but now unsupported?
https://basepath.com/new/detail-ImageVerifier.php
Logged
Andrew Rodney
Author “Color Management for Photographers"

FranciscoDisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1191
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #31 on: June 26, 2018, 07:01:02 AM »

Which may be a bit OT, but I still do not understand why some companies feel the need to 'suggest' that a rendered image, linear or otherwise, stored into the DNG container is useful, it's not raw data.
I suppose if the scanning software were limited, one could argue for saving a 'rawer' rendered image to 'fix' later in say Photoshop. But SliverFast is quite good for a global editor of input scans. Call my old fashion, someone who was scanning back when the Leaf 45 first arrived with software that isn't anywhere as robust then, as SilverFast is today. We did all the heavy lifting at the scan stage. That includes working with really lovely and powerful scanning software later like LinoColor, ScanView etc.

Having been a past user of Silverfast, my take is that they included later the DNG option as a "me too" (was not there in version 6). They develped the HDR Studio with the the option to sptil in half the workflow. You can scan the files in batches and store all information plus the infrared channel in an intermediate file, which are the files considered in this thread.

You could then finalize the process at a later time in the HDR Studio suite. The issue here is that the file is rendered (you can open it in PS) and the suite will not accept a modified DNG to complete the process, so I agree that there is not much value in storing this as a DNG

FranciscoDisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1191
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #32 on: June 26, 2018, 07:02:35 AM »

You can download the DNG SDK from Adobe.  Inside of the included project folders, there are target builds of the command line tool for dng_validate.  For example, in the Mac SDK folders:

... dng_sdk_1_4/dng_sdk/targets/mac/release64/dng_validate

and Windows:

... dng_sdk_1_4\dng_sdk\targets\win\release64\dng_validate.exe

kirk

Hi, this works if you have the validation checksum stored in the DNG file. As was mentioned previously, most of the DNG's generated outside the Adobe applications do not include this piece of information

FranciscoDisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1191
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #33 on: June 26, 2018, 07:15:50 AM »

Is there a tool to verify TIFF files?

Just in case you want checksum validation for long term archiving, (which is what DNG validation does) most of the tools available online just calculate the checksum but I have found a tool that can perform periodic validations, which I'm testing at the moment, called Fixity.

This tool creates a database of checksum of multiples files in a folder structure which you can check periodically to ensure the data has not been changed. This works well only for archived files, meaning files that will not be updated in the future.

This is not exactly the same as what DNG validation does, as in the latter only the image portion of the DNG file is considered, (the raw data does not change if you edit the DNG)

Just in case, during my preliminary tests, the initial checsum generation for a folder three of 6,482 raw images totalling 260GB, took 2 hours 45 minutes. I'll perform a subsequent validation tomorrow and will report on the elapsed time.

One curious fact, is that the LR database stores a hash of each image file on first import and then another hash at the latest synchronization, which could be used for this validation with little programming effor, but I guess is not a priority.

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #34 on: June 26, 2018, 01:29:53 PM »

Sent a question to SilverFast re: DNG validation. Got a reply from a "Senior Support & Test Expert". Said SF8 can not directly validate but other tools like LR can, but he was not 100% certain. Said he would do some testing and report the results.

I returned the mail telling what I experienced as stated above in this thread.

Will try Fixity today. Looks great as no conversion involved. Thanks Francisco.
Logged

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #35 on: June 26, 2018, 06:43:35 PM »

Downloaded Fixity for Mac. Created a New Project but it would not recognize my mounted external drive. So I could not select the folder I wanted to use it on. Sent a query via their contact form. It would only show local sources on the Mac.
Logged

FranciscoDisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1191
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #36 on: June 26, 2018, 07:10:53 PM »

Downloaded Fixity for Mac. Created a New Project but it would not recognize my mounted external drive. So I could not select the folder I wanted to use it on. Sent a query via their contact form. It would only show local sources on the Mac.

Check this thread External-volumes-Mac in their user group to see if it solves your issue

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #37 on: June 27, 2018, 05:17:59 AM »

Thanks Francisco for the link to the user group. It said in the list of local choices there would be one for Volumes. But it was not there.
Logged

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 133
Re: How to run DNG validation
« Reply #38 on: June 27, 2018, 10:57:26 AM »

Found it. When you go to select a directory it opens to the home folder. That is a drop down where if Macintosh HD is selected their is a Volumes choice from which I could navigate to the folder I wanted.

thanks again.
Logged

FranciscoDisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1191
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #39 on: June 27, 2018, 03:36:00 PM »

Great! What I have seen so far is good. The program works well once you learn how to use it. There are some counter-intiuitive issues or things that are not clear in the documentation but it is being maintained by the developers. As my friends in marketing would say, it looks like a program developed by engineers, meaning most effort is put into function and less into form.

Regarding the tests, my scheduled run performed fine and it took about the same amount of time as the first run. One little advantage in windows is that you do not need to be logged on for the program to run according to schedule.

Overall, considering it is an free open-source program, my evaluation is positive so far
Pages: 1 [2] 3   Go Up