Pages: 1 2 [3]   Go Down

Author Topic: How to run DNG validation  (Read 6477 times)

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 236
Re: How to run DNG validation
« Reply #40 on: June 28, 2018, 08:12:02 am »

I ran Fixity first time on a folder with many sub folders that is 97GB. Then changed the Project Settings to filter out DS_Store & thumbs.db. and ran again.

First run took 3 hours. Second run 9 hours 10 minutes. It did remove the filtered files from consideration.

Seems it runs thru 2 passes. First one with ex drive spinning, 2nd pass with ex drive spun down. I set Prefs for drives to shut down never and screen to sleep after 3 minutes. Twice I noticed what I call deep sleep where the screen goes black and the words MACBOOK PRO bounces around the screen. Makes me wonder if the app is still running then? Seems the app has its own way of shutting down the drive when not needed and over rides my temporary Pref for never.

The long 2nd run may have to do with the very many now filtered files to deal with???
Logged

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 236
Re: How to run DNG validation
« Reply #41 on: June 28, 2018, 08:25:04 am »

Forgot to mention above.

On the 2nd run the first pass took 2hr 40min. The 2nd pass took 6hr 30min.
Logged

fdisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1852
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #42 on: June 28, 2018, 03:48:55 pm »

What checksum algorithm are you using? If it is SHA256, try changing it to MD5, which should be faster, and for validation of local files should be enough, otherwise contact support to see if they have any suggestion to  improve performance

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 236
Re: How to run DNG validation
« Reply #43 on: June 28, 2018, 11:04:12 pm »

Was using SHA256. Since it is their default thought they considered it best. Will try MD5, but dread going thru all that time again.

Was having contact with their support and quoted where you were getting rum time of 2hr 45min. on a folder 2&1/2 times larger than mine. She just said every system is different.

I have a robust MBP maxed out mid 2012 model. SSD and the ex drive is a G-RAID enterprise class, mirrored, with Thunderbolt connection.

Hoping to see your rerun speed test.

Wonder if I should start a New Project or just change the algorithm for the existing one. I'll delete the existing and start fresh with MD5 and filter out the DS_Store and thumbs. That will give a less cluttered comparison. Will do tomorrow and report findings.
Logged

fdisilvestro

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1852
    • Frank Disilvestro
Re: How to run DNG validation
« Reply #44 on: June 29, 2018, 09:12:47 am »

It is difficult to tell what may be impacting performance and the code itself might not perform the same across different platforms. Just a few comments:

- MD5 should be around 30~40$ faster. Try with a smaller directory or another drive
- Filtering files should improve performance, as there are less hashes to compute
- In my tests, the execution time of the first run and subsequent runs is about the same. For the 260 GB, it was 2 hr 45 min first and then 2 hr 59 min for the following run

In regards to the difference in checksum algorithms, I made a test on a another with NMVe drive on a 9.3GB folder containing 237 files

MD5: First run 1 Min 51 Sec; second run 1 Min 1 Sec
SHA256: First run 3 Min 11 Sec; second run 3 Min 9 Sec

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 236
Re: How to run DNG validation
« Reply #45 on: June 29, 2018, 10:05:27 am »


Francisco, thanks for the good information. I'm going to redo with new Project  as mentioned above. You have already done the testing.
Logged

saiguy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 236
Re: How to run DNG validation
« Reply #46 on: June 30, 2018, 11:15:08 am »

Created a new Project with MD5 and filter out DS_Store and thumbs.db. In Mac Prefs, Energy Saver, set my usual Computer Sleep to 5min. and display sleep to 3min. There is a check box for Put hard discs to sleep when possible, which I checked on.

Same 97GB folder set. After 15 minutes into the run time I left for an hour. When I came back and the external drive was sleeping and so I think Fixity must have been paused. I thought the processing was so intense that the ex would not go to sleep. Don't like this behavior. So the following run time numbers may be up to 1hr longer than it could have been.

Took 10 hours to run. Seems too cumbersome for me.
Logged
Pages: 1 2 [3]   Go Up