Pages: [1]   Go Down

Author Topic: Using the catalog with filters is slow and inconsistent  (Read 1197 times)

scrollop

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Using the catalog with filters is slow and inconsistent
« on: June 10, 2018, 09:59:11 AM »

I've imported 220 000 (aprox 4.4 tb) photos/videos into a catalog (system: win 10 x64, 24gigs, 4th gen i7, GTX 970, C1 11.1.1 catalog on an SSD). That took a few hours to import and 24 hours to make 900x previews.

There are major issues when attempting to use the catalog.

- It's inconsistent. When selecting a filter of f1.4, for example, where alongside the filter it states there are 15 000 images, perhaps 2000 images appear (slowly). Switching to other filters does not show the correct amount of photos in the browser vs the amount alongside the filter. Sometimes, when changing from one filter to another, nothing happens.

- It's slow. It takes a long time to import photos, and after you import them, the system appears to review every photo when populating the filter figures.

- When selecting one filter eg. 35mm (though the system doesn't show the correct amount of photos) and another filter eg f1.4 (using CTRL), the second filter selection does not refine the results ie. nothing happens.

On paper the filter system appears powerful. In practice, IMHO, it is inconsistent and slow and thus unusable.

If you have any recommendations on how to improve performance, I'd appreciate it.


Logged

scrollop

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Re: Using the catalog with filters is slow and inconsistent
« Reply #1 on: June 10, 2018, 12:23:48 PM »

Also, when selecting f1.4, for example, which apparently has 15000 photos, why does it only show 2 photos?
After waiting 30 seconds or so, slowly photos are added.
I try another aperture - nothing happens for 30 seconds. Then it starts at 30 photos and slowly adds them (maybe 10 - 20 photos per second, though it pauses every 2 -5 seconds, then adds more). Apparently the new aperture has 10 000+ photos. How long will it take to show all the photos that I have selected?

This is after it spent 24+ hrs importing the photos in the first place.
It's unusable. Perhaps it is due to the size of the catalog. In which case, it's unusable with catalogs above a certain size on moderately powerful systems.


Oh, and as it's searching the disk and populating the filter I chose, it's now using 3 gigs in memory and 20% CPU. LR would have accomplished this within 10 seconds.
Logged

Hoggy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 203
  • Never take life, or anything in it, too seriously.
Re: Using the catalog with filters is slow and inconsistent
« Reply #2 on: June 12, 2018, 01:03:31 AM »

Yep.

That's one of the biggest reasons I wouldn't switch from LR - as I'm totally a catalog person.  Not only is that CO behavior happen with filters - but also any smart albums as well.  Simply... Just.. Unusable.  Period.

It's beyond me how they still haven't fixed such a basic usability flaw after so many years.  I just have no words to explain it anymore......
Logged
Cams: Pentax K-3, K-30 & Canon G7X, S100
Firm supporter of DNG, throwing away originals.
It's the hash, man..  That good hash!

scrollop

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Re: Using the catalog with filters is slow and inconsistent
« Reply #3 on: June 13, 2018, 08:57:31 AM »

I read somewhere that the problem could be that I selected 900px previews and C1 was scaling them for the screen size (4k).

It was recommended to use previews the size of your monitor or larger. I selected larger previews, let it alone for 2 days to regenerate them and have come back to see how it's going.

C1 was using 14 gigs of RAM and pretty much at a standstill. It had used the remaining 150 gb of storage space on my second SSD and had generated 40% of the previews.

Perhaps I am doing something wrong here. Perhaps you need large SSDswith 400gigs of space just for the previews and the latest hardware.

Or, perhaps C1 is not designed for large catalogs, which is my conclusion. Am happy to be proven wrong.
Logged

UnfamiliarLight

  • Newbie
  • *
  • Offline Offline
  • Posts: 28
    • Unfamiliar Light
Re: Using the catalog with filters is slow and inconsistent
« Reply #4 on: June 13, 2018, 11:16:11 AM »

Or, perhaps C1 is not designed for large catalogs, which is my conclusion. Am happy to be proven wrong.
Well it is supposed to be designed for large catalogues. Looks like some proof is needed :-) What does support say on the matter?
Logged

chadchat

  • Newbie
  • *
  • Offline Offline
  • Posts: 7
Re: Using the catalog with filters is slow and inconsistent
« Reply #5 on: June 25, 2018, 04:02:10 PM »

I agree the time it takes to load a smart album is not good.
Logged

mrenters

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 79
    • http://www.teckelworks.com/
Re: Using the catalog with filters is slow and inconsistent
« Reply #6 on: August 03, 2018, 04:49:35 PM »

Catalogs don't perform very well (or reliably for that matter) in Capture One. I've imported my old Lightroom catalogs into Capture One and have approximately 30,000 images in the catalog now. Both the images and catalog live on a NAS drive and the performance is terrible in Capture One while being reasonable in other software line ON1 Photo Raw.

As a software developer I've tried to figure out why this is happening and have found that Capture One seems to love to frequently traverse all the image files in a directory, presumably to check whether it has previews or whether meta data is up-to-date. This can tie up Capture One for extremely long periods of time and locks up basic filtering. The issue doesn't seem to be that the underlying SQL database is unable to answer the question of how many files meet the criteria, but that Capture One is too busy doing other things to get the answer.

Filtering all IIQ files from a folder containing 3400 files can take minutes, although this could be answered in at most a second by the database.

I *really* hope that Phase One gets their act together and dramatically improves performance in the next version. I haven't come across any other software that is this slow at doing basic cataloging functions. On top of that there are a number of bugs where it shows a preview for an entirely different image until you make a small adjustment and then it gets it right.

Martin
Logged

Dave Rosser

  • Full Member
  • ***
  • Offline Offline
  • Posts: 163
    • My Website
Re: Using the catalog with filters is slow and inconsistent
« Reply #7 on: August 04, 2018, 09:33:20 AM »

Catalogs don't perform very well (or reliably for that matter) in Capture One. I've imported my old Lightroom catalogs into Capture One and have approximately 30,000 images in the catalog now. Both the images and catalog live on a NAS drive and the performance is terrible in Capture One while being reasonable in other software line ON1 Photo Raw.

As a software developer I've tried to figure out why this is happening and have found that Capture One seems to love to frequently traverse all the image files in a directory, presumably to check whether it has previews or whether meta data is up-to-date. This can tie up Capture One for extremely long periods of time and locks up basic filtering. The issue doesn't seem to be that the underlying SQL database is unable to answer the question of how many files meet the criteria, but that Capture One is too busy doing other things to get the answer.

Filtering all IIQ files from a folder containing 3400 files can take minutes, although this could be answered in at most a second by the database.

I *really* hope that Phase One gets their act together and dramatically improves performance in the next version. I haven't come across any other software that is this slow at doing basic cataloging functions. On top of that there are a number of bugs where it shows a preview for an entirely different image until you make a small adjustment and then it gets it right.

Martin
+1 to all that.  I have given the catalogue function in Capture One many tries and have at last given up on it, it is just too slow and flaky.  I have reverted to using lightroom for cataloguing and run of the mill editing and use Capture one in session mode for images that need Capture One's particular strengths.  In session mode I simply brows to the picture I want to edit in the Library/System Folders - I make no use at all of Session Folders or Session Albums.  Capture One stores all my edits etc in a sidecar file in the directory in which the RAW file resides.

Doug Peterson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3923
    • http://www.doug-peterson.com
Re: Using the catalog with filters is slow and inconsistent
« Reply #8 on: August 04, 2018, 11:26:57 AM »

Based on 11 years of teaching Capture One classes and consulting with photographers and companies, the majority of C1 users that are currently using catalogs are doing so because they used catalogs in LightRoom (the only option in LR) and would choose to primarily use sessions if they had the pros and cons fully explained to them. As a consequence we wrote this article: Capture One Session vs Catalogs to help users decide which is right for them.

That's not to say that I know for sure you specifically would be better of with sessions. Just that it's definitely worth your time and consideration.

IanSeward

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 85
Re: Using the catalog with filters is slow and inconsistent
« Reply #9 on: August 06, 2018, 05:23:16 PM »

I used LR since V1 and therefore only thought in catalogues. :-)

I now use sessions and much prefer it. In LR I always "found" my images by going to the folder in LR.  I realise others "live or die by the catalogue" for whatever reason. I import the sessions into a catalogue at the end of a year out of habit, but rarely actually use it.

I just opened up an old "Master Catalogue" imported form LR and then added to.  It opened in around 15 sec and filtered 1650 red images in a few seconds.

I am using a 5 year old desktop computer with 16Gb ram, I73820 @ 4.3GHz and a Nvidia 1060 6Gb video card.

What I have noticed with C1Pro is that if you have any corrupt images where the Event viewer says "cannot build preview" then C1 Pro is slow. It appears to check every image in the catalogue each time it starts to make sure the images are OK.  After you have removed the faulty images it still appears to check each file for the next 2 start ups and then it is fine.

It might be worth checking the Event viewer to see if it flags up any issues.

The other suggestion is to export the catalogue to a new catalogue, as this can sometimes "sort out" unexplained issues with the catalogues integrity. You need to recreate the previews but this can be an overnight job.

Hope this helps.

Ian
Logged

Hoggy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 203
  • Never take life, or anything in it, too seriously.
Re: Using the catalog with filters is slow and inconsistent
« Reply #10 on: August 12, 2018, 12:05:29 AM »

What I have noticed with C1Pro is that if you have any corrupt images where the Event viewer says "cannot build preview" then C1 Pro is slow. It appears to check every image in the catalogue each time it starts to make sure the images are OK.  After you have removed the faulty images it still appears to check each file for the next 2 start ups and then it is fine.

It might be worth checking the Event viewer to see if it flags up any issues.

The other suggestion is to export the catalogue to a new catalogue, as this can sometimes "sort out" unexplained issues with the catalogues integrity. You need to recreate the previews but this can be an overnight job.

I did this on my circa 2012 4-core AMD A8-3500M APU w/8GB-maxed-ram laptop before moving to my new supercomputer.  In particular, I removed all the LR-merged dng files from the C1 catalog.  (btw, I'm a complete and total catalog person here.)
It was a bit faster, but still dog slow when moving between collections - with catalog and previews being on a SATA SSD, of course.  I didn't try the export to new catalog though - maybe I'll try that on that laptop.

Now on my new supercomputer, C1 is finally not super slow - but still MUCH slower than LR on this same machine.  This is with a reasonably all-core overclocked AMD Threadripper 16/32-core with 128GB of 2933Mhz RAM and an 8GB Nvidia Quadro P4000 video card, with catalog and previews on a ~1TB Intel Optane 905P and images on a ~1TB Samsung 970 Pro SSD (both NvMe PCIE x4 SSD's).

However, the fact remains that one should never **NEED** a $10,000 PC just to say a certain desktop software package is finally not super slow.  They really ought to get catalog speeds fixed one of these decades, as opposed to continually looking at it as a benefit of sessions.
(Hey, I went 8 years with $0 income, so I might finally kinda sorta deserve a bit of bragging rights.  :P ;D )
Logged
Cams: Pentax K-3, K-30 & Canon G7X, S100
Firm supporter of DNG, throwing away originals.
It's the hash, man..  That good hash!
Pages: [1]   Go Up