How many images can you have in a C1 8 catalog before it slows down?

[The View]

I'm on a retina MacBook Pro with 16Gb RAM. It's the 2012 model with 2.6 Ghz, so the speed should not be that much less than a current model.

I will soon be updating to C1 8 - after certain projects have been finished.

I currently have two catalogs, which isn't great, because I have to switch to get to older images.

I think of importing the older images into the same catalog as the new one - but at what size will the new catalog start to slow down?

[Jimmy D Uptain]

I saw your post yesterday but was reluctant to respond as I didn't wanna be a Negative Nancy.
Some folks have no issues with big catalogs.
I'm not one of those people.
I'm in the middle of troubleshooting the issue so I have absolutely no insight.
That being said, all you can do is try. Please report your experience.

[geezerhood]

My main catalog has a bit over 10,000 images. I have not noticed it to be any slower than my smaller catalogs, but it is slower when backing up.

[kirmo]

I worked several decades as a database expert tuning very large relational databases. The ones used in C1 are using the same technolgy.
The number of images is not the critical factor. When all is set corretly these should handle easily hundreds of thousands image references.

But if missing like the needed indexes, internal references, things will get really slow!

I'm currently setting up my C1 environment with catalog over 100 000 images. Just back from trip in China and will contunue to explore
the situation next week. So far things have been ok.

The good news is that if things are getting slow, normally there's a fix around. Just need a proper index. This should be fairly  quick and
easy task for Phase when given the proper info about the situation. Quite normal that these are happening in the early state of a new
version like we now have 8.0 out from Phase.

Remeber to make backups!!!

[Jimmy D Uptain]

I worked several decades as a database expert tuning very large relational databases. The ones used in C1 are using the same technolgy.
The number of images is not the critical factor. When all is set corretly these should handle easily hundreds of thousands image references.

But if missing like the needed indexes, internal references, things will get really slow!

I'm currently setting up my C1 environment with catalog over 100 000 images. Just back from trip in China and will contunue to explore
the situation next week. So far things have been ok.

The good news is that if things are getting slow, normally there's a fix around. Just need a proper index. This should be fairly  quick and
easy task for Phase when given the proper info about the situation. Quite normal that these are happening in the early state of a new
version like we now have 8.0 out from Phase.

Remeber to make backups!!!

Its good to hear from someone "in the know" about databases.
My programmer friend essentially told me the same thing.
You mentioned proper info. Can you elaborate? I really need this handled and I want to give them everything they need.
Hell, at this point, I don't mind looking around in the database via text editor.
Yeah I do make backups too. 

[kirmo]

Jimmy,

with proper info I mean things like:

- what exactly was the slow action doing. If it was a filter search, then which filters where set to be active.
- if like doing a keyword for many images at one time ...


When someone is just saying that it's working too slow. For the programmers it's kind of difficult to do anything
if there's no real input of what was happening or at least what was been tried to do.

Many times the softwar eis done so that there's internal info of what is going on. That's why it make sense to send
the asked crash etc reports to Phase and other software makers to help solve the problems.

I don't work for Phase and don't know the inside of their software to say more precise.

So the more details one can tell and explain what was going on helps and will really make sense on the Phase side.

The number of images in one catalog (rdms-database) will not be the problem! But I'm also not putting everything into
one catalog, but will have catalogs with over 150 000 images and growing all the time.

Kirmo

[Jimmy D Uptain]

The weird thing is, they cannot duplicate the problem.
I have sent multiple crash reports, samples of the database and other things.
The only thing I haven't done, is send the entire catalog with originals. I'd have to do that by mail as the catalog is 50+ gigs and the originals are about 500gb.
I'm almost tempted to do just that.

At first they said it was my HDD causing the problem, but I have no issues with Lightroom in the exact same scenario.
Being as LR uses sqlite as well, I cannot see where my hard drive is part of the problem.
Also C1 is designed to be used with the originals offline so communication between the drives shouldn't be an issue.

The fact that even building a catalog takes over three days (i've done this multiple times trying to resolve the issue) raises concern.
In fact, the the injestion of the fist 10,000 is quite fast, after that, things drop to a snails pace. These issues must have a correlation.

I so want to use C1's catalog, but I'm relegated to just using sessions until this is resolved.

[kirmo]

Jimmy,

I understand your frustration about the issue. I've seen these plenty and spent hunderds of hours solving these!

My answer was to the question: "How many ..." and I'm still saying the number of images  files is not a limiting factor.
Especially if the case is refrenced files not the ones imported into to catalog.

Just checked that so far my bigger catalog has over 41 000 images and growing.
For a smaler I added my last trips images around 9300 and the imprt took about 45 seconds.
The previews will of course their own time!

Programs are never bug free. It's just imossible to test all the cases and that's the life we live in.

I started using Lightroon just before it was officially released and especially the print section was the reason for me.
I do a lot of printing (Epson 9900). After several years and upgrades in LR and MacOs I strarted getting an issue when
printiung just didn't work as it should. Tried to solve this and did all that I could imagine. The problem always came back.
Soi I finaly moved to using Mirage and no problems.

Things are not so dtraightforawd as so many different players taking their share in this game. Operating system, different 3rd
part software's, hardware, ...  The problen might be caused by just things not matching with these players and repeting the cases
are not easy! It could be so any things.

Keeping my fingers crossed and hoping your issue is solved.

Kirmo

[Jimmy D Uptain]

Jimmy,

I understand your frustration about the issue. I've seen these plenty and spent hunderds of hours solving these!

My answer was to the question: "How many ..." and I'm still saying the number of images  files is not a limiting factor.
Especially if the case is refrenced files not the ones imported into to catalog.

Just checked that so far my bigger catalog has over 41 000 images and growing.
For a smaler I added my last trips images around 9300 and the imprt took about 45 seconds.
The previews will of course their own time!

Programs are never bug free. It's just imossible to test all the cases and that's the life we live in.

I started using Lightroon just before it was officially released and especially the print section was the reason for me.
I do a lot of printing (Epson 9900). After several years and upgrades in LR and MacOs I strarted getting an issue when
printiung just didn't work as it should. Tried to solve this and did all that I could imagine. The problem always came back.
Soi I finaly moved to using Mirage and no problems.

Things are not so dtraightforawd as so many different players taking their share in this game. Operating system, different 3rd
part software's, hardware, ...  The problen might be caused by just things not matching with these players and repeting the cases
are not easy! It could be so any things.

Keeping my fingers crossed and hoping your issue is solved.

Kirmo

Your printing issues with LR mirror mine.
I also resolved said printing issues with Mirage.

One more question though, what are your system specs?

[kirmo]

Oops, meant to write 930 images in about 45 seconds!

Just back from China and having mysterious pain in neck/arm: maybe a nerve stuck
somewhere. Makes writing and focusing a bit problematic!

Plenty of work that should be done but ...

Jimmy, if you want to we can continue in private emails kirmo.wilen@valolla.fi


Just putting up my new MacPro + Promise2 R6 Raid. Time shows how this will work out.
So far looking good.

Kirmo 

[sjprg]

   I'm running C8 on an Asus P8Z77-V using Windows 8.1. C8.1 import mechanism is a real PITA. When I click on import it request the folder to import with the subfolders checked. I expect the catalog to import al the subfolders and their subfolders in the same sequence as it finds them in my file structure. The structure is /Images/ 
                                                                       2000-2005/
                                                                                       2000 Master/
                                                                                                         2000_01_01 description
                                                                                                         2000_01_02 description
                                                                                                                   etc
                                                                                       2001 Master/
                                                                                                         2001_01_01 description
                                                                                                         2001_01_02 description
                                                                   repeats through 2014
C8 sometimes follows the structure, sometimes not and inconsistently when deleated and started again, whereas Lightroom 5.7.1 does it perfectly. (49 thousand images)
On my MacBook Pro )early 2013 sometimes puts the structure correctly, but mostly just inputs the files linearly by date with no structure. 

[Jimmy D Uptain]

For a month or so, I have been trying to figure out two things:
1. Why is C1 so sluggish with larger catalogs?
2. Why does it take so long to create a catalog?

Here's what I have found so far.
File structure has played the biggest role in catalog creation.
The way my file structure has been since the beginning has been one folder with all images inside that folder. That's it!
I've always let my DAM do all the heavy lifting and that worked fine until now.

Anyway, after talking with a couple of other guys, I came to realize that this "all in one bucket"  structure may be playing a part in why my catalog takes too long to build.

With all the files in one folder, the catalog took about three days to build.
With the files separated by dates, each day having its own folder, the same catalog took about 23 hrs.
I then used a program that divided everything into 4gb folders. The catalog took about 12hrs to build.
And thats where I am now.
I'm waiting for the previews to build before I do any testing on this newest catalog's performance.

Now as far as catalog performance is concerned (from my 23hr build), I have found that by removing the Filter tool, things are a bit snappier.
One can still use the search function in the browser, and it works pretty good, just don't try to use the search function in the Filter tool.
The catalog still takes way too long to load (two minutes) but once opened, it's not terrible.


 

[Jimmy D Uptain]

Just wanted to let y'all know, that my support case is "Finished"

Oh yeah. With absolutely NO resolution.

Way to go guys.