Luminous Landscape Forum

Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: free1000 on February 28, 2007, 02:52:55 am

Title: Experience with big libraries?
Post by: free1000 on February 28, 2007, 02:52:55 am
My image archive is about 50,000 images which includes everything from little digicam jpgs to 1Gb layered Tiffs.

Has anyone tried creating a single library on this scale and if so, does it kill performance?

Maybe I should be creating multiple libraries... I'd rather not if possible.
Title: Experience with big libraries?
Post by: free1000 on February 28, 2007, 03:00:23 am
How about that, five minutes after my post I had a 'corrupt library' error in LR and the thing crashed...

Thats with only about 3000 images loaded. It did include some big layered TIFFs and did reject one for being large.

I'm a little disappointed to say the least... Hopefully as I turned on the .xmp sidecar feature my changes to files can be found, though I assume I've lost the collections information for good.

Lightroom saved the library in a 'corruptDBs' folder. I have reloaded it and selected the 'check for corruption' option. The library is now open. Is it a zombie/walking dead, or can I continue to use it safely?
Title: Experience with big libraries?
Post by: seanmcfoto on February 28, 2007, 03:29:08 am
I've a Library of 50K images (well about 300 shy of) and it is reasonable. Keywording slows it down, but in general it is usable. I do have some large layered PSD'd there but nothing in the order of 1Gb. While this is not my entire Library, I'm not sure I'm going to add more of my archive stuff to it.
The XMP option will save your info out, if it had time to write. In the notes on the program it does mention that writing out the XMP will have a performance hit.
Title: Experience with big libraries?
Post by: thechrisproject on March 05, 2007, 09:00:50 pm
I've got about 20k images and importing was a pain.  I had to do it two or three times and I'm still not sure it got all the images.  Startup time for me seems to be about 2 minutes (that's until it's responsive, not until it looks open), and it freezes on me a lot.  I have a P4 2.8ghz machine with 2.5gb of ram running XP.
Title: Experience with big libraries?
Post by: free1000 on March 06, 2007, 01:03:18 pm
If only there was a way to download the report of which files failed, it would make life much easier... however you have to scroll through the list and because the log window is modal you have to dismiss it before you can try to reimport any subsequent files.

So far, this dodgy import is about the poorest feature of the software.
Title: Experience with big libraries?
Post by: Mel on March 06, 2007, 10:12:59 pm
Quote
If only there was a way to download the report of which files failed, it would make life much easier... however you have to scroll through the list and because the log window is modal you have to dismiss it before you can try to reimport any subsequent files.

So far, this dodgy import is about the poorest feature of the software.
[a href=\"index.php?act=findpost&pid=105029\"][{POST_SNAPBACK}][/a]
It seems to me that you should be able to reload and LR will not load any already loaded. [at least that is how it works for me]

I only have about 20K images though with the largest folder at close to 7000. I loaded them separately but in one library.

I will say that my Core 2 Duo 6600 with 2GB Ram works much faster than my P4 3.4 GHz with 2 GB Ram did. It seems about 2-3x faster.

Mel
Title: Experience with big libraries?
Post by: CatOne on March 06, 2007, 11:56:11 pm
Quote
My image archive is about 50,000 images which includes everything from little digicam jpgs to 1Gb layered Tiffs.

Has anyone tried creating a single library on this scale and if so, does it kill performance?

Maybe I should be creating multiple libraries... I'd rather not if possible.
[a href=\"index.php?act=findpost&pid=103678\"][{POST_SNAPBACK}][/a]

1 GB layered tiffs are pretty big.  Think about it; Lightroom is really a substitute for ACR+Bridge -- would it make sense to throw a 1 GB TIFF at ACR?  Sure, one is RAW, one is TIFF, but think about design constraints.

On to more practical matters... I heard a few people that were intimately involved with the pre-alphas, alphas, and betas say that LR currently started to slow down around the 10,000 images in the library mark.  That the hope was to eventually hit a million images, but that for now, 10,000 was where it practically started to slow (how much, I didn't test).  So I think if you're looking at 10 or 20 thousand, it's no big deal.  If you're thinking 50K to 100K then you'll start to strain it.  Then again, LR now is following in Aperture's footsteps a bit -- use it to cull your images.  Few people have 50,000 *good* images -- use it to separate the wheat from the chaff so to say.  I culled 6000 shots down to 2500 in Antartica with Aperture, and others did the same with LR.
Title: Experience with big libraries?
Post by: seanmcfoto on March 08, 2007, 09:05:12 pm
Quote
use it to cull your images.  Few people have 50,000 *good* images -- use it to separate the wheat from the chaff so to say.  I culled 6000 shots down to 2500 in Antartica with Aperture, and others did the same with LR.
[a href=\"index.php?act=findpost&pid=105147\"][{POST_SNAPBACK}][/a]

I've been doing this with new imports, but getting time for old shoots is a pain. The glorious X is helping though..