Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Phil Indeblanc on February 07, 2013, 10:14:15 pm
-
Hi LR users,
I wanted to know the number of images you have in one catalog. Please post your numbers and your experience in performance at these numbers.
Thank you!!!
-
I have 20,000 images in my catalog.
I have had no problems running LR 4.3. It is quick with no lags, hangups, etc.
I use a DELL XPS with 16 Gb RAM, a SSD, 3rd Generation Processor.
All my images are an external drive connected with a USB 3.0
Robert
-
Hi,
64457 "images". That probably includes virtual copies.
I am on old MacPro with 16GByte RAM and SSD as system/swap drive.
No problems at all. I don't think database is a bottleneck in LR.
Best regards
Erik
Hi LR users,
I wanted to know the number of images you have in one catalog. Please post your numbers and your experience in performance at these numbers.
Thank you!!!
-
I have about 30,000 photographs on my current LR4 catalog; catalog and raw files are all stored on external HD's (main and bak-up's) which I can connect either to an i-Mac or to a Mac Book Pro Retina.
Never had any issue with the catalog.
-
175K images, primarily raw (about 5K rendered TIFF/PSD files). Works fine on a Mac Pro tower with a lot of ram and really fast drives.
-
70k in my own, though I work closely with someone who has just hit 450k. Not ridiculous hardware in either case, but no significant problems.
It's a mistake to infer that the number of images affects Lightroom performance. Hundreds of thousands aren't big numbers for a database, and you're probably accessing only a few hundred/thousand of these images at a time. Most of the time LR is accessing its previews, and when it does need to grab the originals it's your disc I/O that makes the difference, as Jeff says. What's more, most reported slowdowns have other origins - eg large numbers of keywords in pre Lr4 on Windows, large numbers of dust spots on an image etc - and during the LR3 cycle these were targeted. The volume of images does affect how long backup takes, but that happens when you're asleep or doing something else.
John
-
175K images, primarily raw (about 5K rendered TIFF/PSD files). Works fine on a Mac Pro tower with a lot of ram and really fast drives.
Dam.. You win Jeff (by a lot!). ;D
About 80k RAW and about 3k rendered TIFF's - Mac Pro Tower fully loaded w/ 32 gig of ram, Fast drives, Hardware RAID, etc.
-
29k Raw files
~100 Tiff/PSD
No problems whatsoever with a mid-2010 2.4 GHz MBP.
(Yeah, I hate everything which doesn't happen directly under Lightroom ;))
-
About 35,000 mix of RAW, TIFF, and JPG from scans and various cameras. All images on external HD (now USB 3 prior was USB 2). On Dell XPS Intel i7 processor and 16 GB ram. No performance issues that I see.
Before this has older GATEWAY with less RAM and older processor. At times there were minor performance issues but nothing to really interfere with using LR 4.
Doubt that number of images is really an issue for using LR.
Les
-
About 35,000, mostly personal with some freelance, running on a 27 inch iMac first gen with the Core Duo processor and 8GB of RAM. No real performance issues.
The work photos are not catalogued in LR.
-
I posted this as I was making one catalog as opposed to a catalog per client. I was strongly encouraged by most users to do this. I approached it in a two step process.
First I made a new catalog file. Then imported half of the images into it. At this point I was at 176K+ images. It was still running rather smoothly. Then I wanted to import a couple more and at this point I thought I better import the split multiple catalogs in order for me to have all the processing instructions for the images. At this point I had 187K+ images and on one of the imports the process of importing that catalog didn't complete. It just stayed in the same state for hours, and hours.
So I ended up terminating that process and reboot and try again. After a couple attempts of it hanging in the same area I stopped it again. I needed to jump back in and work, on some of the latest images, and noticed a rather long delay for the main image to fill up the Dev screen(or Library screen). The thumbs populated, but a bit randomly as I scrolled over them, they appeared. At first nothing appeared to show up when selecting different images. Then I guess after some lengthy background processing they started to appear. This was at the end of day and I didn't really work the file much, but things looked OK. I will be working on this catalog today. But wanted to ask and share to see if others had such behavior and particularly some thoughts on the one catalog that didn't complete the import. Should have I let it try and import overnight? I even set the process priority to a High level in the Task Manager.
FYI: I'm running...
Intel Corei7 860, ~3.2Mhz QuadCore on Asus P7P55DPro
x64Bit Windows7
16GB ram
256GB SSD Crucial M
64GB System memory (Patriot Torqx)
64gb Swap for Photoshop (Patriot Torqx)
Gigabit LAN on 50-60MBS network on a number of servers.
nVidia GeForce GTX 460 graphics card/ Total available 4096MB, Dedicated vid mem 1024MB, System vid is zero memory, Shared Sys memory is 3072MB
2 screens @ 2560x1600
-
Your hardware is not an issue. The problem may be in the catalog you are importing. Try creating a new catalog and importing this in the empty catalog.
You may want to open the catalog that is causing problem when importing, and optimize it or export to a new one and import.
-
good advice Sunshine err.... ;)
-
About 170K here. Lr4/3 on a 2.66Ghz iMac. Images are stored on a 9TB Drobo connected via FW800.
-
FW800 is a good idea.
My servers have a eSATA connection besides the 1gigabit network connection.
I wonder if I can have the network connection stay as is for the different computers to access the content, while having the eSATA to the main editing computer for LR/C1 ?
I would have to have multiple eSATA connection inputs to my editing computer via some card slot on my Mobo. But if it would work smoothly, this maybe worth doing.
-
Phil, my suspicion is that the problem is more likely to be in the older catalogue you are trying to import into the new one. To get round this, select all the pictures and do a File > Export as Catalog. This should create a clean catalogue which you can import.
Long delays are not unusual when doing the Import from another Catalog into a big catalogue. Checking for duplicates is often the reason. If you are certain there are no duplicates, you can disable this routine.
John
-
If I do a File>Export as Catalog with all images selected and then re import that new catalog file, do I have to regenerate the preview file?
Thanks,
Stan
-
Yes - if you don't include the previews in the export. It's an option.
-
John, good points. Although it states that it does the file check which it had finished that portion. It was after checking for dupes. I will do a couple more things before Monday and see.. Thanks. Just to clarify, when you mention to select all pictures, I wish LR would use more specific vocabulary, as often when we say pictures, it should or could mean the reference file, or preview file..... as I always think of picture as the main file/ the raw or main derivatives that reside on a server. Maybe this is open to interpretation, and that makes things unclear.
Anyway, I will try it out. thanks to all posts I'm sure it will clear up.
-
Currently as of today I have 41,558 images (mostly Raw) stored in my Lightroom 4.3 catalog. Main computer being used is an Apple 15" MBP Retina with 16GB Ram. The images are stored on an external Seagate 3TB Thunderbolt Hard Drive (write/read average 190MB/s) along with a number of USB 3 external drives (backups).
-
130,000: RAW and PSD files mostly. All image files, catalog and preview files are on a FW800 drive. A maxed out iMac, but without a SSD drive, is the host. The performance is pretty bad when working from the bulky master catalog, so I tend to work from much smaller assignment-based catalogs, and then import final work, once it's complete, into the big master catalog. I gather from reading here that the biggest improvement I might find would be in placing the catalog file on the boot drive, and making the boot drive a SSD - but I've not tried this.
John Caldwell
-
125k mostly DNG with about 20k TIF. Catalog on an SSD, other files on fast RAID0, everything is instant.
-
...Catalog on an SSD, other files on fast RAID0, everything is instant.
Regarding the practice of keeping the catalog on an SSD, am I right that the same SSD is also best used for operating system and program files, in order to achieve the sppeding up of LR?
Thank you,
John Caldwell
-
Regarding the practice of keeping the catalog on an SSD, am I right that the same SSD is also best used for operating system and program files, in order to achieve the sppeding up of LR?
Keeping everything on an SSD is the fastest, given SSDs are substantially faster than spinning drives these days. It's just a question of how much you can fit.
But yes, SSD for OS, programs, most data files, and Lightroom catalog is good. You can keep the RAW files on a mechanical drive as they're infrequently read. You may wish to keep the previews and camera cache on the SSD.
-
CatOne, Thank you. I better get up to speed with the technique you suggest. As it now is, with catalog, preview and image files an a FW800 rotating drive: I dread doing any work from my large catalog. Small catalogs, say 5000 or fewer files, are fine with that technique though.
Many thanks,
John Caldwell
-
There are 177k images in my main 'work' catalogue. The pictures are a mix of raw and jpeg. For weddings I usually dump the raw files after a year or so and just keep jpegs. The system is a 2011 MacPro quad core with 16gb of ram. Pictures are stored on an internal raid for recent stuff, and an external DROBO for older work. All seems to run smoothly. I have a separate catalogue for personal work.
Jim
-
There are 177k images in my main 'work' catalogue.
Jim, Where is your catalog file? Are any SSD involved?
Thank you,
John Caldwell
-
Hi LR users,
I wanted to know the number of images you have in one catalog. Please post your numbers and your experience in performance at these numbers.
Thank you!!!
Currently 91,046 images of which most are RAW, movies, TIFF and PSD files, virtual copies. Total about 1.7TB. It's all on my 2012 MacBook Pro 2.6Ghz, 16GB RAM and 1TB SSD. I have seen no slowdown due to catalog size and all single image operations happen immediately. I have all my images in one catalog, otherwise I would not use the power of Lightroom.
-
...Total about 1.7TB. It's all on my 2012 MacBook Pro 2.6Ghz, 16GB RAM and 1TB SSD.
Hans, your image files live on what type of drive please?
John-
-
Hans, your image files live on what type of drive please?
John-
I should have added that my images live partly on the MBP SSD's and on an external 3TB USB 3 drive. The most recent images are on the MBP and are moved over to the external drive after some time and when I'm mostly finished editing them.
-
136,411 in my main catalog. About 20k in two other catalogs. I have all my image files (12 years worth), plus the LR catalogs, previews (etc), on a single WD Black Caviar 3TB drive. It's plugged into a USB 3.0 drive dock.
This makes it easy to back up the entire lot to other drives, which I do both nightly, and rotate other drives in and out of a fireproof safe and a remote storage location.
I could get a tiny perf boost by separating the catalog and the images, but it isn't worth the ease of use (peace of mind).
When I began with LR, all my images fit on a 1TB drive. As my images volume grows, the drives keep pace. I'm now using a 4TB drive for some other files I manage and it works beautifully.
-
Regarding the practice of keeping the catalog on an SSD, am I right that the same SSD is also best used for operating system and program files, in order to achieve the sppeding up of LR?
Thank you,
John Caldwell
Yes. I would keep everything on SSDs if they were cheaper. What i do is import to the SSD, work on the project on the SSD and when it's complete 'archive it' to the RAID0 array of HDDs. Still very fast when I need it but this way I enjoy the maximum SSD benefit. I also do not use RAW, I convert to DNG on import, no cache files to worry about anymore either.
-
Jim, Where is your catalog file? Are any SSD involved?
Thank you,
John Caldwell
Hi John
I have two discs in my Mac set up as a mirrored RAID which hold the most recent files, and the Catalogue also resides here. It is backed up onto another disc too. The drives are 3TB ones.
Jim
-
Well, I did manage to get them into the "main" catalog. 184,127 for now. I likely have another 80K to add. This is my personal, stock and "events" work. My client work is another catalog.
I think I will have to be much stricter on the "keepers". More than I have been, as I delete blurred or test shots, but my instinct to document tells me to keep others :-)
-
One catalogue, 125,600 images mostly CR2, some psd ( well all the edits) and some tiffs from advertising.
A separate scratch drive, for PS and LR. An old Mac Pro 2x dual core 2.66mhz, 9 GB ram. Images on a 7200 rpm internal.
IF I went to a SSD can I just move the dat files on the scratch disk to the new one or will they be lost as the directory is different?
-
125K would take a bit of time to populate and re-cache, as you work through them.
It maybe a good time to do a clean install of OS and apps. At that point you can create the cache folder on the scratch. Once you move over the dat files, then put the catalog on the OS drive, when you open it, you would hope all points to the same location.
Cross your fingers.
If you were upgrading 1 SSD drive wouldn't you want that to be the OS drive?
I have both the OS and Scratch on separate drives and they are both SSD. But left the cache on the OS drive. I wonder if it would make a difference being on the cache? Not sure of the read access timings.
I have read that the SSD drive can "wear" faster since the scratch gets a lot of repeat use with R/WR. I still use one as the one I bought has a 10 year warranty! (64GB). I have had earlier SSD simply wipe all info off the drive. I don't know if this has been corrected over time with the different controllers on the drives, but I have a Raptor HDD with a backup in case. (the cache being on the OS drive is also backed up)
à bientôt