Pages: [1] 2   Go Down

Author Topic: Lightroom Library Strategy  (Read 17888 times)

cookielida

  • Newbie
  • *
  • Offline Offline
  • Posts: 29
    • http://
Lightroom Library Strategy
« on: October 06, 2007, 10:43:24 am »

Hey All!
Until today, the above question wasn't of an issue - I thought that the best way is just to create a library for each month, thus have a comparatively lighter library (with several hundreds of photos) that will respond quickly and also I will bypass the "all-eggs-in-one-basket" issues. However, I noticed that:
a) On each opening of a new library I loose user presets as well as Metadata presets I have configured in the past.
 I will loose the ability to do a search of ALL photos, as LR doesn't enable the search of several libraries.

I would like to hear what's you strategy and how do you manage with big libraries of thousands of photos?
Is it worth to have a big mega library and then each month export to a new catalogue, thus keeping user presets and Metadata configuration?

10x for your time!
Chen
Logged

john beardsworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4755
    • My photography site
Lightroom Library Strategy
« Reply #1 on: October 06, 2007, 12:26:55 pm »

As you've found, just because it's easy to create multiple libraries doesn't mean you should do so for all your work. You've noticed a couple of disadvantages - there's also a risk that images will get into multiple catalogues or slip between the gaps between catalogues. For normal work, use one catalogue unless performance is really too slow, and back it up properly to take out the risk. But take advantage of multi catalogue features when you move work between computers, or as a temporary working space.

Does a book have a separate index for each chapter?

John
Logged

Nat Coalson

  • Full Member
  • ***
  • Offline Offline
  • Posts: 195
    • http://www.NatCoalson.com/
Lightroom Library Strategy
« Reply #2 on: October 07, 2007, 12:21:06 am »

Deciding how to set up your Lightroom catalogs (databases) is as important as establishing a file naming convention and standardizing your file/folder architecture.

Generally, most people will benefit from using just one Lightroom database. If you follow good working habits your single Lightroom database can contain tens of thousands of images without noticeable performance degradation. My main database contains 25,000 files with no trouble.

If, for some reason, you choose to use separate databases, the images in those databases should be those that you expect will never have any relationship.

For example, if you're shooting for separate clients, and won't have any crossover in the usage of those images, separate databases is good there.

If you have a HUGE body of work and want to put different subject matter in separate databases - weddings, portraits, architecture, etc., that might make sense.

Where you'll run into bottlenecks, though, is when you want to combine these separate images into collections or portfolios of different bodies of work with images located in different databases. In this case, at some point, you'll need to re-import images to combine your selects into one database.

So work with one database until you have a strong reason not to.
Logged
Nathaniel Coalson
Author of [url=http://

cookielida

  • Newbie
  • *
  • Offline Offline
  • Posts: 29
    • http://
Lightroom Library Strategy
« Reply #3 on: October 07, 2007, 03:36:26 am »

Quote
Deciding how to set up your Lightroom catalogs (databases) is as important as establishing a file naming convention and standardizing your file/folder architecture.

Generally, most people will benefit from using just one Lightroom database. If you follow good working habits your single Lightroom database can contain tens of thousands of images without noticeable performance degradation. My main database contains 25,000 files with no trouble.

If, for some reason, you choose to use separate databases, the images in those databases should be those that you expect will never have any relationship.

For example, if you're shooting for separate clients, and won't have any crossover in the usage of those images, separate databases is good there.

If you have a HUGE body of work and want to put different subject matter in separate databases - weddings, portraits, architecture, etc., that might make sense.

Where you'll run into bottlenecks, though, is when you want to combine these separate images into collections or portfolios of different bodies of work with images located in different databases. In this case, at some point, you'll need to re-import images to combine your selects into one database.

So work with one database until you have a strong reason not to.
[a href=\"index.php?act=findpost&pid=144325\"][{POST_SNAPBACK}][/a]

Hey John and Nat!
Thanks for your tips - I will see how I will work things out in my case...
Chen
Logged

Goodlistener

  • Full Member
  • ***
  • Offline Offline
  • Posts: 120
    • http://www.pbase.com/goodlistener
Lightroom Library Strategy
« Reply #4 on: October 07, 2007, 12:30:14 pm »

I'm not clear on the difference between a Catalog and a Library.  

A few things I do understand but want to confirm:  Somewhere in the background, LR keeps a database that simply lists all of the edits applied against a particular image file. When you view the image, those edits are applied to the display of the image.  My understanding is that the edits are not applied to the image, unless and until one exports the image, at which time the edits become "baked in".

But what is the diffference beween Catalog and Library?  Collection is simply a set of pointers to individual images.

BTW, these forusm are GREAT! for sharing and extending knowedge.  Thanks everyone very much.
Logged

john beardsworth

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4755
    • My photography site
Lightroom Library Strategy
« Reply #5 on: October 07, 2007, 12:57:20 pm »

At first the database was called a library, but that confused some people because there is also the Library module. So now you call the database a catalogue.

John
Logged

reyn_two

  • Newbie
  • *
  • Offline Offline
  • Posts: 34
Lightroom Library Strategy
« Reply #6 on: October 07, 2007, 02:23:58 pm »

Quote
At first the database was called a library, but that confused some people because there is also the Library module. So now you call the database a catalogue.

John
[a href=\"index.php?act=findpost&pid=144404\"][{POST_SNAPBACK}][/a]

So we have a Catalogue (which is really the database), we have Folders, then we have Collections and last but not least we have a Library in the Library module which contains a Quick Collection.
In my simple way I have to see it that I have one or more Databases which contains data, I have Folders (Directories) which contain image files that's it as far as the real world is involved. We then have the virtual world where anything's possible, Collections and their ilk (Virtual Copies, Stacks) are organisational sub entities which allow you to group/duplicate images in various ways.

By the way, why does my UK version of Lightroom have Catalogue spelt incorrectly, and if it's an international version same goes.

Michael - how do Canadians spell Catalogue?  
Frank
Logged

fnagy

  • Newbie
  • *
  • Offline Offline
  • Posts: 35
    • www.franknagyphotography.com
Lightroom Library Strategy
« Reply #7 on: October 09, 2007, 08:26:26 pm »

Quote
So we have a Catalogue (which is really the database), we have Folders, then we have Collections and last but not least we have a Library in the Library module which contains a Quick Collection.
In my simple way I have to see it that I have one or more Databases which contains data, I have Folders (Directories) which contain image files that's it as far as the real world is involved. We then have the virtual world where anything's possible, Collections and their ilk (Virtual Copies, Stacks) are organisational sub entities which allow you to group/duplicate images in various ways.

By the way, why does my UK version of Lightroom have Catalogue spelt incorrectly, and if it's an international version same goes.

Michael - how do Canadians spell Catalogue?   
Frank
[a href=\"index.php?act=findpost&pid=144419\"][{POST_SNAPBACK}][/a]


Face it!  There is the International English (US spellings) and you and I are in small numbers and limited to a couple of peculiar countries where we cling to the "Oxford" Dictionary as in catalogue.
Logged
Love & Peace
Frank

Wayne Fox

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4237
    • waynefox.com
Lightroom Library Strategy
« Reply #8 on: October 24, 2007, 07:14:48 pm »

Quote
Generally, most people will benefit from using just one Lightroom database. If you follow good working habits your single Lightroom database can contain tens of thousands of images without noticeable performance degradation. My main database contains 25,000 files with no trouble.

[a href=\"index.php?act=findpost&pid=144325\"][{POST_SNAPBACK}][/a]


I'm not sure what you mean by good working habits, but LR is getting so slow for me I'm using Bridge/CS3 more and more.  I probably only have about 10k images, Running on a dual Quad MacPro, 8gb of ram with 1.5 terabytes of Raid 0+1 hard drive space, with 750gb free.  Some of my files are pretty large (Phase One P45), but most of the time I"m twiddling my thumbs when using the library function to look at individual images and it doesn't matter which camera they're from. (mixture going all the way back to DCS back and original 1Ds).

I would love a link to "good working habits", as well as the best settings for speed that might make it more efficient.

Thanks
Wayne
Logged

mikeojohnson

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Lightroom Library Strategy
« Reply #9 on: October 24, 2007, 08:57:45 pm »

Quote
I'm not sure what you mean by good working habits, but LR is getting so slow for me I'm using Bridge/CS3 more and more.  I probably only have about 10k images, Running on a dual Quad MacPro, 8gb of ram with 1.5 terabytes of Raid 0+1 hard drive space, with 750gb free.  Some of my files are pretty large (Phase One P45), but most of the time I"m twiddling my thumbs when using the library function to look at individual images and it doesn't matter which camera they're from. (mixture going all the way back to DCS back and original 1Ds).

I would love a link to "good working habits", as well as the best settings for speed that might make it more efficient.

Thanks
Wayne
[a href=\"index.php?act=findpost&pid=148490\"][{POST_SNAPBACK}][/a]

If you go to preferences, then find the catalog preferences, then the metadata tab, check to see if you have the automatically write xmp changes checked.  If you do, uncheck it and you will see the performance increase significantly.  I have a 60k image data base on a similar mac and have almost no delays in switching between images, or folders, or...
mike
Logged

James R

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 364
Lightroom Library Strategy
« Reply #10 on: October 25, 2007, 10:17:30 am »

Quote
Hey All!
Until today, the above question wasn't of an issue - I thought that the best way is just to create a library for each month, thus have a comparatively lighter library (with several hundreds of photos) that will respond quickly and also I will bypass the "all-eggs-in-one-basket" issues. However, I noticed that:
a) On each opening of a new library I loose user presets as well as Metadata presets I have configured in the past.
 I will loose the ability to do a search of ALL photos, as LR doesn't enable the search of several libraries.

I would like to hear what's you strategy and how do you manage with big libraries of thousands of photos?
Is it worth to have a big mega library and then each month export to a new catalogue, thus keeping user presets and Metadata configuration?

10x for your time!
Chen
[a href=\"index.php?act=findpost&pid=144216\"][{POST_SNAPBACK}][/a]

Keep it all in one basket.  Just develop a good rating system so your best shots are easy to find. 
Logged

jdoolitt

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Lightroom Library Strategy
« Reply #11 on: October 25, 2007, 11:37:12 am »

Chen -

Neither your metadata or develop presets are saved with your catalogues.. so I don't understand how you are losing them??

Me?  I have ~ 24,000 images and use a single catalogue.  I may break that up at some point down the road, but things are plenty fast on my system with a single catalogue.  I don't see an "All eggs in one basket" issue, if you are diligent about backups (both lightroom and your pc).
Logged

jjj

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4728
    • http://www.futtfuttfuttphotography.com
Lightroom Library Strategy
« Reply #12 on: October 25, 2007, 02:56:20 pm »

Quote
If you go to preferences, then find the catalog preferences, then the metadata tab, check to see if you have the automatically write xmp changes checked.  If you do, uncheck it and you will see the performance increase significantly.  I have a 60k image data base on a similar mac and have almost no delays in switching between images, or folders, or...
mike
[a href=\"index.php?act=findpost&pid=148507\"][{POST_SNAPBACK}][/a]
By doing that, you risk losing your edits if your catalogue goes squiffy.
Plus Bridge won't see the edits, so those of us who use both or want to have our edits with the images, like to have edits in XMP.
There are no solutions, only new problems.
Logged
Tradition is the Backbone of the Spinele

jdoolitt

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Lightroom Library Strategy
« Reply #13 on: October 25, 2007, 03:04:02 pm »

Quote
By doing that, you risk losing your edits if your catalogue goes squiffy.
Plus Bridge won't see the edits, so those of us who use both or want to have our edits with the images, like to have edits in XMP.
There are no solutions, only new problems.
[a href=\"index.php?act=findpost&pid=148665\"][{POST_SNAPBACK}][/a]

You don't have to make Lightroom auto-write in oder to share metadata with Camera RAW/Bridge.  You only need to select the images in Lightroom and hit Control-S (Photo-save Metadata to File).  It's two step - but its pretty painless.
Logged

jjj

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4728
    • http://www.futtfuttfuttphotography.com
Lightroom Library Strategy
« Reply #14 on: October 25, 2007, 03:33:44 pm »

Quote
You don't have to make Lightroom auto-write in oder to share metadata with Camera RAW/Bridge.  You only need to select the images in Lightroom and hit Control-S (Photo-save Metadata to File).  It's two step - but its pretty painless.
[a href=\"index.php?act=findpost&pid=148668\"][{POST_SNAPBACK}][/a]
That only solves one problem I mentioned. And adds yet another tedious thing to remember to do when using LR with it's numerous workarounds. Especially if I have to open LR and navigate to the folder where the images are so then I can see the edits in Bridge. Not a good way of working, LR was meant to make dealing with large no.s of images easier, not more complicated.
Logged
Tradition is the Backbone of the Spinele

Wayne Fox

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4237
    • waynefox.com
Lightroom Library Strategy
« Reply #15 on: October 25, 2007, 10:43:24 pm »

Quote
You don't have to make Lightroom auto-write in oder to share metadata with Camera RAW/Bridge.  You only need to select the images in Lightroom and hit Control-S (Photo-save Metadata to File).  It's two step - but its pretty painless.
[a href=\"index.php?act=findpost&pid=148668\"][{POST_SNAPBACK}][/a]

I don't agree it is painless.  Painless is not having to worry about it.  The entire reason for .xmp sidecar files is for access to the information outside of LR and it should be simple to accomplish this. I often copy a folder or 2 while traveling to work on some files.  I don't have a problem telling LR to read in my .xmp changes when I copy the data back over to my  master drive, but I have no clue why LR can't just update the .xmp file when I've made a change, but other than that completely disregard them

LR's logic in regards to the .xmp files seems peculiar, and almost as though LR regards the.xmp files as a nuisance,or someone just didn't think things through.  I've done some testing, and am completely stumped as to why it works like it does.

After opening LR to the last folder opened with .xmp autowrite enabled, it can take a long time to render sharp previews in the library window.  With .xmp autowrite disabled, it renders them very quickly.  It's as though it is reading through the .xmp files, but I'm not sure why, because if you make a change in bridge/ACR, it won't apply that change anyway.  You have it explicitly read in the change. So what is it doing? Why should this setting have any affect on speed at all?

 If you make a change in Bridge, and read in the metadata from the .xmp file, it can take as long as 30-60 seconds.    Sometimes it takes 30 seconds or longer to write out the .xmp file.  If you happen to make a change in ACR, then make a change in LR, it will actually ask your permission to overwrite the file because it has been changed in an external application.

The only purpose I see for LR to use .xmp files is to pass information to other programs.  When I make a change, then move to another file, spit the xmp out and move on.  (This is a 8-12k file ...it should take virtually no overhead)  Other than that, LR should ignore .xmp files unless explicitly told to read them in. How hard is that?
Logged

jdoolitt

  • Newbie
  • *
  • Offline Offline
  • Posts: 11
Lightroom Library Strategy
« Reply #16 on: October 26, 2007, 11:14:51 am »

I should have qualified my "it's painless" comment.  For ME . . it is painless.  I rarely need to see bulk LR changes in ACR.  I typically will need to see a few image xml files updated for ACR to recognize what is there.  That is, when I know I will be importing a RAW image into CS3 as an object - and I want my LR adjustments to be the basis (and on-going) set of adjustments for the Bridge/ACR interaction.  

I've played around quite a bit recently with xmp files, and they can be beefy..  Beefy, if you are parsing, updating, writing a bunch of them.  No wonder auto-write xmp slows things down.  It forces LR to do a bunch of disk io.  

I agree it would be nice if it were automatic, without the overhead.  For ME... it is painless though.  ;~)

Quote
I don't agree it is painless.  Painless is not having to worry about it.  The entire reason for .xmp sidecar files is for access to the information outside of LR and it should be simple to accomplish this. I often copy a folder or 2 while traveling to work on some files.  I don't have a problem telling LR to read in my .xmp changes when I copy the data back over to my  master drive, but I have no clue why LR can't just update the .xmp file when I've made a change, but other than that completely disregard them

LR's logic in regards to the .xmp files seems peculiar, and almost as though LR regards the.xmp files as a nuisance,or someone just didn't think things through.  I've done some testing, and am completely stumped as to why it works like it does.

After opening LR to the last folder opened with .xmp autowrite enabled, it can take a long time to render sharp previews in the library window.  With .xmp autowrite disabled, it renders them very quickly.  It's as though it is reading through the .xmp files, but I'm not sure why, because if you make a change in bridge/ACR, it won't apply that change anyway.  You have it explicitly read in the change. So what is it doing? Why should this setting have any affect on speed at all?

 If you make a change in Bridge, and read in the metadata from the .xmp file, it can take as long as 30-60 seconds.    Sometimes it takes 30 seconds or longer to write out the .xmp file.  If you happen to make a change in ACR, then make a change in LR, it will actually ask your permission to overwrite the file because it has been changed in an external application.

The only purpose I see for LR to use .xmp files is to pass information to other programs.  When I make a change, then move to another file, spit the xmp out and move on.  (This is a 8-12k file ...it should take virtually no overhead)  Other than that, LR should ignore .xmp files unless explicitly told to read them in. How hard is that?
[a href=\"index.php?act=findpost&pid=148735\"][{POST_SNAPBACK}][/a]
Logged

johnvr

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 58
Lightroom Library Strategy
« Reply #17 on: October 31, 2007, 04:07:12 am »

I have three different libraries:

- one for all my personal pictures, i.e. family, friends, vacations etc.
- one for all my non-personal pictures, i.e. subjects that hold no emotional value to me
- one for my model photography, which would normally go into the non-personal pictures library, but since some of it contains nudity and I have two small children, I seperated the two so I can work on non-model pictures without the risk of a model shot popping up

This has the advantage of lessening the clutter when browsing libraries with a specific goal. It also lets me pick keywords that pertain to that library (e.g. people, milestones, activities for the personal one, but stock-driven keywords for the other ones).

All libraries sit on one 500GB external hard drive and all images are backed up on another 500GB external HDD (plus on two more hard drives outside of LR).

The only issue I encounter with this setup is that I sometimes take pictures while on a family outing that belong in the second library, so in order to keep it clean, I have to move them from one library to another.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Lightroom Library Strategy
« Reply #18 on: October 31, 2007, 10:09:06 am »

I started out with 50 gig buckets for catalogs. Have three. But that sucks because the image I want is usually on another catalog. Seth Resnick carries around one HUGE FireWire drive, beat to crap thanks to TSA with one giant library (catalog). That as we know has limitations. Or does it? I guess it depends on how much you shoot but how many photographers can we address that can live (today) with a 1TB catalog and images. I'd personally prefer to travel with one or more FireWire drives that can get their own power from the bus. That means something like 200 gig's at this time (I have a few 120's and 160's).

So I tried something interesting yesterday. Not sure how well it will work out.

I imported JUST the catalog info from all three Catalogs into a new catalog, leaving all the images in the same folder structure. This first presented a problem that was easy to fix. I have two Santa Fe folders in two different libraries. The first of 50gigs has a folder with this name, as does the 2nd on the other catalog, obviously in a different folder structure but one that makes sense to me. So I just made a collection of both. Now I can see all Santa Fe images (I also have keywords to use). I don’t really care now that half the pic’s are in different folders although both folders have the same name (which in a way might make searching in the finder no more difficult than having one folder). The two original folders still remain in the original location, the OLD catalogs are fine seeing this. Did the same with my two Dog related folders. Now should I combine them into one folder and update the catalog? I’m thinking not. If I leave the structures the same, I can launch Catalog 1 or 2 or 3, they still see the data in this bucket (which arguably could and should be larger. I’m thinking a bucket that one can easily fit on 120-160 gig FireWire drive with some left over space for new images). To update the smaller buckets I think I can use Import Catalog without pictures (haven’t tried going the opposite direction) or just Sync Folders if not. Then if I travel, I can copy Catalog 1 to a Firewire drive and work on it/show stuff. Or build a new catalog on location and import into Catalog 4 and start a new bucket. Each individual bucket is imported into the master catalog of all images.

The key here seems to be thinking of one group of catalogs in one location as the Master. For me, its a 5 drive Raid I have which hooks up to my Macbook (go on location, drive remains of course). If I update this group of catalogs, I then sync that to other drivers (fire proof safe, drive for location, plus one non buss powered, big ass Fire Wire drive that can be a complete clone of the Raid.

There isn’t a big overhead for this master, all containing catalog. The question becomes, does it make any sense to build buckets reflected in a catalog that is also contained in the big, master catalog. If you want everything, that’s what you get. If you want a subset, or you want to use buckets of catalogs, does this bring anything useful to the party? Maybe in the end, one big single catalog is all that’s needed. One could export needed images to other drives. But keeping that in sync could be an issue.

One MAJOR advantage of this new, one big catalog is working with keywords! Its so much easier to find any image I want, no matter the location.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

jjj

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4728
    • http://www.futtfuttfuttphotography.com
Lightroom Library Strategy
« Reply #19 on: October 31, 2007, 11:26:12 am »

Quote
I should have qualified my "it's painless" comment.  For ME . . it is painless.  I rarely need to see bulk LR changes in ACR.  I typically will need to see a few image xml files updated for ACR to recognize what is there.  That is, when I know I will be importing a RAW image into CS3 as an object - and I want my LR adjustments to be the basis (and on-going) set of adjustments for the Bridge/ACR interaction. 
It's for the odd few images and not large groups where this is a real pain. To have to open up LR navigate to those images and then save changes, simply to open up in PS via Bridge is a complete PITA. If LR could do all the clever things Bridge can do then it would be less of an issue.
LR's remit was to make things easy for the photographer with lots of images, but it only introduces lots of new complications and too many annoying workarounds.


Quote
I've played around quite a bit recently with xmp files, and they can be beefy..  Beefy, if you are parsing, updating, writing a bunch of them.  No wonder auto-write xmp slows things down.  It forces LR to do a bunch of disk io.[{POST_SNAPBACK}][/a]
XMP files are anything but beefy, they are small text files. And to not use them counteracts what Adobe say on this page.
[a href=\"http://www.adobe.com/products/xmp/]http://www.adobe.com/products/xmp/[/url] especially this paragraph
"Built-in XMP support within Adobe applications fosters metadata capture and exchange for improved workflow and increased production efficiency"
Rendering what the text file says, yes that takes more grunt. But if LR has already rendered the image, adding a few bytes to a text file should not seriously slow down as mentioned above a dual Quad MacPro, 8gb of ram with 1.5 terabytes of Raid 0+1 hard drive space, with 750gb free, unless Adobe have got something very wrong with their software. Have they?
Logged
Tradition is the Backbone of the Spinele
Pages: [1] 2   Go Up