I have about 50,000 images all together. I've indexed about 6,000 of them - various folders on all six drives. I plan on indexing all of them.
I have 42,000 images, starting off with a D-Lux, them a Konica Minolta A2 and then a Canon 5D and like you I found that everthing slowed to a crawl, and I was about to throw in the towel...
However I solved my problem ... by going to file>catalog settings>metadata and unchecking 'Automatically write changes into XMP'. I'm currently reorganising my keyword tags into a nested structure, so am changing vast amounts of metadata. Once I've got it sorted I may try turning it back on again, but casual observation on my mac 'activity monitor' (in applications>utilities) shows lightroom grabbing 90% of CPU time when 'automatically write to XMP' is switched back on, vast periods of cogitating silence with an occasional dramatic burst of disk writing every hour or so, but far from complete after 12 hours. If I can't solve this I'll just do the update manually in batches, and I will be inputting through lightroom in future so the issue should become less pressing.
Actually I compounded my difficulties as I had been trialling lightroom against aperture (came to the conclusion that they were both pretty good, just different - but I seem to have chosen lightroom) on an external drive, and had checked 'automatically write to XMP' as the final act before I copied the library over to the internal disk, and it was then also simultanously checking on the location of every file as well as going through the 'XMP crawl'.
Also, I was finding that creating previews for every picture (by selecting Library > 'all photographs') on the left hand strip was taking ages at 2048 image size, so I decided on a strategy of 1024 for all photos on the grounds that 90% will rarely be looked at, and then created higher resolution previews forthe perhaps 15% of total 4 and 5 star rated pictures. The catalogue and previews are now on a MacBook Pro (2.4 GHz, 7200 rpm internal 160GB drive and 17 inch screen, 2GB RAM). The Lightroom.Ircat file is just under a MB and the Lightroom Previews.Irdata file is 27GB. The photos and their backups are on external drives, some 650GB. It means that I haven't had to make a choice of which files to keep on laptop, even the 1024 previews are fine for what I need in most cases, and I don't have to lug an external drive around. Performance is fine either stand alone or hooked up to externals and working on full resolution files, most of which are RAW.
I hope this is what your problem is - ps I think others on this forum have mentioned the same issue and solution
PPS I'm idly curious as to why it is updating metadata takes so long, if anybody knows ...
PPPS I see that adobe has responded along the same lines on their forum
I should have thought to check there before writing this!