Haring,
There are a lot of good information in this post, and I think everyone can confirm that they are all valid. Maybe it could help to sythesize :
Your CPU is already a great piece of hardware, even if it's not the last generation. Changing it would mean changing the motherboard and re-installing everything which is likely to be a costly and tedious solution.
If you're working in Lightroom primarily, I don't think a RAM upgrade would be beneficial. IF you're doing a lot of stitching (panoramas & such) from your 24MP if might help a bit BUT it won't be night and day even with 32GB of RAM.
The graphic card... Well if you use the integrated GPU at the moment, any GPU would be beneficial. The higher the better. Just don't buy "professional" graphic cards since these are underpowered and overpriced (I'm working in mechanical engineering and have been using a lot of CAD softwares for which these pro cards have always been recommended, well... we saved thousands of $ and gained performances by using "gaming" cards for years). If choosing Nvidia, stay in the x70 / x80 (780, 980) range, lower tier offering (960, 860, etc.) are really underpowered.
What will really make you gain a lot... I mean, a LOT of speed when switching between two RAWs and changing modules in LR, is an SSD drive. The main bottleneck in any PC is the hard drive (HDD). Even a cheap SSD is several times faster than the best HDD and will load your heavy 24MP files several times faster. It is also less (if at all) affected by simultaneous write/read operation, while a HDD slows down exponentially with an increased number of simultaneous operations. For the SSD to work however, you'll need to have both LR and your catalog and RAW files on it.
tl;dr : buy a bloody SSD drive for LR, your catalog and your raw files. In all cases its the cheapest update you could do after adding RAM, and it will bring you a much more enjoyable experience with your PC. If this is not enough, then a high-end graphic card will also help.
Also note that LR is badly optimized, just as every high-ressource piece of software out there, and I don't think that even the best hardware will allow you to have a real lag and latence-free experience.