I did it on my Thinkpad T61p with the RC version, I thought, "Well, 64 bit is here, might as well jump in". I haven't really suffered that much, to my knowledge. I don't buy the idea of doing anything but a clean install though, no matter how painful reinstalling all the apps is.
To be honest, I see no real, obvious, day to day differences between 32 and 64 - (I'm not a speed freak) although I might if I wanted to address 16Gb of RAM. I ignored Vista and went straight to W7, although I STILL have XP on it's original partition. I don,t have lots of PS plugins, there may be a problem here, you'd have to check with each plugin supplier.
So my advice would be:
Do it
Do a clean install
if you're still nervy:
Install W7 32 bit.
Keep your existing OS, install W7 to another partition.
W7 isn't perfect, but it's better than any Vista setup I've seen.