I'm (still) wrangling with some of these issues.
I have the 500/4 IS and optically it is all it is reputed to be. I have hiked with it on a CF tripod and Sidekick with it over my shoulder, so set-up time is as long as it takes to put the tripod down. As others, often used with 1.4x with good success. I tried a similar rig but with a monopod and swivel head/Sidekick combo : lighter and easier to hike with but lower ratio of keepers (or another super tele technique learning curve to improve on).
What kind of wildlife are you thinking of - skittish small (4-6") birds or something a little bit less challenging? If it is these smaller critters, 500 is a minimum (with frequent 1.4x and often 2x) unless you have the dedication and time for hides etc. If they are not the main priority, you might well get away with the 400DO (± converters) much of the time with all the consequent weight, spontaneity and hikability benefits you mention. So physical fitness, hiking distance and target species play a role.
Transportability is important - it's tough getting the 500 on planes these days, while the 400DO is not only lighter but significantly less bulky. For car travel and "do-able" hikes to all your anticipated sites, again, not an issue. How often would such issues be a concern for your anticipated shoots?
If, however, reach is really important, such that you expect not to be able to fill the frame with the subject at the time of exposure, and will be cropping, then the issue is not FF or crop, but pixel density which will affect image size in pixels and therefore maximum print size. In that case, the longer the lens and the higher the pixel density (such as 20D/30D) the better - all other things being equal, which of course, they never are. There are the normal prePhotokina rumours about a (long awaited) 1 series full frame Canon with 22MP which brings the best AF system together with the high pixel density, but with the 1 series weight.
No answers, I'm afraid but perhaps a couple more issues weigh in the balance pan of priorities and compromises For me, I'm leaning towards the 400DO as a lens I would be likely to have with me on more occasions and therefore use more often and from that perspective, be of better value. I'll just have to be a little more stealthy....