Or don't want to carry them, or use a tripod.
I'd buy myself a 500/4 in a heartbeat if it were easy to tote about.
Also, I can't even stand the attention I get with the 100-400. The 500 would draw attention of more bored and boring people, from a greater radius.
I think I probably used an inappropriate expression. For something that costs no more than a small motor car, 'can't afford' implies that one is a pauper.
I meant to say, 'can't justify the expense'. There are quite a few expensive Canon lenses which I'd love to be able use in certain circumstances, such as the 85/1.2, 300 & 400/2.8, 600/4 etc, but I know from experience that I would be unlikely to use such lenses often enough to justify their purchase, not only because of their weight but because in general I find zooms more useful.
The lenses I use most are the Sigma 15-30, Canon 24-105 IS and Canon 100-400 IS. My main complaint with the 100-400 is sharpness is a bit lacking at full aperture. When I upgraded from a D60 to a 20D, I felt my 100-400 lens had also been upgraded because I could more often avoid using f5.6, and in general use faster shutter speeds, as a result of the very much improved noise of the 20D at high ISOs, but I'm disappointed both the 20D and 5D cannot autofocus at f8 because I would then use my 1.4x extender much more.
Cameras and lenses are tools, and tools are to be used.