I have the 35mm f/2 and the 24-105mm f/4L, and subjectively I have to agree that the zoom is very good, on a par with the prime and maybe a bit better when it comes to IQ... I'm no pixel peeper, but my own subjective impressions bear out this test.
However, the prime still spends a lot of time on the front of my 5D for everyday "street" and candid people photography. Why?
1. F/2. Yes, the zoom has IS and the prime doesn't, but this doesn't help with subject motion blur, nor does it help the AF to perform accurately in low light.
2. The zoom weighs 670g and is the size of a beer can, the prime weighs 210g and is the size of a small coffee cup. I tend to feel a bit self concious when I'm carrying the big zoom around, and I'm sure this rubs off on my subjects. I tend to get much better shots of people when I'm using smaller equipment.
Just my 2p worth.
Ed