I have owned the non-IS 70-200, and now use the IS version. I have not used the f4 version but from reports from many users the f4 and f2.8 non-IS comparable.
The IS in this lens is quite spectacular and makes non-tripod photography possible again. Your choice however will depending on your style of photography as well as your pocket. If you take it slow, a sort of contemplative photography using a tripod.. then the IS may not be necessary. If you do any hand-holding though, even occasionally, then the IS is a must.
One con on the IS version... the bokeh is somewhat different to the non-IS version, and in some circumstances distracting. I have yet to work out what combintations of factors that brings this "funny" bokeh on, but it is nevertheless there... Having said that in the same photographic situation with a non-IS lens, I would probably just have ended up with a fuzzy shot.. :-)
Good luck in your decision, but for me the IS in this lens was really worth every penny, as I love hand-holding candid photography.