I have several Canon "L" lenses: 16-35, 24-70, 70-200 f2.8 IS, and 100-400 IS. I have a B+W 77mm UV Multicoated filter "permanently" attached to each one. I also always use the appropriate Canon lens hood for each lens.
I have taken shots of the same subject, mostly landscapes, wth the camera on a tripod with and without the filters. I blow them up in Photoshop using a 21" LCD and I can't see any difference in sharpness. So in my own informal tests, I decided to leave the filters on all of the time.
I think that the key is purchasing the highest quality filters, like B+W, and not Hoya, Tiffen or others that are stamped from sheets of glass instead of precision ground.
My 2 cents.