I'll use grad filters only under certain conditions. First is that the SBR exceeds the dynamic range of my sensor (I'm shooting digital) and second, there are no objects (such as trees) that extend up into the highlight region. For example, I might encounter a scene such as Moraine Lake in Banff NP in Canada. Quite often, clouds cover the peaks surrounding the lake. These clouds are often very bright and the lake is quite dark since it might be in shade early or late in the evening. I could just shoot the lake, ignoring the clouds and peaks, or I could just shoot the peaks and clouds and ignore the lake. If I want both areas with detail then I think about using a grad filter.
How do I know if I need a grad? Well, most digital cameras have about a five stop (or more) dynamic range. You need to test this yourself and it isn't completely objective (it's up to you to decide just how dark shadows can get before they are too dark). For the sake of argument, say we have five stops to play with. This means that I can see detail in a print or on a monitor in areas that are five stops apart in brightness. Let's suppose I meter the clouds and they read EV 16. Most spot meters allow readings in EV's or exposure values. Then I meter the lake and I see that it is reading EV 9. That's a 7 stop difference. Either my highlights (the clouds) or the shadows (the lake), or both, will lack detail, depending how I determine exposure. In this case, if I place a two step grad filter, I'm hoping to bring down the highlights two stops to be 5 stops, equivalent to my dynamic range. But the clouds may not be stopped down two stops since the filter is only 2 stops at the top edge. In this case, I might reach for a 3 stop grad, try that, and then go back to the 2 stop for insurance. The placement of the graduated portion of the filter matters too. Using more strength than is needed can be quite counterproductive. You also have to remember that your foreground exposure will increase if your highlight exposure is increased. Something bright in the foreground might be overexposed as a consequence (I had a stick in the water in one of my shots that got overexposed and it looks like there is a bolt of lightning in the water...).
So I base the whole thing on knowing my dynamic range and what the SBR of the scene is. Then I try to position the camera so that I don't have trees or anything vertical protruding into the highlight region since trees will be dark on top and light on the bottom.
Placing the grad properly involves using your depth of field preview button if you have one. Stopping down the lens makes the grad line more obvious in the frame. Jiggling the filter a bit helps you see where the line is.
Also look for really neutral grad filters. I've been using Hitech and Singh-Ray's. If you can afford them, go for the Singh-Ray filters. Never buy the round grads. Only buy the square or rectangular systems. The round ones cannot be adjusted as far as where the grad line is falling.