I mostly use the Lowepro Slingshot 200, and swear by it, but only for carrying the camera with one lens (plus the usual bits and pieces). This bag is not only excellent for quick access, it also acts like a table when it is in the front access position, which is great for applying filters, keeping the lens cap safe, etc. In the winter, I can place my mitts between the bag and my chest, keeping them out of the snow while I work.
I also have the Lowepro Flipside for when I want to carry more lenses and a tripod. This one works like a table, too, but you can't keep it out front for too long, because it hangs off your waist by a very narrow strap. It is also not quite as quick to access as the Slingshot, but still pretty good.
I would never dream of having a camera loose on my body (around the neck, or shoulder) when scrambling around on hills, rocks, etc.