Whilst travelling with a lightweight, ball-head tripod, as I am currently, I have an L-bracket attached to both cameras, the 5D and 20D, and a clamp attached to the ball-head. This allows for better stability when orientating the camera in a vertical position as well as allowing the camera back to be moved in an opposite direction to a lens shift with the TS-E 24mm, which I'm also carrying.
However, taking multiple shots for stitching purposes sometimes needs to be done rapidly when the scene is changing for whatever reason. At Angkor Wat, for example, during the extra time it takes to shift the camera body and re-tighten the clamp, a group of tourists is quite likely to wander into the scene, or, when the sun appears briefly from behind the clouds, it can begin disappear again before the shooting is complete. In such circumstances, speed is of the essence, so I found myself abandoning this approach of shifting the camera back.
However, as I have a professonal stitchng program, Image Assembler, which can use up to 7 pairs of flags at each join, I have no problem getting a seamless stitch whether I shift the camera body or not, or whether there are objects in the foreground or not. However, if I'm not usng a shift lens, objects in the near foreground can be a problem.