Moving the lens changes the point of view.
That makes fine tuning composition more challenging and stitching more problematic.
Wherever possible, and otherwise not problematic (e.g. keeping the lens static requires an adapter that is a less secure mounting point than on the body), it is preferable to move the back. The point of view remains 100% static.
Note that the challenges above may or may not affect a specific photographers workflow. They will be more problematic when you have significant subject matter in the foreground close to the camera (rather than just stuff at/near infinity) and more significant if you do any shift-and-stitching.
Doing a quick left-right panoramic stitch by moving the back is one of my favorite parts of using a tech camera like the
Arca Swiss RM3Di. The corners and edges of the final frame stay put (unlike rotate-and-stitch where the image gets warped) meaning I can do the composition in-the-field.