I think that you are confusing LR catalogs with your image files. Your image files may be, and apparently are, in multiple folders. You don't need to touch them to upgrade.
The LR catalog is one file, in one folder, that contains information about your image files. Open that single catalog file with LR4 and it will upgrade it and you will be on your way.