This has become a big subject and many older threads will be very out of date.
There are many different solutions to geotagging, what works best for you will vary with your specific requirements; What sort of accuracy you need, what software you use already, what OS you're using, what sort of unit you're prepared to carry.
Most smartphones with GPS can use an app to create GPS logs in standard GPX data format, whilst very convenient, battery life can be hit hard with the GPS enabled all day.
If you're not far from your car many Sat Navs will also save a GPS log of the route that may suit you.
Beyond that you're into either handheld GPS navigation units like the Garmin etrex or a dedicated data logger. The former have good battery life, but tend to be a bit big. The data loggers can be tiny, cheap and surprisingly accurate, but some need their own special software to get the data into standard GPX format.
There are also clip on units that fit onto your camera and will tag the images as taken. Convenient, but expensive and rare, but also not very helpful if you've more than one camera.
I now use a combination approach when travelling. Keeping the Sat Nav in the car on all the time to record a base track, I use a Garmin Nuvi. Then keeping a small data logger(an i747 from eBay) in my top pocket when walking. The Garmin saves the tracks internally and can just be dragged and dropped from Windows Explorer into a suitable folder. The i747 is more complex as the tracks need to be downloaded via it's own software and re-saved as a GPX file.
I really couldn't recommend the i747 now as the software uses an old Google map API that been cancelled and the software isn't 64bit, so it's a bit of a clunky workflow, but the unit is very accurate and compact and my travel netbook is happy running the software anyway. The final advantage for me with the data logger is that it can record waypoints and taking a photo of the unit whilst pressing the waypoint save button provides a good way of matching GPS and camera time later.
If I forget anything, my phone does pretty well.
With respect to tagging the images; The map module in Lightroom is very good and easy to use, I don't bother with anything else now.