I use a combination of GPS loggers.
Around civilisation I use the GeoLogTag app on my iPhone to record trails. It does chew battery a bit, and it does make the phone a bit sluggish compared to having the logging turned off, but it's really convenient to just always have the data being saved by a device that's always with me.
I have a tiny Qstarz logger which I use on trips. I've configured this to have over a week's worth of space, but it does need specific software (BT747 works fine) to download the data via USB and convert to GPX format. It uses Nokia BL-5C/6C batteries, and I have several and an external charger so I can cycle a new battery into it each day to make sure there are no gaps in the data.
But my primary device is still my Garmin eTrex Legend HCx (which is the same as the Vista HCx except without a magnetic compass or barometric altimeter). As has been mentioned, it gets great battery life (and uses AAs like my flashes/etc), it's waterproof, it has a screen for navigation, and in fact I've loaded up lots of maps on the internal microSD card. Topo maps of my home continent, a worldwide basemap, and additional maps of regions such as Antarctica.
With the Garmin, you set it up in the menu to record tracks, and set the time interval you want it to record a location. Having done that, it just records automatically when you turn the device on. You then connect the Garmin to the computer with a USB cable and import the tracks into the geotaggins software you use. Some geotagging software will not import directly from the Garmin, in that case you have to use the Garmin free software do download the track and save it to the computer as a gpx file which you then open in the geotagging program.
Actually, it's even easier than that! No Garmin-specific software is required.
If you're using one of the HCx units and you've installed a microSD card, head into Main Menu
-> Data Card Setup
where you can tell the GPS to record GPX files directly onto the card.
I've set mine to have the normal internal tracklog wrapping when full (so it forgets old data) but this is just used for drawing breadcrumb trails on the screen. The GPX files just keep getting created (one per day, with names such as 20110212.gpx) until the card fills. I have a 1 GB card in mine, with about 300 MB taken up with the map data. There's enough room there for months of trail data.
To get this data off the device doesn't need any special software either. Go to Main Menu
where you can select USB Mass Storage, at which point the GPS becomes a USB flash drive: it gets mounted on your PC/Mac and you just copy/move the GPX files to somewhere else. Power-cycle the GPS and it goes back to normal.
If you're not using a device with a microSD card then yes you need special software to extract the internal tracklog data and save GPX files. I used to do this with my older Garmins, but you are limited by the number of log points supported in the device's internal memory. Prior to getting the Legend HCx I was using a plain Legend (the blue one) which usually had enough storage for a whole day (depending on the day's movements though).
Once you are in the geotagging software, the geotagging software will match up the timestamps of the trackpoints with the timestamps of the camera images, so the matchup is done automatically. Most programs will let you set a fudge factor (for instance, I set the Garmin to record a track point every couple of minutes (if I'm walking), and tell Geotagger to match the trackpoint to any image within 90 seconds or so of it). It's actually pretty straightforward once you get it set up.
Actually the biggest issue may be how to organise your collections of GPX files. I have mine split by device, then in a date-tree of folders.
I really would like someone to write a Geotagging script for Bridge so I could dispense with the separate geotagging program and incorporate it into my image management workflow in Bridge. But that's a different topic...
I use Jeffrey Friedl's Lightroom plug-in. It allows me to attach location data to images independently from importing them. Often I'll import photos during the day, but only get around to using the appropriate GPX files later.
For instance, on ship operations such as in Antarctica I have the Qstarz fixed on the ship where it can see satellites, giving me a backup trail of where the ship went. When on-deck, on-shore, or in boats my eTrex is recording trails (it doesn't work inside the ship's steel body). So later in Lightroom I'll select a chunk of photos, and repeatedly run the Geocoding plug-in across them. First each appropriate day's GPX file from the eTrex, then the appropriate file from the Qstarz to fill in the gaps when I was inside. If I forgot to take the eTrex with me, to have fresh batteries in it, etc, at least the photos will get the location of the nearby ship as a guide.
As you can imagine, having a decent filing/naming system for the GPX files makes this process easier.
I do need to be careful about things such as leaving a camera on a tripod with an intervalometer and walking away somewhere else with the logger in my pocket, so I find the interactive process of the Lightroom plug-in to work well (e.g. using the metadata filter for "GPS Shadow" status to see which images don't yet have locations).