Even where they drive on the right, they still seem to mount a horse from the left. Also said to be back to how you wear your sword.
Mounting a horse is more natural from the left, which is probably why riding (or at least mounting and passing) used to be mostly on the left -- until the French revolutionaries swapped it and then Napoleon imposed it on much Europe, and the rest of Europe gradually feel into conformity out of convenience (many parts of Europe used the left side into the 20th century). Note that all the great island nations have stayed lefties: the UK, Eire, Japan, Indonesia, Australia, New Zealand ...
And what was good for horses is still good for bicycles: as a cyclist, riding on the left has a distinct advantage of letting you mount from the curb side of the bike when pushing off into traffic.