Are you using the rear control pad as a direct focus point selection? Is this not the same method as a Canon 1D or 5D with joystick? Between that and using the Focal Point home Custom button, I find I can jump back and fourth between two points instantly and make faster adjustments than my DSRLs ever could.
I don't know how the 100-300 focuses...but I used the Panasonic 20mm and felt I stepped back in time. The Olympus Pro lenses would give you a much different experience. Unfortunately that comes with a price.
Well a Porsche Carrera and a Trabant both have wheels which are roughly circular (excluding the flat bit at the bottom) but the performance is wildly divergent.
The controls on the EM5 are numb and imprecise, the x/y control set appallingly so. The whole design of the camera is fundamentally dreadful, the firmware spectacularly so. As for the 100-300, it's not that bad at all - the focusing deficiencies are a consequence of the camera's inherently poor CAF and the abysmal af point selection controls. Even getting a focus on a bird sitting still isn't guaranteed; I've become accustomed to losing opportunities much of the time. Obviously the half dozen shorter lenses I have are better, mainly because the subjects shot with them are usually easier to hit.
In comparison with the couple of DSLRs I've owned (I still shoot the antique D700 occasionally) using the EM5 for small moving subjects is an exercise in frustration. Luckily this isn't that important to me. What it has taught me is to ignore the hysteria which accompanies the latest "must have" camera/system that appears on the market. It seems to me that this stuff is usually released as poorly tested and with deficiencies intentionally incorporated in order to maintain the constant churn required by the industry - ably assisted by websites and we participants in forums.
A recent example is the Fuji XPro 2. I almost bought its predecessor, having been sold its miraculous attributes by several websites (not this one though). The geniuses at Fuji couldn't even incorporate diopter adjustment on this expensive product! I tried one (I wear glasses) and it was unusable. Apparently it was possible to exchange the eyepiece for some Leica unit and....but... I didn't buy one, thank goodness. Now the XPro 2 has - diopter adjustment! Miraculous. Actually these guys are playing the punters for dimwits. I guess if you can afford to buy anything and everything that comes on the market (as some participants here seem to do regularly) it really doesn't matter; these are just consumer slightly-durables, like socks.
I've withdrawn from participation in this stupid game although I'm still an amused observer.