Well, the two main drawbacks for the Olympus would be its so-so optical performance (the 20mm is a good bit better), the fact that it's about a stop and a half dimmer than the 20mm, and that it focuses more slowly. That said, it's not as if it's a bad lens. Just that the 20mm is better and the fov difference is pretty minimal.
I have not used the Panasonic 45mm macro (actually, very few have...) but given that Leica slapped their name on it, it's bound to be excellent... aside from that, the Olympus 50mm f2 is known to be superb (but won't AF on a Panny camera) and there are a wide range of other 40-50mm primes that are easy to adapt to m4/3, though they will be manual focus. There are some exceptional optics available at bargain prices, if you're willing to manually focus...