After much reading, gathering of advice, and shooting with both systems, my take comes down to:
If you are doing most of your shooting on-the-run (event, travel, street, etc.), you may like the E-M1 better. If you do most of your shooting more slowly (studio, portrait, landscape, etc.), you may like the X-T1 better.
Why?
The systems have more in common than not. They share high quality construction (except for the X-T1's doors), ample lens selection, high quality optics (if you avoid the lower-end Olympus M.Zuiko lenses), large and excellent EVF, convenient size, significant updates to firmware, and excellent image quality. Neither have above average video capabilities nor dSLR-competitive auto-focus tracking (yet).
Some things are simply personal preference. Do you like more marked control dials? A deeper grip? Are raised control buttons important to you?
The differentiators, in my mind, are:
Fuji X-T1 advantages
Slightly better image quality if you use the "right" RAW converter (but the situation is steadily improving)
Slightly lower high ISO noise
No shutter shock to manage around
Olympus E-M1 advantages
5-axis IBIS - all lenses are thus stabilized
Somewhat more consistent interface
Lenses are slightly smaller and lighter
Off-camera optical TLL flash control is possible
I'm taking both to a friend's wedding in a couple of weeks and that will drive my own personal decision to one or the other.