Jem,
I have both systems, the original Olympus E-M1 and the Fuji X-T1/T2 and like them both quite a bit. The lenses for both systems are also, on the whole excellent.
While I have a few Oly primes, I personally use the fast Panny zooms, the 12-35 and 35-100 f/2.8 zooms for the vast majority of my work with the E-M1. I very much like the build quality, responsiveness and overall performance of the E-M1. The nice thing about it is you can take it, and the two zooms in a small camera bag (e.g. the Think Tank Mirrorless Mover 20) and have a very capable system for travel, days out, etc.
I also have been shooting with the Fuji X system since late 2012; originally the X-Pro1, and subsequently the X-T1 extensively until September when I received the X-T2.
The cameras in both systems are about the same size, but the lenses in the Fuji X-system are slightly larger than the Olympus lenses due to the larger sensor size, but they are still compact enough to be able take easily for a day out shooting or for travel. Of course, they are fully capable for use in a professional context.
On the whole, I personally prefer the Fuji X system. The images overall have a certain "magic" in them that is hard to put your finger on, but that I don't see in the Oly files. The quality of the files of the Oly are excellent, but for me personally, they don't have the overall qualities and color I get from the Fuji X system. I also really prefer the Fuji files for their rendering of skin tones, which I find to be gorgeous, and in comparing the 16 megapixel sensors, the Fuji has better noise performance at high ISO and considerably better auto white balance.
Regarding Fuji RAW (RAF) conversion, I primarily use LR along with Iridient Developer for RAW demosaicing. I use it as a plug-in and and "out and back" only takes about 30 seconds, and have virtually no problems rendering detail in grass, leaves, foliage or other types of high frequency detail. The detail that Iridient can pull out of a Fuji RAF is really quite remarkable. It also has better and more control over recovering shadow and highlight detail than LR does.
Regarding IBIS vs. OIS, personally I prefer OIS because it works better than for larger lens movements than IBIS, but Oly has really done a remarkable job with IBIS on the E-M1 MkII, achieving, from what I've heard from reputable sources, about 6 stops of image stabilization. As I have extensive experience shooting motorsports with big glass handheld, image stabilization has not been a problem for me per se, and in low light, I use a tripod. I even turn the IS off in my Canon pro lenses as it gets in the way for motorsports.
The X-T2, on the whole, and others have pointed out, is quite a step up in performance over the X-T1 on a number of levels, and when you put the vertical power booster grip on it, it turns into quite a veritable performance beast, basically a mini-Canon 1Dx MkII. It's the first mirrorless I've shot with that will allow me to finally sell my Canon 1D-body for my pro motorsports work (my back can't take that big beast anymore...)
Some racing images from the X-T2.
I think the suggestion here to rent an X-T2 and couple of lenses for a week or two and gather your own assessment is an excellent one. They are both excellent systems.