Every system will have compromises, but I can live with the ones on this camera. Here are the things that I would like to see improved:
1) The EVF/LCD setting is time consuming to find. I want to be able to change those settings at the touch of a button. =
I assume you're looking at an XE2. Fuji did away with the view mode button that allowed immediate switching between EVF and LCD, and move the Q Menu button to the position. As I use the View Mode button all the time, I would miss that if I upgraded.
To the OP:
The two zooms are very good; I own both and use the long zoom regularly and the short zoom on occasion. I find myself shooting mostly with the 14 and the 35 mounted to an XE1 and an X Pro 1 respectively, with another XE1 for the 55-200 when I want to shoot with three cameras. When Fuji releases the 56/1.2 I'll pre-order as early as possible, which will give me my own perfect 3-camera, 3-lens shooting kit.
I find both zooms more difficult to use than my Canon zooms. Partly that's because they zoom in the "wrong" direction, but also because the zoom and focus rings feel the same and are right next to each other. So I'm constantly turning the focus ring when I want to zoom, and even when I get the right ring, I turn it in the wrong direction

This is more of an issue with the short zoom, and it's probably just me being an idiot.
I've considered selling the 18-55, but I'm keeping it for now as I'm pretty well set on selling the Canon gear instead. Once I do that, I feel like I'll get more use out of the Fuji zoom for landscapes and general tripod work, especially with filters. Until then, it mostly stays in the locker.
The Fuji 14mm is one of the star reasons to buy into the Fuji system, IMHO. I have stopped carrying my Canon 16-35 -- images from the Fuji 14 are so significantly better that it's just not worth even attaching the Canon zoom to a camera (and on a 1D Mark IV, the widest field of view is almost exactly the same as the 14mm on the Fuji.) I have not tried the Zeiss and have no reason to seek one out to do so.
If you are going to buy the zooms, there is no really obvious reason to get the X Pro 1. The OVF isn't particularly usable with the 18-55, and doesn't work at all with the 55-200 (though with the short zoom it's kind of cool to watch the OVF frame lines zoom with the lens -- until you try to shoot using those frame lines, anyway.) The XE2 has a number of significant upgrades that particularly affect the long zoom -- better autofocus from the phase detect pixels on the sensor, for example, and a better EVF refresh rate. These are things that drive me batty with the 55-200 on the XE1 and might cause me to get an XE2 sooner rather than later.
Finally, don't worry if it takes awhile to sort out these cameras. It took me several months to figure out all their little quirks, but the image quality and the sheer joyful usability still put a smile on my face.
Here's a shot from our Lighting of the Quad ceremony earlier this week. ISO 6400, 35mm f/1.4 wide open, handheld at 1/40 sec using the X Pro 1 and the EVF. The camera nailed the focus instantly. The only light on the students is being reflected from the building in front of them.