Zorki5 and SZRitter both have a point: the size and weight trade-offs depend on things like how much "reach" you want, and how many lenses one wishes to carry.
At one extreme, if carrying just a normal or wide prime, or a small, slowish 3x kit zoom, the size difference is not much, and the balance shifts towards the advantages of a larger sensor. At the other extreme are people like me who often want a lot of telephoto reach - I carry a 70-300mm with my MFT kit, for that "up to 600mm field of view", and so of course need at least one other lens for the more normal shots, and also like to have a macro lens with me. That FOV range definitely benefits from a smaller format, or to be pedantic, from smaller photosites. Even when I carry just one lens, it will be one with more reach than a 3x kit zoom; say the Olympus 12-50, and that is distinctly more compact than any 4X zoom I know of in larger formats.
I can even see photographers like me moving towards 1" format as the technology and lens systems develop.