First, to clarify, my question is about how small will the smallest succesful interchangeable formats be, not any bizarre claim that all formats chosen by all photographers will shrink! I have no doubt that formats up to at least 36x24mm have many years of success ahead of them amongst sufficiently demanding photographers who are willing to pay several times more for a camera and lenses than most system camera buyers do.
Petrus:
On those diffraction limit numbers, f/4.5 is a completely viable value, so those numbers show that even 4/3" format can comfortably handle 30MP, which is far, far more than the great majority of mainstream system camera buyers care about. At the suggested limit of f/2, even Nikon's 1" format could go to about 80MP, way beyond mainstram needs. For most camera buyers, the current maximum of 24MP (NEX7) is already quite excessive, and for that, the limit is about f/5 in 4/3" and f/4 in 1" format; completely doable.
As to the talk of "range of f-stops", The large DOF options are essentially the same for any format (just with the large format having to use a higher f-stop, and so either a longer exposure time or a higher ISO speed), so that is just another way of saying that a smaller format is more limited in its shallow DOF options. The persistent popularity of f/3.5-5.6 zooms, and the fact that this same f-stop range has remained the mainstream favorite through the whole progression from 35mm film format down through APS-C to 4/3” to 1", despite the shrinking options for shallow DOF, indicates that the mainstream market for system cameras is far, far less affected by the craving for shallow DOF than are many participants in photographic forums. Instead, the market keeps voting for using technological advances mostly to allow smaller and lighter kits, with lens speed apparently valued mostly for what that nickname implies: an increase in the range of usable shutter speeds, not an increase in the blurriness of backgrounds.
This persistent preference for what more demanding photographic enthusiasts call "slow lenses" is probably due to the fact that any increase in shallow DOF options or low light performance comes through larger, heavier, and probably more expensive lenses, due to requiring larger effective aperture diameters and thus larger front elements (for example, through having the same minimum f-stop at a longer focal length as needed to get the same FOV on a larger sensor, since aperture diameter is roughly focal length divided by aperture ratio). A larger sensor with lenses of similar size and similarly sized front elements gains nothing in either shallow DOF options or improved low light performance, and the mainstream market seems to take account of that size/weight/cost/performance trade-off better than many forum contributors.
The bottom line is that as far as I can see, a very substantial proportion of mainsteam system camera buyers get all the resolution they need from a sensor of 10 to 20 MP, and seem satisfied with the "selective focus" options given by standard consumer level f/3.5-5.6 zooms along with a few faster options, and for them formats like 4/3" or 1" or maybe even a bit smaller are likely to offer the most appealing balance of size and weight against performance. On the other hand, there are valid arguments for the desirability of the larger formats, from APS-C to 35mm to MF, but probabably only relevant for smaller, more demanding sectors of the market, not for the mainstream of system camera buyers.