It's not the greater number of pixels per se - it's the greater quantity of information. You can subdivide a fixed sensor size into more and more, smaller and smaller pixels, but at some point when the image is oversampled, there is no further gain in information. Maintaining the pixel size and lens performance while increasing the sensor area is still the best route to increased information (detail).y
Downsizing of pixels and upsizing of sensors have something in common: if done right, each can produce measurable improvements in image quality, but taken too far, each hits the law of dimishing returns, and only spec. fetishists really benefit from going further.
With pixel downsizing, one limit is the increase in read noise from having more smaller pixels on the same-sized sensor; another is lens resolution. Note however that some photographers will have reason to push sensors to the resolution limits at center of field of the best lenses at their optimal apertures, and we are not there yet.
With sensor upsizing, one limit is the ever smaller increments in IQ versus ever larger increments in size, weight, cost, and the effort and technique needed to realize those IQ gains. Another is the issue of shutter speed, as soon as adequate DOF is an issue. Because once the larger format and so longer lens needs equal DOF, the aperture ratio must go up, and to gather more "information" as Ray puts it, meaning gathering more light means that ISO speed cannot be pushed up enough to keep the shutter speed equal: the aperture is admitting photons at the same rate, so to feed the larger sensor with more photons needs longer exposures. So once high resolution needs imagery impose tight limits on both motion blur and out of focus blur, there is a hard limit on how much light ("information") the sensor can gather, no matter how large the sensor.
As a rough guideline: in situations where DOF and lighting constraints force the use of a higher than base-ISO senstivity in a particular format, a larger format won't help much.