Well, am saying that, artistic rendering aside, image quality is a function of the selection of pixel color. The only difference between the output of a $300 sixteen megapixel digital camera and a sixteen megapixel flagship DSLR is the instrument's selection of color for each pixel. Obviously a large sensor with considerable dynamic range will produce a better image than a small sensor with a limited dynamic range, but ultimately, it is the instrument's selection of a color for each pixel of the output file that defines quality.
In one sense you are correct. If two images have the same number of pixels, the only difference between them will be the quality of the pixels. The larger, more expensive camera will usually produce better pixels for a number of reasons:
Less noise. You can think of noise as a random difference between the recorded brightness of each pixel and its "correct" value. The smaller sensor will generally have more noise, so the pixels will deviate more from their correct values.
Better dynamic range. Limited dynamic range can make it impossible to accurately capture both the brightest and darkest areas of an image with a single exposure. If you make the exposure long enough to capture the detail in the shadows, the highlights will be recorded as pure white. If you make the exposure short enough to preserve the highlights, the detail in the shadows will be lost amongst the noise. A larger dynamic range reduces this problem, although all current sensors (and the human eye) will have difficulty with some scenes.
A reduction in various other artifacts (blooming, etc.) which I don't know enough about to comment on.
Larger pixels allow you to use a narrower aperture before diffraction starts to blur the image. Some small, high pixel count cameras are limited by diffraction even with the lens wide open.
On the other hand, smaller sensors do have one advantage - more depth of field for a given aperture and field of view. This can also be a disadvantage if you want a shallow depth of field for artistic reasons.