And even with Firefox, you need to specifically install the color manager extension.
Don't use Firefox as your only reference, as it does an excellent job of representing color images even when they don't have an embedded profile. The latest version of Chrome correctly represents images with embedded profiles, but those without embedded profiles look as gross as ever. Not sure about IE9, I suspect it's more or less like Chrome. And yes it's very informative to open the same page on side by side browsers.
Some minor additions: you don't need that colour management extension; it just makes it slightly easier to control Firefox colour management features. Without it, you need to manipulate the config settings in the "about:config" screen. But colour management works with or without that extension, and it defaults "on".
Also, by default Firefox colour-manages only images with embedded profiles. If you set (in about:config) gfx.color_management.mode to 1 (it defaults to 2) then it also manages images and graphics with no embedded profiles (assuming them to be sRGB).
Firefox is the only browser (as far as I know) that attempts to colour-manage graphics and images without embedded profiles.
Chrome colour-manages only sRGB images. IE9 ignores monitor profiles, which makes it useless on wide-gamut monitors. (I don't know about IE10 - I haven't checked yet on Windows 8, will do so when I get a chance.) Safari is like Firefox, but only for images with embedded profiles.
Update: I've not checked IE10, but according to several blog sites, those that have tested it say it's the same as IE9. That is, it does half the job of colour management, and ignores the monitor profile. Instead of the actual monitor profile, it always uses the sRGB profile. In other words: all colours in all images are incorrect unless the monitor happens to have exactly the same gamut as sRGB. Not good for any monitor, a disaster for wide gamut monitors.
(Edit: I mis-typed IE8 for IE9 first time)