Actually, I feel it's a huge concern, if you use custom color profiles. Anytime you take a image from a custom profile (J H Holmes for example) or just from Prophoto or Adobe sRGB 1998 down to sRGB you lose a ton of color, range etc. You basically have to rework the entire image and even then you won't get it to be the same.
I agree that most people viewing don't have color managed monitors etc. but if their browser will support a color profile besides sRGB, then that is a big deal for me. Most Art Directors/designers I have worked with over the past few years understand that they need to look at images I have sent them in a icc color aware software, not a default windows photo browser. But with a web gallery, I realize it's a crap shoot and you can't control what it being used on the other end.
I just didn't want to rework the images, but see the points made here and agree, that it's best for now to stay with sRGB as long as I am working the web from windows.
Paul C