For what it's worth, I think it's possible that some manufacturer's ICC printer/paper profiles might be colour-accurate, but make the shadows relatively dark for "esthetic" reasons.
I haven't looked at this scientifically, so I'm not really sure ... but I did encounter a similar problem with an Epson media profile. I'm afraid that I took the "easy" way out, and built my own ICC profile for that media (I used a ColorMunki and Argyle CMS, which I think is a reasonably decent budget-minded approach to building your own ICC profiles).
In my own case (and remember that I took the lazy approach), I could match my profiled monitor quite closely if I was willing to bump the "shadow" slider in Camera Raw or the print proof in Lightroom a bit. I wasn't willing to do that, and I also wasn't willing to spend a ton of time debugging the whole situation, which is why I built my own ICC profile. I got a quite satisfactory delta-E using 480 patches, and the prints are very close to what I see on my calibrated NEC PA302W, so I left it at that.
I'd be interested to know what the real reason for my over-dark shadows was, but never did seriously try to find out. I don't know if my situation is like yours or not, but I thought I'd share my experience, in case its helpful.
Hello John, My challenge was and to some degree still is very similar to your. Only the deep shadows printed too dark (compared to what I had softproofed image) while midtones and highlights were just right. The image has a very wide dynamic range, having been constructed from three explosures; +/-2EV and 0 EV.
I did two things that has helped but not completely resolved this dilemma.
First>> I printed a reference image with Ilford's factory profile for Gold Fibre Silk. I then calibrated my monitor, tweaking the parameters until I closely matched the print viewed in a GTI viewing booth dialled to ~50% brightness, next to the monitor which was displaying the reference image with soft proofing on, again with the Ilford ICC profile. The monitor calibration parameters that I settled on are close to what DigitalDog and Bert suggested, 150 cd/sq.m for intensity, 5500K for white point and 300:1 contrast ratio.
Second>> I decided to make my own profile using XRite i1Pro2 spectro and iProfiler software. I made two profiles; one with UV included and one with UV-cut. I then printed the same reference image with each of the ICC profiles I made and compared to the image printed by factory profile - all in bright daylight indoors - my target viewing environment. The two images printed by the two profiles I made look almost identical. Further, these two closely match the image using the factory profile! Also all three look great in bright daylight and in the GTI viewing booth. So what did I learn from these trials and where do I stand with my original problem?
>I partially solved my problem. I still need to lighten up the shadows in soft-proof more that what I want in the final image, in order to get what I want to see in the final image!
>The UV included or cut makes little difference with this paper.
>Ilford likely used the same hardware and software to produce their profiles.
> I am more convinced that Photoshop or Lightroom are printing with the given profiles fine. I am tempted to tweak the monitor calibration parameters, specifically the contrast ratio and intensity to see if I can bring the initial image softproof closer to what it prints.