Paper surface is an artistic choice which ideally should serve the needs of the image. For what it's worth, I think black & white images are strongly dependent on a deep black d-max for impact. This limits you to Epson's photo black ink and compatible papers. I'm fond of Epson's premium luster paper, which yields a good dark d-max and has a slightly pebbled surface texture that works well at least with my photos. Premium glossy is a bit too smooth for my liking, with more evident bronzing. Crane's silver rag does sound intriguing.
For color, I've always preferred the lush surface of cotton rag fine art papers with matte black ink. The glossier papers with photo black ink do provide a bit more snap from the deeper black, but the difference is pretty subtle, and to my eye they lack the character of rag papers. Hannemuhle's photo rag is very nice, though I've gotten better d-max and superb prints overall on Epson ultrasmooth fine art paper. Finally, I really like Somerset velvet paper for images like snowy landscapes; the pronounced texture of the paper works beautifully to add apparent depth to snow or sky.