In my experience, most of the people who are left cold by landscape photos have never seen (or at least never paid attention to) landscapes of the sort, so it means nothing to them. Why some of us love landscape photos so much is that we've been to beautiful and/or interesting places like that, and the photo reminds us of the experience. Without that experience to fall back on (or, in some rare cases, enough imagination to imagine it), it's just a bunch of colors or shades of grey. As someone said above, "There are two people in every picture: the photographer and the viewer." The past experience of the viewer matters.
I live in the west, at the foot of the Rocky Mountain Front Range. Pikes Peak is just above me. To the east of me are the Colorado and Kansas prairies. I frequently shoot landscapes on the prairies -- landscapes that almost always include artifacts created by the hand of man. I also shoot landscapes in
the mountains, but, again, they almost always include the hand of man. I'm no city boy. To see landscapes, all I have to do is step outside. But I also agree with what HCB said. I learned a lot about wet photographic technique from Ansel Adams's prints and books, but I never saw, and still don't see Ansel's subject matter as important stuff.
Some landscape painting
is very moving, but that's partly because of what it leaves out. What the pictorialists never understood is that painting is outside time and place. It's in the mind of the painter. If a painting is really good, it gives you a transcendental flash because of what the painter managed to convey, not only through good composition but through simplification.
Photography, on the other hand, is about
time and place, and can't subdue detail without pretending to be something it isn't. A photographic landscape can be very beautiful, but it's still, in Walker Evan's and HCB's terms, a "so what?" It may be true that someone who's lived all his life in Manhattan can't truly appreciate landscape because he's never seen a real landscape, but even though he's never seen a farmhouse he'll respond immediately to a picture of a deserted farmhouse with abandoned toys on the floor. On the other hand, someone who's lived all his life on the prairies can respond very strongly to a street photograph from Manhattan that conveys something important about human life -- even if he's never been to Manhattan.
I think the difference lies in the degree of significance. That's why street photography, going all the way back to Atget, survives and continues to stimulate, while early landscape photography largely has been forgotten. It's true that Ansel hangs on, but mainly because of the technical perfection of his prints.