I don't consider any subjects you mentioned off limits, but I do try to approach everything from a different angle, exposure, etc. For example, my main subjects are wildlife, but instead of standing way back with a 3,000,000mm lens, I work to get close and use nothing longer than 200mm.
I don't shoot homeless, but do see something special in some of the older characters I encounter.
A good arguement for shooting homelss people, is that society in general tends to ignore them and what they indicate about our "affluent" society. People don't look them in the eye or stop to talk to them. To me this is more morally deficient than taking their picture and maybe exposing their plight to the masses.
There's always a number of ways to look at things--and that's what photography is about.