Hard to put an objective fair price on something when it's being bought to feed an ego. Other than the tax avoidance angle, it's not much different than a designer t-shirt, is it? Why do people spend more for a t-shirt with a logo on it than one without? To show off to friends, to show off to themselves, etc.
But that may be unfair. It could very well be that the buyers see something in some works that others don't. No law against that and no a priori reason to doubt it. Lots of people didn't like rock and roll when it first appeared. And if the tax avoidance is the reason, then those cases should be investigated tax as fraud, no reason to blame popularity of an artist on that.
I would be a bit happier if there were a "royalty" paid to the original artist on subsequent sales of a work of art. We have royalties on music, so why not art? Maybe we should have royalties on 2nd hand book sales. It's probably only due to arbitrary reasons why we have one and not the other. Adams ruins everything has a video about why car dealerships are so bad to deal with (
https://www.youtube.com/watch?v=CeDOQpfaUc8), which strikes me as similar. That retail sector is a closed shop, and there is no good reason for it other than someone wanted to do it that way and we let them.
More generally, is this worth getting upset over? The number of such sales are minuscule and don't really affect that many people, including the few of us reading this thread. It may affect us in the sense that prices are then inflated for museums, who may have legitimate educational or archival reasons for purchasing some artists' works. OTOH, museums who want to clear out their inventory may find price inflation convenient.