Presumably, you need to live somewhere. If it's an apartment or other rental, that's an extra cost vs. the house plan.

That said, I don't necessarily disagree with you on the economics of it. I sort of see it like buying a diamond ring: it's something you do because you decide you want it, and people who do it thinking it's going to be an investment are probably suckers.