That appreciation you have seen on your principle residence isn’t really accessible though. You have to live somewhere. If you were hit with capital gains tax on your primary residence, it would impart a whole lot of financial friction on ever being able to move.
Let’s say you need to re-locate, or upsize/downsize; the house you are moving too also has appreciated, but you have to buy that house at the new, appreciated price, and ALSO pay the capital gains on the 400k. That financial friction on being able to move is bad for everyone. It keeps people in inappropriate houses, or commuting long distances, and doesn’t do anything to improve housing affordability.
It does effectively become a transfer of wealth (from lack of taxes) from young to old, as they downsize, and reap the financial windfall. However, that could be clawed back with estate taxes. If you penalize downsizing, you create an even bigger incentive to stay in oversized housing, as you mention. If anything, not being allergic to property tax increases is probably the only thing that would encourage people to rightsize.