I think the point is moreso that the party you support typically is indifferent about minorities/LGBT/immigrants/poor people, etc.
This seems antithetical to the morality we are taught as children (ie: the Golden Rule) which is why people question how you generally survive in that type of relationship when both people seem to have blinders on regarding empathy for others.