It really does diminish the power of tailoring your party to specialties.
If the guy I’m talking to starts being a dick why can’t I choose to have my badass character intimidate him? If they’re nice why can’t I use my charismatic character? Should be able to choose who responds to what whenever you like, it would make dialogue with NPCs so much more involved.