Visibility. When you back in you have full awareness of your surroundings.
When you back out there’s a gap of time between getting in your car and backing out (opening the door, starting the car, seatbelt, adjusting radio, etc…). If you have cars parked on either side of you, you won’t be able to see the cars driving past you.
Then there’s efficiency. If you get a call while you’re in wherever and have to go somewhere quickly, it’s faster to have your car pointed in the right direction.