Similarly I lose immersion when I’m going into a corner and I struggle to see the car on my inside. That’s why I want to try VR.
I don’t play with a shifter, just paddles on the wheel (driving F4 in iracing), so I don’t think I’d have any trouble with where the controls are. Maybe the Apple Vision Pro will end up being the best racing headset though and solve your problem.