That sounds like a problem that you might have if you live in a war zone or something. But anyone else who has some downtime can easily jump into a VR game and have a blast.
I guess another situation that would make VR hard to use is if you have dozens of small creatures running around near your feet. If you’re raising an army of cats or ferrets, something like that.