I somewhat disagree with you overall, because almost all magic have countermeasures. Invisibility for example is good to bypass humanoid, but you still need to be stealthy, because creatures with good ear or smell don’t care about seeing you. And the time constraints is the biggest limit.
Sure, you can use spellslots, but you only have so many of them, and if you take a day to recover them, evil has its plans going forth too. Which city will you sacrifice to Sauron while you sleep to recover your spellslots?
Usually spells are far less potent than people think they are. But it’s hard to be accustomed to them to easily prepare or think about all the countermeasures. It’s not hard usually though. A dog will do a lot against any kind of infiltration for example. And there are far worse creatures than that.
Another note is that Gandalf is only a tier3 sorcerer. Between lvl11 and 16. And you can see how the character is treated in the story: he has the eye of Sauron on him, unlike the hobbit.
Ironically Larian is doing far better for countermeasures and time pressure than most dm I feel.
For mage hand, it’s a programming constraint I feel. Summons don’t have inventory you can access, so it’s easier to prevent you from stealing with the hand than having players lose items because they pick them with the hand. It’s a limitation of the game engine.