No, this doesn’t remove the need to stay up to date.
However, it works on my server and was very easy to setup: a few ufw rules so that port 22 is blocked everywhere, allowed only on the VPN IP range and my local network range. Nmapping from outside does not show port 22 accessible, and indeed you can’t SSH to it without the VPN.
Security is quite tough to get right eh? I tried my best to harden everything opening ports on my server, having a fail2ban, VPN for maintenance, webserver to expose some personal services…