Passwords will be brute forced if it can be done offline.
Private SSH keys should never leave a machine. If a key gets compromised without you knowing, in worst case you will revoke the access it has once the machine’s lifespan is over. If you copy around one key, it may get compromised on any of the systems, and you will never revoke the access it has.
And you may not want to give all systems the same access everywhere. With one key per machine, you can have more granularity for access.