I would switch to certificate based SSH authentication.
All the server keys gets signed by your CA, all clients also gets signed by your CA. Everyone implicitly trust eachother though the CA and it’s as safe as regular SSH keys.
You can also sign short lived client keys if you want to make revocations easier, the servers don’t care because now all it cares is that it’s a valid cert issues by the CA, which can be done entirely offline!
HashiCorp Vault can also help managing the above, but it’s also pretty easy to do manually.