IMHO, it’s a personal preference. One thing is for sure - a phone that is not getting OS updates and can’t run current App Store software isn’t just going to feel old, it will be old. And the SE has the newest silicon, it is likely to be supported the longest.
If you want Face ID and small form factor, that old Mini is a good idea. But you’re likely going to be sacrificing an extra year of software support and you’re definitely going to be getting much worse battery life.