Not who you were responding to but, my company does this in AWS. To be fair, the entire platform is running in EKS so it’s not much more difficult than updating the CI build pipelines to build multi-arch containers, adding additional nodepools, and scaling down the amd64 ones. This was tedious but not difficult to do. I keep a small set of amd64 nodes for off the shelf software that doesn’t support arm… I think the only thing left on those now is newrelic agents. Once we move off of them the x86_64 nodes can be killed entirely.
This ended up saving us tens of thousands of dollars per month. The next step is to move the bulk of workloads to spot instances. I’ll be preferring arm but if there is only capacity for x86_64, I’ll have that option because of the multi-arch containers. This is going to save even more money and force developers to build applications more tolerant of node failure in the process.