Anyone can set up a server indeed, and you have no real way of knowing if you can trust them or not.
The official SimpleX chat website has this nice advice in this regard:
The servers have separate Anonymous credentials for each queue, and do not know which users they belong to. Users can further improve metadata privacy by using Tor to access servers, preventing corellation by IP address.
But IMHO if you need Tor to get the resemblence of metadata privacy, why use SimpleX at all and just use XMPP with Tor with works great?
To me this SimpleX is pure techno-solutionism that tries to solve a hypothetical problem and ends up as basically security LARPing and not solving many real-world threat-models at all.