To me, the biggest problem with Matrix is that Synapse and Dendrite are both really heavy. I use an alternative server called Matrix Conduit that’s more like an xmpp server in how light it is. Only problem then is that Conduit doesn’t have that many resources so it’s always a few steps back from Synapse or Dendrite.