I definitely do find this a little disappointing as I think the Lemmy community is too small at the moment to create unnecessary divides and schisms. Success in my mind is predicated on many communities from Reddit coming to servers and forming a common denominator community that achieves critical mass.
It’s clear to me that some of the communities on the 2 you are defederating from you instance have become more popular and are already the defacto “place to be” for certain subreddits.
All that said, I’m happy that my main server (infosec.pub) has not unfederated from those 2 instances so I am able to still participate on those 2 servers AND interact here on my “main” account. This lets me get the best of both worlds. It’s very exciting to see the Lemmy model working in that regard!