On top of that, Voat got their main population-spike around the time reddit was cracking down on racist and extremist subreddits, so those are the type of users who shaped the culture of Voat. Lemmy, on the other hand, is getting their population spike from enthusiast users, I.E. the 10% of people most responsible for voting, commenting, posting, and just in general contributing to the site. Therefore, those are the people shaping the growing culture of Lemmy, doing so in a mostly positive way.
There is a phenomenon known as the "Eternal September". In the earliest internet, the vast majority of internet users were college student. Therefore, every September when freshmen started school, the online communities would get a massive influx of new users; These new users were often poorly behaved or disruptive to the culture of the communities, but over time they would acclimate to the local culture and become just more normal users, and things would settle back to normal. This was known as the "September Effect".
And then one year the internet started gaining small mainstream attention, and suddenly these chatrooms were being constantly flooded with new, ill-behaved users all the time; And because this "September" never ended, the culture of these communities ended up being washed away by the new people, and irreversibly changed forever; hence the "Eternal September".
The moral of the story, too many new people to a community too fast can overrun the existing cultural dynamic, and so either you need to be restrained in how quickly you let new people join so they can gradually assimilate, or you need the people joining to already share the same culture you desire.