You should check out Xisuma’s YT, he did a great 3-part series on how he runs Hermitcraft (popular Minecraft SMP)
In this he mentioned for example letting the community come up with minigames and events, that they can enjoy with the rest of the community. It’s also definitely important to talk with the members about what could be done better.
I do recommend modding though, it helpes a lot with Minecraft’s lack of content.