I don’t see that a lot, because I’m only subscribed to the most populated communities whenever there are multiple.
Anyway, I don’t know much about bot programming, but it seems possible to make the bot check the community for similar URLs in the last couple of days so it could skip the ones already posted.