Who have logs of how users vote?

Besides it’s a very important question for our privacy, I suspect some of my comments getting downvoted at set periods of time from other accounts. I don’t understand how the fediverse work and what I’m asking is who has vote logs? Who I can ask, if a bot, like on reddit, got dedicated to downvote every post from one user....