Yeah you’re not wrong, it sucks. People treat it as a “I disagree” button while it was intended for off topic comments and spam.
Then the hive mind takes over and people pile on. I used to call it the reddit hive mind, but I guess it’s not limited to that.
I know it’s frustrating but I don’t think we can change human nature. If you are sure you said nothing “wrong” (like hate speech, off topic comment, etc) then it’s best to shrug and move on.