You’re being really thoughtful and this is a good discussion.
I read through all the other comments and I’m disagreeing with a few other viewpoints from others pretending like aggressive downvoting/brigading is an individual’s problem.
Perhaps a feature request to Lemmy could be an option. Rather than a binary choice.
Votes as is, upvotes visible/downvotes not visible but measured, up/down not visible but measured, no downvotes, no votes at all.
Discussion instances could work around what works best for sorting and discussion, general could work what’s best for them etc.
Forums didn’t have upvotes for years and it worked just fine.