I think the taste of raw tomatoes just bothers me. Keep them away from my sandwich, and I never tried them with sugar because I didn’t know that was a thing that existed until right now. Sauces lose points the chunkier they get.
Things like ketchup, marinara, not-chunky salsa and so on don’t taste like raw tomatoes so maybe that’s all it is. Interestingly, I do like sun-dried tomatoes in a salad or maybe on a pizza.