You are very much in the minority as someone who has studied Latin. Very few non-Catholic high schools even offer it, much less make it mandatory.
And sure, Catholic mass was held in Latin back in the day. Personally, I suspect that’s a reason it’s associated with rituals and magic. What is a priest doing, if not invoking mystical powers beyond the understanding of man? What language would someone use to invoke the powers of Satan?