Honestly, as far as I’ve seen most spells aren’t an issue. Only sorcerer quickened spell really makes it an issue, but that’s mainly an issue with quickened spell rather than anything else.
I also believe Jeremy Crawford or someone has mentioned that balance wasn’t the concern when the role was put in place. I’m not able to look for the source right now, but I think Treantmonk had it in a video about this rule.