Whatever the issue is, I think you should clean it from outside and towards inside, instead of going outwards from inside.
This usually requires a sewer truck. Or you can put a hose up the Y clean out. That can also be done with a power washer snake attachment, but at that point it’s easier to call the sewer truck, because these consumer gadgets aren’t really that great. I’ve tried everything and in the end I had to call the truck anyway. You’re already hundreds of dollars into gadgets and chemicals. Stop it. Cut your losses and call a professional next time.