I don’t think age needs to be the limiting factor. I’ve met plenty of 70+ year olds who are mentally capable of performing any job. My grandfather is in his 80’s and he’s a kick ass doctor.
I strongly feel that it needs to be test and check up based. Something impartial treated with an air of dignity so that people are raised respecting that it’s perfectly alright to not pass it. That should help avoid stigma while ensuring people like that judge are a non-issue if not nearly a non-issue.