You’re right. His teachings (on how the individual should act) are politics, and they are viable as the basis for a political system. It was actually put into practice, too, in monasteries and other communes.
I think the problem stems from cognitive dissonance. The popular political ideology that most closely reflect his neighborly teachings, is anarcho-communism. That is the exact opposite ideology from fascism (i.e., in-group authoritarianism) which is the ideology practiced by most of his adherents.
They are motivated to find ways to convince themselves and others that the teachings aren’t political, so that they don’t have to reconcile the teachings of their in-group identifier/shibboleth with their practices in the real world.
(I’m not saying this is why everyone says it isn’t political, I’m saying that this is the source of the meme that religion-isn’t-political.)