What rule says that every distro needs its own package and manager to install any package?
There isn’t, it just happens that usually when you have ideas for a new distro to the point where you don’t want to fork from an existing one, you’re also typically not happy with how the packages are handled/built/versioned or whatever, so you end up making your own packages. Maybe you have an idea to make it different, and before you know it, you end up with a brand new package manager as well. Sometimes you want it to be simpler, maybe you’re trying to avoid Debian’s dependency hell and you end up creating pacman. Maybe you’re targetting routers with 8MB of flash and you come up with opkg to keep it absolutely tiny.
Then there’s things like Nix/NixOS where they’re like, what if we do everything completely differently?
and maybe even a specific programming language
This one’s a bit weird. Historically, that was left to the distributions to provide you with libraries, but as things keep moving it becomes increasingly complicated. On Debian, you can install a whole bunch of Python/Ruby/NodeJS libraries with apt, but because they’re tied to the OS version, they can quickly become outdated. And really that only works on Debian, or Linux distributions. There’s no such things on Windows and macOS. Supporting so many potential versions of your dependencies can quickly become a nightmare for developers just trying to get an app out. So typically there’s always a way to bring your own dependencies as the official support environment. Distros can still decide to do otherwise if they wish, but at least there’s a known good environment.
So each programming language tend to also have its own package manager so that it’s uniform across the ecosystem, so that when you try to run a NodeJS app, on Windows, you do it about the same way as you would on Linux. These typically install the dependencies in the project’s folder, so they’re fully independent of the system ones which could be too old, too new, patched in incompatible ways. It’s a bit like a sandbox for your project.
Ultimately, it’s organic growth and people preferring some things over the others, or people experimenting with different things and it takes off. That’s the freedom of open-source, you can achieve things in many different ways based on whatever you prefer. If you want to avoid pip and just install python3-* packages for your personal scripts, power to you.