The Android source code is available, but unfortunately that doesn't mean that all phones are based solely on that source code. Almost all vendors (including Google) have closed-source additions to it.
There are indeed people who agree with you. I do in principle too, but I can't say this is something I think about much, which is probably how much people who even understand the issue feel. And most people don't have a clue the issue exists.
Google could ban F-droid on some phones, but not all. OEMs could overrule Google on such things with their custom Android builds, and even if they didn't, users could create their own ROMs to solve the issue for rooted devices.