Okay, now your argument has officially gone off the rails.
To clarify my point, governments don’t have rights, they have powers. The charter grants people rights. The notwithstanding clause gives the province a power to override a charter right. Exercising that power only ever removes people’s rights. And yes, the country can become less free if rights are overridden. Nothing necessarily “balances that out.” Losing charter rights is often a very bad thing, and even if it’s necessary in a particular case, everyone should be honest -it’s a loss of rights.