Generally a good approach is to try learning the rules of logic. Logic is all about proving things to be true using only facts. It can also be helpful to try some logic puzzles or riddles which can only be solved using hard logic. Note that this won’t automatically make you a better critical thinker, but it will help you exercise that muscle.
Also, it’s helpful to play devil’s advocate. If you hear someone making an argument, try to imagine how you would dispute that argument if you disagreed with it. It doesn’t matter if you actually agree or not, just imagine you did and think about what your counter argument would be. This is what high school debate teams have to do; they are given a topic and a position and have to defend their position.
It always helps to be aware of the facts, or at least of how to find facts. If you see a debate happening where you can’t tell who is right, do your own research on a site like Wikipedia and try to see what the truth is for yourself. Not every argument has a correct answer, but you will at least be able to see where each side is coming from.