Earth's field is very large and very weak compared to small strong household magnets.
Whatever senses we've evolved for navigation could be dealing in a completely different scales so as to be useless.
It might be like trying to use a 5ml teaspoon to measure the volume liquid in a of a bathtub.
or using an alcohol thermometer to measure the temperature of molten iron