It isn’t a difficult science from a learner’s PoV.
It is, however, difficult in a sense of trying to figure out why in the world what happened happened, and most importantly, making it possible to do again.
That’s because not only can you not experiment, you only gather data from observations, but once you share the product of your studies the reality changes in reacton to it.
In same Physics the object of your studies doesn’t simultaniously study you.
Math gets involved to get a result that is somewhat reproducable. But even then since we can’t factor everything we use degrees of probability/certainty.
Theoretically speaking if we managed to fully understand human behaviour then we coult predict the outcome of everything. As you can imagine, we’re nowhere close to being able to do that.
Back to original post, yes, economics is closer to psychology than it is to physics. At least for the fact that we study human behaviour, but on a different scale. So sociology and political science are the closest, then psychology, next all of biological sciences, and chemistry, physics and everything related come last pretty much.
Math doesn’t fit anywhere here, since it’s a tool for measuring reality and not a study of reality itself.