The political aspect is especially true. The FOSS confusion is often similar to the communism confusion, especially when it comes to small-scale things.
Take the concept of a neighborhood garden that no one is expected to pay money into, for instance. “Wait, so the people here who like gardening don’t expect me to pay or provide labor unless I’m able to? What do you mean i should take only according to my needs? What about Jimothy, he never helps but he takes way more than I do! What do you mean Jimothy contributes as he is able or in other ways? How can i trust everyone to be fair?”
Take the money for goods/services exchange out of the equation and it can really throw people off.