Under postcapitalism, the factory would be commonly-owned. The company that operates the factory would worker-controlled. That being said, there is nothing wrong with the holder of the building even in common ownership setup being compensated. What is unfair is to demand control rights over the firm for this capital and make the workers at the company your employees. Not everyone against capitalism is a communist. There can still be economic incentives for productive activity