It could be argued that other video games are also designed for us to waste time on. It’s just that the method of wasting time is different. In one you make numbers go up, in another you kill enemies (which might just be to make numbers go up: referring to grinding in RPGs) or try to make the car go fast in the right direction.
I personally enjoy idle games, but I understand that others might not like just clicking some buttons that will make the numbers go up faster.