To be clear, I’m not talking about underlying mechanisms, either, but the approach to the task. A human writer, even one bad at writing and not understanding the topic, will approach the writing with a goal and write to that goal and topic. They can even research if they so choose, but even if they are just making things up, there is intent and context there.
An LLM doesn’t have any of that. It literally just generates words that match certain patterns, with no actual purpose or goal. It may have been programmed with a goal in mind, but it doesn’t have one of its own. It can’t reason, it can’t research, it can’t make decisions. I think that is an important distinction that people who are just saying “Who cares? It’s all bad writing anyways” are missing.