This isn’t true, provided that their dataset is large enough. The models are stochastic, and with a large enough number of parameters and a large enough training set, can generate truly unique content. For example, I strongly doubt you’d be able to find anything remotely resembling the following anywhere, ever (look up what the movie is about, and watch it, to understand the absurdity of my request), and yet it was generated by ChatGPT:
If you read interviews from the development of these models, you’ll see the creators saying what can be clear from the above link: With a large enough training set, these models start to learn something about the organization of language itself, and how to generate novel content.
The model architecture that these things are based on tries to replicate how our brains work, and the process by which they learn language isn’t unlike how we learn language.