ChatGPT Is Not a Blurry JPEG of the Web. It's a Simulacrum. https://blog.domenic.me/chatgpt-simulacrum/
In which I try to provide a more accurate analogy for large language models, by summarizing @repligate's simulators thesis.
The question of how intelligent simulacra like ChatGPT can become is not at all settled, and we shouldn't expect there to be fundamental limits. (But there may be practical ones.)
But the best part of having the simulators analogy handy, is that it prevents you from getting stuck in the contentless framing wherein LLMs are "just" text predictors.