What is the chance that the Sun will supernova?
Well, our model of stellar lifecycles says it won't, so the chance of it happening is dominated by the chance that our model is wrong.
How likely is it that our model is wrong?
@WomanCorn we can validate the model with many stars, hence low?
@k4r1m that's what I think too.
I'm looking for an intuition pump on how to reason about things when most of the weight leans on the model being right or wrong, not the specific facts of the matter.
@WomanCorn I am not familiar with the term “Intuition pump” - Did my answer go in the right direction for you? I am assuming yes (if the weights are on the model, then a model with better validation can be better trusted?)
@k4r1m an Intuition Pump is a thought experiment that helps you to understand similar situations.
So, the likelihood of the sun supernovaing is tiny, but if that happens it means I was seriously wrong about how the universe works.
I also think LLMs will not become AGIs. If one does, it means my model of how intelligence works is seriously wrong.
It would be nice if I could discover that in a non-catastropic way.
@WomanCorn thanks!
On the LLM/AGI issue, my take is that regardless of my model of intelligence, I am fairly confident that the models of intelligence of the LLM/AGI people are heavily flawed. And secondly, I am not aware of other instances where complicated things were built accidentally, as it’s claimed here…