A lot of AI discussion presumes that an AI can just get answers to hard questions at will.
Has anyone asked ChatGPT what's in the nuclear notebook?
@WomanCorn ChatGPT's structure doesn't really have an analog to "will". It's predicting tokens using a massive library of examples. It doesn't act in any sense.
However, it's also just a lower bound on what AIs of the future will be capable of
@WomanCorn fair. ChatGPT will compress and translate information but it won't invent things, and a scaled up ChatGPT won't either. But a future breakthrough in AI might produce an AI that can form and test novel hypothesiss, and that might be able to produce ingishts and take actions beyond human understanding.
I'm not sure how quickly an AI would go from novel insights about reality to producing nanotechnology (or similar scifi). But that's necessarily a question beyond cureent understanding