Early singularitarians had a three step plan:
1. Create an AI God2. Give it power over everything3. Tell it to create Heaven on Earth.
A lot of AI alignment come out of realizing this plan doesn't work.
A lot of the flaws of AI alignment come out of the fact that the discussion was started by people who thought this plan was a good idea in the first place.
a Schelling point for those who seek one
A lot of the flaws of AI alignment come out of the fact that the discussion was started by people who thought this plan was a good idea in the first place.