Corn Woman 🌽 @WomanCorn@schelling.pt

In version B, we're talking about Inner Alignment failures, where the AI is programmed to maximize human happiness, and the "paperclips" are 10-neuron constructs that count as human to the AI and can only feel happiness.

In version A, we're talking about the Orthogonality Thesis, and the paperclips are actual paperclips*, because the point is that a superintelligent AI might not care about what you care about.

* This also applies to bolts, or Facebook share prices.

The Paperclip Maximizer is a type error.

There are at least three versions in play, depending on what's being talked about.

@empathy2000 is this just because we use the jargon "tacit knowledge" for that category, or do you think there's more discussion missing?

@flats I think the instrumental convergence argument is still pretty good. It does rely somewhat on the idea that the AI will be trained to optimize a single metric.

When reinforcement learning seemed like the winning technique, this was a big risk. Now that LLMs are the most promising technique, it's less clear. <Minimize next token prediction error> doesn't obviously call for conquering the universe.

@flats right. The question is how many of the fundamental arguments were worked out assuming that the goal was to build a CEV sovereign and never rechecked to see if they still apply now that that goal has been abandoned.

@flats If the AI isn't going to acquire godlike power, how many of the issues devolve into the principal-agent problem?

But no one wants to double check 1000 pages of blog posts to see if the conclusion relies on an unstated assumption.

@flats I think the problem is that a lot of their thinking on AI has a presumed final step <then we give it control over everything and it instantiates heaven on earth> and a lot of the threats hinge on the implicit assumption that you will give the AI control over everything.

So, an AI might conceal its real goals... Is that an issue if it is only going to get enough power to run the factory?

Maybe, maybe not. But we have to check every argument.

Waiting for service marks:

Networkless: Doesn't communicate with the seller or nay service provider.

Updateless: Simple enough that they won't ever be shipping a software update.

Microprocessor-free: The ultimate in assurance that it doesn't spy on you.

How many people have died since you decided nonviolent protest was the right option.

If you had just gone to war, the death total would have been smaller.

Alice: LLMs are not thinking. They're just rephrasing things they've already read. Their alleged essays revel that they have no understanding.

Bob: By that logic, most humans are not thinking either.

Alice: Okay.

Does anyone have good examples of Progressive Web Apps that are not based around disconnected use?

(Disconnected use is the acid test in a lot of ways, but it's not the build-around goal.)

@flats it looks like I won't have time to write a real post anytime soon, so I'll point you to this short summary instead:

https://twitter.com/WomanCorn/status/1631696104403107844?s=19

@RevCyberTrucker

What I find amazing is that none of the glass parts of the lamp broke. I'd expect those to break easiest.

Visa:

You've got to solve the problem where I set automatic payment on something, and the bank thinks it's suspicious and asks me if it's legit, and then they payment doesn't happen even if I say yes.

Work this out. Enabling transactions is literally your main business.

I knocked over my lamp and broke it.

Not the bulb. Not the glass cover.

The bit that the bulb screws into that connects it to the wires.

Just amazing.

People ask <what if China makes AI before us?> I retort: <who is us?>

You think the Chinese government is running an AI development program? Why?

If it's Alibaba, how is that different from if it's Google?

<This company is headquartered in my country> is no help.