niplav @niplav@schelling.pt

@Paradox i agree

Lel

I may be overescalating

@Paradox And I didn't think about it as a *risk*, just as an observation of "there can be powerful subagents/mesaoptimizers who can be specialized and not really expand in their domain very much"

Also, I think a lot of mental illnesses are subagents with specialized values overpowering the other subagents

(e.g. OCD as a dictatorship by a specific drive, or schizoid personality disorder as too-strong-self-protection drive)

@Paradox Oh, I was being hyperbolic

but also Ramanujan didn't seem very good at using that powerful optimizer to reach his other life goals

So there was no "infectiousness of optimization" of this subagent coming up with infinite fractions

I do think that many very smart people have strange fixations/don't really manage to attain goals of some well-balanced life

Which could be modeled as the chess-subagent overpowering the other drives

@Paradox Feeler: a random (observational) text (such as "I just saw a cat that looks exactly like you")

She responded with a counter-question, which is a good sign, but I don't expect much further from the interaction

@Paradox Ramanujan was very good at math

So he had a subprocess in his brain doing advanced math, which might've been a powerful optimizers

Powerful optimizers (or so the story goes) display instrumental convergence, but the powerful search in Ramanujan's brain was happy just continuing to do mathematics and not turn the world into math-goo

@Paradox didn't respond to the initial text, but just now she responded to my feeler

lol

@Paradox do you want me to elaborate?

(god that sounds like I'm an LLM chatbot :-P)

@Paradox (it didn't and that's weird)

@Paradox though: did the mathematical search in Ramanujans brain display instrumental convergence?

@Paradox bearded shoggoth

@Paradox true

Getting better at this though

^ is the most shippable one, here's what I was going for

Do you notice the bug?