niplav @niplav@schelling.pt

“danger lives at my hairtips” she pouted and downed the day-after pill with two finger widths of tequila

In the context of making inconsistent preferences consistent, these are fairly strong results.

Not sure about their approximation behavior, but I think this makes becoming a coherent agent very difficult.

• PCS and all c∈C have minimum graph edit distance of i. (Proof sketch: There is a graph for which all acyclic tournaments with the same (minimal) graph-edit distance don't contain a specific subgraph). Graph in picture, the minimal edit distance is 3, the non-preserved consistent subgraph is a2→a4. This extends to arbitrarily big consistent subgraphs (replace all edges with acyclic tournaments with n nodes).

Then you have the following impossibilities:

• PCS and f has polynomial runtime. (Proof sketch: finding an s of size k is in NP. Finding all of them is in NP as well.)
• PCS and C has polynomial size. (Proof sketch: You can construct a graph with exponentially many acyclic tournaments as subgraphs).

Let f be a function that takes as input an inconsistent preference i and produces a set C of consistent preferences. Let S be the set of subgraphs of i for which it holds that for every s∈S: s is an acyclic subgraph, s is a subgraph of i, and s is maximal in i for those properties.

S shall contain all s that fulfill the above conditions.

Let PCS be the property that for every s∈S, s is a subgraph of at least one c∈C (every consistent subpreference appears at least once in a consistent version)

Pearl is of course the GOAT, but the book doesn't look like the thing I'm looking for (not quite practical enough)

I kind of want to learn Bayesian Statistics real bad

But like the practical kind you can actually use to calculate likelihood ratios of experiments

Any reccs for textbooks? Ideally with code[1] *and* exercises

[1]: Best of Julia, 2nd best if Python, I don't wanna learn R :-/

Queering Hume's Guillotine: Ontological Crises, von Neumann-Morgenstern-inconsistent Preferences and Two Impossibility Theorems

Functional Decision Theory has a WP page

I should do more forecasting again

the answer to the Needham question is that China glorified wordcels too much

The last points especially might be ameliorated by literally just appending "and don't optimize too hard" and "let yourself be shut down by a human" to the prompt?

Man I feel confused, but assuming that language models aren't infested with inner optimizers now I'm more hopeful?

Or am I missing something crucial here…

• Last point especially crucial in situations where such an agent starts recursively improving itself (e.g. training new models)

Thinking out loud what still doesn't work with giving AutoGPT agents instructions like "do X but respect human preferences while doing so".

• Inner optimizers are still a problem if they exist in the GPT models
• Do LLM agents have sufficient goal stability? I.e. when delegating & delegating further does the original goal get perturbed or even lost?
• Limited to the models' understanding of "human values"
• Doesn't solve ambitious value learning, model might generalise badly once in new domains

land-value taxes on arbitrary graphs

Hm maybe Hodge decomposition can be used to define the goal-directedness of a system?

If your system is in loops, it's not accomplishing much, but the potential part also needs to be high (rocks have no loops but also no direction)