Show newer

This incidentally is also why a high level of interpretative charity is so important for cultivating good intentions in a community. Whenever you socially reward/punish weak proxies for good/bad intentions, you invite Goodhart into the reward function.

Show thread

The simplest case for why voluntary lie-detection technology ("Cognitive Veracity Meters") is essential for the longterm morality of earth-originating life is this:

A) Most of our prosocial motivation develops as a consequence of being rewarded for making other people like us. (Indirect reciprocity.)

B) That incentive cannot reach the deepest parts of our decision-making algorithm, unless our intentions are transparent / cheaply verifiable.

C) CGM makes our intentions cheaply verifiable.

@kokogiac what Dennett calls a "deepity".

the trick is to have two interpretations, one that's true and trivial, and another that's false and big-if-true. a motivated reasoner can then read it as a combination of the two interpretations: borrow the truthiness from A and the big-if-true from B.

@niplav wiggle? move around? hm. defending against unanticipated attacks. honte against aji. i don't have slack to defend against the unanticipated, when there's so much anticipated stuff to defend against.

insight fm 8 months ago that i shudn't allow myself to forget

it's harder to spot when features are missing compared to when unique features are present. the semantic difference may be non-obvious, but the neural implementation is indisputably different.

Nvm, it doesn't make sense to base anything on "the utility of being where you're at" or "your current stock of utility". Utility is a relative measure of choice-worthiness / preference between options.

Show thread

In reality, what you should be measuring is the utility of *being in the state* where you've won or lost. The trick is to somehow estimate that based on the measured size of the bet and the experienced utility of being where you're at.

How does the brain do it? And where does it go wrong?

Show thread

What's the utility of winning or losing $50 when you only have $100?

One way to model it is that the marginal utility of money is propto your current wealth, so e.g. the utility is +0.5x (win) vs -0.5x (loss).

But is the marginal utility better measured relative to your pre-bet wealth or your post-bet wealth? Considering that your post-bet wealth is what you actually have to live with, I'd say the latter.

Does neural plasticity depend on the additive increment or proportional impact?

Bing is the most adorable chatbot around, no contest.

*proceeds to happily interact with the batch file*

too many interesting things in this video to process within reasonable time… prob near best YT-vid i've consumed! channel is good too.

youtube.com/watch?v=vwLb3XlPCB

(re critical brain hypothesis, scale-invariance, Ising-model, dynamic correlation & correlation-length, power-laws, phase-transitions, chaos & order (and big relatedness to global workspace-theory)

task-hydrafication, marginal-choice trap, motivational salience-splitting, & recency bias are the 4 greatest bottlenecks to personal effectiveness in my opinion right at this very moment from my perspective as i'm writing this

my nutshell model of hippocampal formation is this:

entorhinal cortex (EC) encodes generalized dimensional structures / types of relations via grid-cells & object-vector-cells, and hippocampus maps specific (sensory, abstract) cues onto those relations, thereby making place-cells, landmark-cells, splitter-cells, etc.

iow, EC is where abstract math is stored when it's deeply intuitive, and hippocampus is the thing that *applies* those mathy relations to practical stuff.

Show thread

[Curie-point (critical temp below which electron-spin correlates across atoms)]:[macro-scale magnetism]::[error-threshold (critical mutation rate/noise below which genetic info net accumulate across gens)]:[macro-scale evolution]

also relevant for the transition from [subliminal⇄conscious (aka self-organised criticality)] & [default-mode network⇄dorsal attention network]. but ppl hv already made the connection btn consciousness & Ising model.

wikiwand.com/en/Error_threshol
wikiwand.com/en/Curie_temperat

@niplav …and/or be subsumed by the everlasting octotopia! 🐙🌌

@niplav infixquine: retroviruses, relative to a [reverse-transcriptase→integrase→DNA-transcription]-interpreter.

i suspect one of the primary benefits i've gained fm maintaining a modular densely-connected idea-network (in RemNote specifically), is that the nodes themselves begin to form a place-cell map, and mks it easier fr me to quickly embed new things i lurn in an abstract-space-efficient manner.

i so goshdarn hope smbody mks a rly good memory-palace-like 3D note-taking environment fr VR/AR soon. it'd be so overpowered fr encoding world-model.

Show thread

why do u want to be able to self-generate allthings u lurn? bc what matters fr ur ability to generate new insights is the *transformations* btn abstract subspaces. and that means u want to integrate as much of ur world-model into a fractally-coherent small-world network of hippocampal place cells as u can. if u lurn things in isolation, cryptically-related abstract models cud be neurally embedded far apart, which means u hv to uproot/refactor if u wish to flexibly swoosh twixt them.

Why do you think people are so quick to sneer upon those who create new jargon with insufficient license? And why do ambitious people try? Why do you think people still write "with respect to", "ran", "because" and "between" as opposed to "wrt", "runned", "bc", and "btn"? It's hard to coordinate on new ways to say stuff, but the marginal bottleneck on rate of improvement here is not Moloch—we first have aggressive semiotic control loops via inferred/heuristic social implications to contend with.

Show thread
Show older
Mastodon

a Schelling point for those who seek one