So if we can compress the world as much as we already have, using tools (brains & GNW) for which there we have strong reasons to expect are very limited, that suggests to me that there's a wealth of untapped simplicity beyond the horizon. But it can only be accessed by top-down non-myopic deliberate practice.

Show thread

One reason to expect brains to merely be scratching the surface is the deep inevitability of pleiotropy and build-up of technical debt (increased refactoring-cost) for all hill-climbing algorithms. "Consciousness" (i.e. global neuronal workspace) is the only adaptation on the planet that even marginally addresses it. But it too is powerless against the tide.

Show thread

Are the smartest systems today (brains/AI) near the limit of how much a world-model can be compressed, or are we hitting the limit? Given all the data we observe, how many intrinsic dimensions is it distributed over?

rime boosted

@cosmiccitizen otoh, if—more realistically—my rationalization-strength wrt unendorsed motivations was merely extremely high, i wud focus on trying to understand and gain leverage over them, instead of blindly "compensating" wo understanding.

a balancing strategy like "ok, spend 30 sec thinking abt pros, and 30 sec thinking abt cons" is blind in the sense that it has no model of the enemy, which also means that it fails to generate opportunities to *learn* abt the enemy.

ju say "a 1000-bit exact specification of X," and I ask "relative to which interpreter?"

I've been misled by naive information theory for a long time. there is no representation-of-thing which "has the structure" of the thing-in-itself. information is only ever the delta btn contexts. communication is always a bridge over inferential distance.

there is vars that more or less dereference as intended, but it's Gricean all the way down.

(thoughts prob counterfactually inspired by niplav)

the Nature function herself takes arbitrary coordinates in spacetime, and returns exact distributions of matter within those bounds.

"Nature speaks differential equations" pfff!! differential equations are like gobbledygook to her, and she takes offense at the notion that this is *what she's made of*.

Show thread

> "Since Newton, mankind has come to realize that the laws of physics are always expressed in the language of differential equations."

this is wrong. differential equations is what we must *settle for* when finding the generators of the data is intractable. if u knew Nature herself, u wudn't be restricted to computing her step-by-step w infinitesimal step-sizes, u cud j interpolate btn arbitrary points w no loss in accuracy.

- the highest-avg-IQ academic subjects are mathematics and philosophy *because* they're also *less* financially profitable (thus, ppl go into them bc they're rly intellectually interested in them). the statistics doesn't seem to bear this out, but that's bc there are confounders—the underlying pattern still holds. :p

- more idr

Show thread

- if u concentrate hard on finding ideas related to X, u increase the rate at which u become aware of X-related ideas, but u also decrease the threshold of X-relatedness required for becoming aware of them. thus, if u want to maximize the quality/purity of ur X-related ideas, u may wish to *avoid* looking for them too hard. this is the advantage of serendipity as an explicit search-strategy.

Show thread


- if u pay ppl to do X, u increase the number of ppl who do X, but u also dilute the field bc now ppl do X for monetary incentives PLUS intrinsic incentives, wheareas bfr it was only the latter.

Show thread

in the context of AI alignment, it implies that the most aligned training-regimes are also likely to be really weak (and therefore expensive).

Show thread

the paradox of selection: the purity of your selection-pressure is inversely proportional to its strength.

sorta obvious when phrased that way, but the point is that you can increase your selection-pressure over the true target by additionally selecting for increasingly less precise proxies. this increases the rate of true-positives and decreases false-negatives, but it also increases false-positives (confounders), and sometimes at a higher marginal rate.

tbc, I haven't processed this at all. I just needed to make a salient note to myself bc I'm in a hurry, and don't want to forget my uncertainty abt this. huge-if-true but still v unlikely.

Show thread

reminder to self: resolve whether actually all of so-called "randomness" or "many-worldedness" in physics is a result of "incompressible logic". I'm torn btn finding this lecture completely baffling or utterly mundane. mby it's utterly mundane but its implications are huge? prob neither. noting this down bc I didn't expect myself to become uncertain abt many-worlds-interpretation based on a *wishy-washy lecture on logic*. WHAT THE HECK.

Please consider funding Johannes for alignment research or help make that happen:

I endorse his approach as much as I can endorse anyone. (See e.g. Emrik's comment, and TsviBT's endorsement.)

rime boosted

@rime you were looking into lie detectors and their SOTA, right? Any good resources?

I've become convinced this might be really really important, thanks to you

Show thread

spelled out: IF u can interact w smth, it's real.

sometimes social reality falsely colors unreal things real, and we're better off seeing thru it; but don't forget to also look hard for that which has been made falsely unreal! there are colours there beyond our imagination.

Show thread

Having soulfwly recited the Litany to myself on numerous (~500) occasions for just about one decade now, this is the first time I'm seeing it in the "one man's modus ponens…"-way.

"Anything untrue isn't there to be with." 🔄

Show older

a Schelling point for those who seek one