a binary point in the middle

I will also make the case that modesty—the part of this process where you go into an agonizing fit of self-doubt—isn’t actually helpful for figuring out when you might outperform

the overall competence of human civilization is such that we shouldn’t be surprised to find the professional economists at the Bank of Japan doing it wrong.

We have a picture of the world where it is perfectly plausible for an econblogger to write up a good analysis of what the Bank of Japan is doing wrong, and for a sophisticated reader to reasonably agree

start off by following the majority opinion; and then only adopt a different view for good and convincing reasons.

While hallucinating, she might “act as she wants unencumbered,” but she could hardly be said to be acting of her own free will.

be those

It’s not the type of power we think of ourselves as having.

that about

part why

think of all these cases as involving an inconsistency between the policy that an agent would want to adopt, at some prior point in time/from some epistemic position (e.g., before the aliens invade, before we know the value of the X-th digit, before Omega makes her predictions), and the action that Guaranteed Payoffs would mandate given full information.

if you understand all of this, and make your decisions in light of full information, additional disputes about what compliments and insults are appropriate don’t seem especially pressing.

the only remaining dispute is whether, given these facts, we should baptize the action of paying in the city with the word “rational,” or if we should instead call it “an irrational action, but one that follows from a disposition it’s rational to cultivate

if, in the desert, I could set-up some elaborate and costly self-binding scheme – say, a bomb that blows off my arm, in the city, if I don’t pay — such that paying in the city becomes straightforwardly incentivized, I would want to do it. But if that’s true, we might wonder, why not skip all this expensive faff with the bomb

Guaranteed Payoffs: When you’re certain about what the pay-offs of your different options would be, you should choose the option with the highest pay-off.

should you pay him? Both CDT and EDT answer: no. By the time you get to the city, the risk of death in the desert is gone.

CDT ignores letters like this. But CDT also gets given termites once a year. EDT, by contrast, pays, and stays termite free.

I predicted that

paying seem

“you win your”

Imagine, for example, that I decide to refrain from smoking a million times in a row. If the case’s hypothesized correlations hold, then I will in fact spawn, consistently, without the lesion. In that case, though, it starts to look like my choice of whether to smoke or not actually is exerting a type of “control”

maybe the right thing to say here is just that the correlations posited in smoking lesion don’t persist under conditions of “play around however you want”

it turns

case a

view just

some via

e.g

give

my conviction about one-boxing start

one boxing

one box

you grandfather

a living in

it prompts CDT

/or

All of three

does feel

that I think

been been

that shows

these distinctions

one who want

in an already-painted painting, the future has already been fixed, too: you just don’t know what it is. And when you act, you start to find out. Insofar as you can choose how to act – and per compatibilism, you can – then you can choose what you’re going to find out, and in that sense, influence it.

As long as you and your copy’s choice are correlated, CDT is going to ignore that correlation, hold p constant given different actions

CDT imagines that we have severed the ties between you and your copy, between you and the history that determines every aspect of you.

you can change the past, here, about as much as you can change the future in a deterministic world: that is, not at all, and enough to matter for practical purposes.

We must distinguish between the ability to “change things” in this sense, and the ability to “control” them in some broader sense.

If you choose cooperate, it will always have been the case that Monday-Joe + process P outputs cooperate. If you choose defect, it will always have been the case that Monday-Joe + process P outputs defect. In this very real sense – the same sense at stake in every choice in a deterministic world – you get to choose what will have always been the case, even before your choice.

because only one of (a) or (b) is compatible with the past/the physical laws, and because you are free to choose (a) or (b), it turns out that in some sense, you’re free to choose the past/the physical laws (or, their computational analogs).

it is genuinely up to you what you write, or do.

your copy is your puppet. But equally, you are his puppet. But more truly, neither of you are puppets. Rather, you are both free men

it feels like what compels me is a direct, object-level argument, which could be made equally well before the copying or after.

if you find yourself reasoning about scenarios where he presses one button, and you press another – e.g., “even if he cooperates, it would be better for me to defect” – then you are misunderstanding your situation. Those scenarios just aren’t on the table.

you can make him do whatever you want

for all intents and purposes, you control what he does.

CDT, in this case, defects. After all, your choice can’t causally influence your copy’s choice

you get public criticism for doing things and making mistakes, not for failing to do anything at all.

the perception of extravagance often has little to do with the amount of money actually being spent.

most organizations don't have plans, because I haven't taken the time to personally yell at them.

it is still Eliezer Yudkowsky writing up this list, says that humanity still has only one gamepiece

are (a) people who might not be able to do equally great work away from tight feedback loops, (b) people who chose a field where their genius would be nicely legible

on tackling its enormous lethal problems.  These problems are in fact out of reach; the contemporary field of AI safety has been selected to contain people who go to work in that field anyways.  Almost all of them are there to tackle problems on which they can appear to succeed and publish a paper

if you're fighting it in an incredibly complicated domain you understand poorly, like human minds, you should expect to be defeated by 'magic' in the sense that even if you saw its strategy you would not understand why that strategy worked. 

Any pivotal act that is not something we can go do right now, will take advantage of the AGI

corrigibility runs actively counter to instrumentally convergent behaviors

CEV specifically, is unworkable because the complexity of what needs to be aligned or meta-aligned for our Real Actual Values is far out of reach for our FIRST TRY

If you perfectly learn and perfectly maximize the referent of rewards assigned by human operators, that kills them.

lethal-to-us possibilities exist in some possible environments underlying every given sense input.

When an outer optimization loop actually produced general intelligence, it broke alignment after it turned general, and did so relatively late in the game of that general intelligence accumulating capability and knowledge

around half of the alignment problems of superintelligence will first naturally materialize after that one first starts to appear.

if you're starting with an unaligned system and labeling outputs in order to get it to learn alignment, the training regime or building regime must be operating at some lower level of intelligence*capability

proposals for alignment fall apart as soon as you ask "How could you use this to align a system that you could use to shut down all the GPUs in the world?"

a

You cannot dissociate intelligence from the context in which it expresses itself.

all intelligent systems we know are highly specialized.

GOFAI

argument

This argument has the flaw of potentially conveying the beliefs of ‘reduce AI and bio x-risk’ without conveying the underlying generators of cause neutrality and carefully searching for the best ways of doing good.

when I look for technologies emerging now, still in their infancy but with a lot of potential, AI and synthetic biology stand well above the rest.

are

as the pool of meat/dirty energy consumers.

mitigated

the fend

with context is judged.

can'd

"what the hell do you then?"

of of

facebook

with show up on street corners with clipboards

is

One thinks that moral claims such as “X is wrong” just mean the same thing as, e.g., “X reduces net happiness,” and she thinks that some of these claims are true. Another thinks that moral claims such as “X is wrong” refer to an irreducible property of wrongness