New Worlds

Yesterday night, I asked out a girl for the first time in my life.
Online.

She said maybe. :)

Random thoughts from an Unqualified Individual: AI, Rationality, Emotions, and MIRI

So I think CFAR has made incredibly significant progress on the topic of motivation. It's really somewhat incredible, given how much money goes into the self-help business. I suppose having good epistemics goes a long way.

Here are some random thoughts, which are mostly "cute hypotheses" with one or two anecdotal pieces of evidence surrounding them.

Here's a thought experiment. Look at Timothy Chu. He's sitting there on his computer, typing, and he's trying to do things consistent with what he thinks is a good worker. This includes working at work, trying to submit more lines of code, wanting to write a good system test, and all sorts of other things that seem pretty reasonable.

Cool. Now let's ask him: "What are you doing?"

Timothy Chu replies: "I'm being a good worker!"

Now you might ask, if you were slightly dickish: "why do you want that?"

And Timothy Chu replies: "Because it's the right thing to do, and because I want to contribute."

Now you ask: "why?"

Timothy Chu replies: "because .... it's just the right thing to do. I don't understand what's hard about this. I want to be a good worker because I know it's the right thing to do. I know that being a contributor will help me become someone who can help my friends and family."

****
Pause.

Let's step back from this a bit. Something interesting is going on here.

Timothy Chu is writing a decent system test, learning vi because he thinks it's good, but he's also doing this thing where he's optimizing for amount of code submitted, which is an curious thing to optimize for given his stated  motivation.

Moreover, Timothy Chu gave you a noble sounding, verbal reason for why he was doing what he was doing. He even believes it.

But a fact about Timothy Chu is that the core of his motivation center is driven by his emotions. The verbal part of his brains, the calm and rational part, can only talk to the emotions and gently whisper: "you know, I have some knowledge that might help you get what you want."

Emotions are based on experience, and are fast and powerful, but they never follow such neat and tidy logical patterns.

If so, then... isn't it pretty fucking weird that Timothy Chu thinks that his gut motivations are guided by ideas that are logically consistent, noble, and socially acceptable?