I posted this on the /rational subreddit, but I’m mirroring it here as well. I want to finally talk about Three Bodies at Mitanni, now that the people I talk to most have access to it. :)
Lots of spoilers below!
.
.
.
I’m still into themes of consciousness, specifically as discussed in Meditions on Moloch and Peter Watts’s Blindsight. I find them deliciously frightening. But what I really like about 3BM is that it makes me really freakin’ worried about myself.
Take Bayesians vs Barbarians which makes the case that, if an actual Rational Society was attacked by fanatics, the Rational survival action is to delegate the amount of people and resources necessary to defeat the fanatics and convert them to war-making purposes until victory is assured. People would likely be selected based on some combination of effective fight ability and lottery. This would include whatever self-modification is necessary in order to WIN as efficiently as possible.
I completely agree. I would hope such a lottery wouldn’t choose me, but if it did I would submit to warrior-modification for the good of my society. This IS what I want.
Which sounds suspiciously like the horror-punch of 3BM.
Which also leads me to realize that any time I give up any immediate pleasure for the promise of future utility (saving money to invest rather than spending on hedons; working on podcast or writing rather than going out with friends; etc) I am in effect saying “This is fine. This is what I want.” And it really is! A life of pure in-the-moment hedonism would suck, I want to make things that last, and to have future financial security! But, well, how far am I willing to push that before I become Mitanni-esque? Before I’m not really working for me anymore, I’m just benefitting “the system” and deluding myself that it’s for me?
Basically, anytime my core values are convincingly attacked like this, I feel really creepy and shivery, and I like that feeling.
2 Responses to “Three Bodies At Mitanni (again)”
Leave a Reply to embrodski Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
See, to me the Mitanni represent an overly simplified moral code. There are some things which are not justified in the name of staving off extinction. I’m not sure where I’d draw that line exactly. There’s even an argument to be made that killing off the Mitanni is crossing that line.
On another note, most of the reviews I find seem to criticize the reasons for choosing genocide, in effect saying the Mitanni aren’t an existential risk. That seems to be missing the point. I’m glad you didn’t jump on that bandwagon.
I also like your take on this. :)
> most of the reviews I find seem to criticize the reasons for choosing genocide, in effect saying the Mitanni aren’t an existential risk.
I’ve heard it said that SF is a conversation that enthusiasts are having among themselves. But the field has grown big enough that some parts of the conversation aren’t heard by others. I think that the reviews written by people who said there was no risk (and I read a couple myself) are made by people who are not involved in the same part of the conversation that we are. They do not follow existential risk thinkers. They do not think of deep-time, or about utility-maximizers, or about value-corruption, because they haven’t been exposed to people who warn about those things.
So when they read 3BM they are missing all that vital conversation that came below, and essentially joining a conversation right as someone says “You make a compelling case for infanticide” without having heard the case at all, and assuming the speaker must be crazy. I think once they are caught up on the conversation (which may be never, if they aren’t interested in it) I’ll be interested in their opinions, but until then there’s not much that we can meaningfully say to each other about this story.
Of course running into such a person is a wonderful opportunity to bring up all those things, and catch them up, if one is so inclined. There’s still tons of great conversation to be had, it just has to start back a step or two.