Lots of spoilers below!
I’m still into themes of consciousness, specifically as discussed in Meditions on Moloch and Peter Watts’s Blindsight. I find them deliciously frightening. But what I really like about 3BM is that it makes me really freakin’ worried about myself.
Take Bayesians vs Barbarians which makes the case that, if an actual Rational Society was attacked by fanatics, the Rational survival action is to delegate the amount of people and resources necessary to defeat the fanatics and convert them to war-making purposes until victory is assured. People would likely be selected based on some combination of effective fight ability and lottery. This would include whatever self-modification is necessary in order to WIN as efficiently as possible.
I completely agree. I would hope such a lottery wouldn’t choose me, but if it did I would submit to warrior-modification for the good of my society. This IS what I want.
Which sounds suspiciously like the horror-punch of 3BM.
Which also leads me to realize that any time I give up any immediate pleasure for the promise of future utility (saving money to invest rather than spending on hedons; working on podcast or writing rather than going out with friends; etc) I am in effect saying “This is fine. This is what I want.” And it really is! A life of pure in-the-moment hedonism would suck, I want to make things that last, and to have future financial security! But, well, how far am I willing to push that before I become Mitanni-esque? Before I’m not really working for me anymore, I’m just benefitting “the system” and deluding myself that it’s for me?
Basically, anytime my core values are convincingly attacked like this, I feel really creepy and shivery, and I like that feeling.