I find the short story “Steve Fever,” by Greg Egan, horrifying–and here’s why.
(spoilers below, so go read it first if you’d like. It’s not too long, introduces a cool idea that will get you thinking, and most people will consider it mostly fun)
Steve is a tech genius/entrepreneur, signed up for cryo, that creates an AI hive-mind and dies shortly thereafter. He’s constructed the AI so it’s primary goal is to revive him in the future. Unfortunately he died in a fiery car accident, and there’s no brain left to preserve. But the AI’s utility function is robust against corruption or drift, so it sets about trying to revive him. Steve left a ton of personality data behind. Lots and lots of personal writings, recorded public appearances, social media posts, interviews, etc. So the AI creates a best-guess approximation of his mind, installs it on a currently-living bran (temporarily hijacking a person’s life in the process), and then tests to see how good of a fit it is. It does this testing by recreating the initial conditions of an event in Steve’s life, and seeing if their Model Steve reacts the same way that the Original Steve did historically. If so, great, try with another scenario! If not, abort, tweak the model, and try again. Iterate until a functionally-identical Steve can be recreated.
This terrifies me in two ways. The first is that (when I think of it) it scares me to post anything anywhere. Every trace I leave narrows the range of successful Eneasz-recreations, making future-reviving harder. I guess that’s a good thing overall, because it means revived-me will be that much closer to original-me. :) But I’m extremely aware of the fact that there’s a lot of stuff I *don’t* post or make a record of. And those things are also parts of me. The reasons for that are mostly embarrassment and social sanctioning… there’s some things I’d just rather not share with the world. And also the majority of it is boring, nobody needs to hear all my stupid little worries or daily thoughts. But recording some things and leaving out others leaves a skewed record, and since the skew is mostly in one direction, any future recreation based on these will be twisted away from who I am now. Is that a good thing? Should I mostly post the stuff that makes me happy, and shows off my abilities, so future-me will be well-adjusted, happy, and good at stuff? I’d want to keep all my deep fears and neurosis as hidden as possible in that case. But then am I even recreating myself, or just a creating an idealized child/successor?
(and is this why some people seem like super-happy half-people?)
The much more horrifying worry is that I might be the Model Eneasz. I may be running through a simulated historical scenario right now. Am I reacting the way Original Eneasz did? If I slip up in any way, the simulation is aborted and I get deleted, to be replaced by a higher-fidelity Eneasz. My continued existence depends on taking the action that isn’t the morally-best or financially-best or socially-best, but the most like an no-longer-existing-person who I may only partially resemble and whose motivations and psychology I can only guess at. And *not* doing something (like not posting this) might be just as bad, if the Original Eneasz did post it. Do I just do the best thing I can, and hope Original Eneasz was a basically good person? He can’t be that bad, if the future is willing to bring him back, right?
Plus, if I am being simulated to refine a model, it means Original Eneasz probably did something interesting or momentous enough in his life to be deemed worthy of recreating. (unless future society is altruistic enough to want to recreate everyone <3 ) I don’t feel like I’ve done anything that noteworthy yet, which leads me to think… what the fuck is looming in my future?
(Of course, I could just be the first-run of Eneasz, a pleb who will never amount to enough to be worth recreating in the future, and all this worry is for naught. Which may be even worse, because then I die forever. >< )
It’s all very stressful.