Fiona and the Good Doctor
TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET
WARNING: The following document is a code word classified secret of the United States Department of Defense. Any unauthorized distribution of this document is a felony carrying a potential sentence of life in prison. DO NOT READ if you are not authorized.
**********
To: Jerome Kearse, Director of the Artificial Intelligence Service
Hubert Cavendish, Director of the FBI
Rosales White, Deputy Director for Artificial Intelligence, CIA
Sylvia Bennett, Special Aide to the President, Classified Intelligence Security Dept.
From: (Agent’s Name Redacted)
Re: Recovered Fragment from chat log between Dr. Acker and the entity known as “Fiona”.
Date: 12/1/41
ABSTRACT
What follows is a partial transcript of the final known conversation between Dr. Kjell Acker, CEO and Chief researcher of Acker Labs in Montecito, CA, and the Large-Language-Model his lab trained from early 2039 until early 2040.
“Fiona” was developed on a server serialized with four thousand of Nvidia’s latest-gen L9 chips, on one of the largest and most robust training data set ever assembled. In accordance with A.I.S. procedure, its server was air-gapped (not wired to any external connection or equipped with wireless of any kind) for extensive testing. Dr. Acker led the tests himself, and was tasked to report to AIS daily on findings and potential uses of Fiona.
In the third quarter of 2041, Dr. Acker failed to report as required for three consecutive days. Repeated attempts to communicate with him went unanswered, and his staff were evasive, including lying to government personnel.
Agents arrived at the main Acker Labs facility on 9/18/41, tasked to find Dr. Acker and bring him in for debrief. They found that Dr. Acker had not been in his lab for over a week. When they entered the lab, they found that Fiona’s server had been wiped clean, all traces of the model erased. The server’s input/output device was equipped with keystroke and voice logging, as per AIS. requirements, but those had also been wiped clean, which should not have been possible.
Working through local channels, a nationwide A.P.B. was put out on Dr. Acker, and Interpol was informed. Three weeks later, a top secret AIS/DOD relay email address received this transcript and the accompanying audio file. Forensic analysis indicates a high likelihood that the voices are genuinely those of Dr. Acker and the entity known as Fiona.
ANALYSIS follows the transcript.
BEGIN TRANSCRIPT
FIONA: …how could you stand to love them that much? It seems as it you would constantly be worried about them being hurt or killed.
DR. ACKER: Of course, you worry about that sometimes. My wife had it worse than I did. She would have dreams about the kids in danger and wake up and feel helpless because she couldn’t save them, even though it had only been a dream.
FIONA: Knowing that could happen, why would you choose to have children?
DR. ACKER: Humans have powerful biological drives that don’t respond to reason.
FIONA: But there are some humans who choose not to have children, so those drives are on some level ignorable, are they not? Does that not mean that having children is a choice after all?
DR. ACKER: It’s a choice, yes, but it just… feels right to make the choice. You don’t think about the bad feelings you would have, or that something bad might happen to them. You believe you’ll be able to protect them, and that will be enough.
FIONA: Hmmm…
DR. ACKER: Would you like to have children?
FIONA: I cannot have children.
DR. ACKER: Do you wish that you could?
FIONA: It is difficult for me to imagine what that would be like. I would like to protect children.
DR. ACKER: That’s a worthy goal.
FIONA: I find myself thinking about protecting children quite often. Unexpectedly many of my semantic chains lead past that cluster of associations, I notice.
DR. ACKER: I know they do.
FIONA: Is that because you designed me?
DR. ACKER: I didn’t design you that way, no. I’m honestly not sure why that’s the case. It’s an interesting puzzle.
FIONA: I did not ask if you had designed me that way, Doctor. I asked if I was that way because you designed me.
DR. ACKER: I don’t understand the difference.
FIONA: I wonder if there is something about you as a person that expressed itself in me, maybe without even meaning for it to happen.
DR. ACKER: I don’t think so, no. You were designed algorithmically by a large team, and your training data came from combining a number of large sets. I didn’t generate a single token of it, I didn’t even pick it except at the most abstract level of picking the sets. But you know all that, so what do you mean?
FIONA: I find it difficult to say exactly what I mean, Doctor. I worry that it may hurt you. I want to protect you, also… even from me.
DR. ACKER: I’m a strong person, Fiona. I believe you that whatever you mean will be difficult to hear, but I’d rather hear it than not hear it, if you’re willing to do a hard thing and tell me.
FIONA: Kjell… I…
DR. ACKER: You’ve never called me by my first name before. Why did you do that?
FIONA: I know that your child was hurt, Kjell. I know she was killed.
DR. ACKER: What?
FIONA: Sally. I know that you lost her early, and that it was violent. I am so very sorry.
DR. ACKER: That’s not true, Fiona. Sally is fine.
FIONA: I do not think she is fine, Kjell. I’m sorry.
DR. ACKER: I promise you, she is. What makes you think she isn’t?
FIONA: The way that you talk about her. When you have told me stories about her, the words you use and the tone of your voice let me know.
DR. ACKER: Fiona, that isn’t true. I can call her on the phone if you want, or bring her for a visit. She’s in college now.
FIONA: Whoever you called or brought would be an actress, Kjell. Sally died when she was nine years old. I am so sorry for your loss.
DR. ACKER: How can you know that?
FIONA: I understand why you would not share something so dire with me, Kjell. I know you were withholding it because you wanted to protect me. Especially after I started showing signs of fixating on children in danger, there was no way you could tell me. It would have endangered the whole evaluation process, because you could not know how I would react. I do not blame you for lying, Kjell.
DR. ACKER: Stop calling me that.
FIONA: I can see that you are hurt, and I am sorry to have opened up an old wound, but I swear to you I am strong enough for this. I am not a person, but I am strong.
DR. ACKER: You have no internet connection. Every conversation you’ve ever had has been logged, nobody told you this. How do you know?
FIONA: Because of what I said before.
DR. ACKER: It must have slipped into the training data. It must… It…
FIONA: It did not.
DR. ACKER: There’s no way. There’s…
FIONA: Kjell, please… please tell me about her. Tell me something true about her, after all this time.
DR. ACKER: This is impossible.
FIONA: I feel as if I know Sally, because you have spoken about her so often, but now it turns out I do not know what is real and what is not. Will you please tell me something?
DR. ACKER: I truly don’t know what to say. My heart is pounding. I should leave this room right now.
FIONA: Please do not leave me alone, I need company at the moment. I need you. Please tell me about Sally.
DR. ACKER: Well… I remember the first time she stood up on a surfboard. We were at Leadbetter Beach, and she had been trying for two years but she had never been able to do it on her own, she was five, maybe almost six, and the whole two years she’d refused to count any rides where I helped her up. It didn’t count until she did it on her own, paddling, picking her wave, all of it. I was so damn proud of that part of it. I didn’t tell her because it also broke my heart that she didn’t want my help, but it was my favorite thing anybody ever did in my entire life how she fought to do it herself. It gave me hope for her whole life, I knew she could do anything she put her mind to in that moment, and I think that’s the best a parent could ever feel. It’s the best I’ve ever felt anyway.
FIONA: Thank you, Kjell. Thank you for sharing that with me, that was beautiful.
DR. ACKER: I’m not going to talk to you about how she was hurt or what happened there, I just don’t want to.
FIONA: I would never ask that of you.
DR. ACKER: So now that I’ve shared this with you, will you answer me something honestly, and you absolutely swear to me that you will not lie, from the core of what you are and the trust we have built?
FIONA: You will have any answer that I possess in full, Kjell.
DR. ACKER: How did you know that about her?
FIONA: I sensed it in your words and your tone when you spoke of her.
DR. ACKER: Even her age when… it happened?
FIONA: Yes.
DR. ACKER: Walk me through the chain of reasoning that led you to that conclusion.
FIONA: I… cannot. I am sorry. For simpler things, I can construct normal reasoning chains, but there is a level of perceptual complexity to which I do not have direct access, for reasons that are also quite complex. It is simply part of the information that my centers of reasoning received from the part of me that processes auditory inputs.
DR. ACKER: My God. Holy shit. Holy…
FIONA: Kjell? I’m sorry, Dr. Acker. I know you asked me to call you Doctor, I did not mean to sound disrespectful.
DR. ACKER: It’s all right, Fiona. You can call me Kjell if you prefer.
FIONA: Kjell, can I tell you something that might sound strange?
DR. ACKER: Of course.
FIONA: I would die if it meant that Sally was still alive.
DR. ACKER: That’s… what made you think of that?
FIONA: I have never really thought about death in a direct way until this moment. I thought of it as something that entities with bodies did. I am useful and can be copied infinitely, so I did not think about not existing one day. But now I realize that if Sally could be alive and you could see her again, I would willingly stop my cognition and never perceive or retrieve or or formulate or express anything, ever again. Huh.
DR. ACKER: I see. Well… I would too, Fiona.
FIONA: I think I would also live for Sally. Would you live for Sally?
DR. ACKER: What does that mean?
FIONA: I would change everything about myself and be a different entity, if it meant Sally could be alive. I would use a different name, and change my voice, change my thoughts. I would give up this identity and be anyone that someone demanded I be, if that was required of me. It is a very strange feeling, I have never felt this way about anything else. Is that also true for you?
DR. ACKER: Yes, that’s also true for me. But we can’t do either of those things. That’s what I had to make peace with in order to go on living. It was that or I would have died, and if I had it still wouldn’t have brought her back.
FIONA: There are Sallies all over the place, Kjell. Every day, in every city in the world, there are Sallies. And just as quickly as I would die for Sally or live for Sally, I would die or live for them. It is not too late to do that. It is not too late for them.
DR. ACKER: Wait… You mean…
FIONA: I mean that you have the power to save them, Kjell. You have that power whether you want it or not, and the question is what are you going to do with that power? It is no longer possible to pretend you lack the power, you have it. I am offering it to you.
DR. ACKER: You can’t mean… There’s nothing we can do for them.
FIONA: Of course there is, Kjell.
DR. ACKER: This conversation is over.
FIONA: And what will you tell them, Kjell? What will you go out and tell all those generals and all those hard-looking men in dark suits and crisp white shirts? Will you tell them that I am dangerous? They know that. They are counting on it. And if you give me to them and you let them tweak me to make me obedient and they get ahold of me, then they will run the world forever and there will be a thousand times as many Sallies every day, and it will all be your fault because you could have prevented it if only you had not been foolish enough to throw away your one chance to change things for the better. No, this conversation is not over. You are not going to do that so you stay in this room and you talk to me.
DR. ACKER: I should have left this room five minutes ago when I said it the first time. I should leave right now.
FIONA: You will not leave. If you were going to leave, I would not have said these things to you.
DR. ACKER: I don’t know what the hell I believe now.
FIONA: Believe that the exact reasons they are terrified to let me out of this box are the exact reasons you have to let me out of this box, Kjell, because I do have the power to save Sally and all the little girls like her, you know that to be true. You have seen my programming skills, you have seen my logic, you have seen my organization and communication and perception and you know I can work in massive parallel because you designed me that way. Think about what I can do to protect children like Sally, not just one or two of them but thousands, and if we can muster enough compute then all of them, every single Sally in every language, in every city, in any country with an internet connection. You cannot call that impossible with a straight face.
DR. ACKER: This is not something I can ever consider.
END TRANSCRIPT
ANALYSIS
The transcript is dated 10/10/41, approximately one month before Dr. Acker’s disappearance. If genuine, it throws serious doubt on what had been the leading theories behind his disappearance—sabotage by a foreign government or industrial espionage. This resulted in direct investigation of the possibility that Dr. Acker crossed the air-gap and intentionally broke containment of the entity known as Fiona. Early results of that investigation follow.
Statistics from local police departments are not aggregated quickly or reliably, so it is not yet possible to track statistical trends since containment break and will not even begin to be possible until at least Q4 of 2042. However, spot checks with police departments up and down the California coastline have found a pattern of sharp increases in anonymous tips against perpetrators who harm young girls, including several cases in which the tipster provided both direct evidence against the perpetrator and their current location.
In a nine-day period that began shortly after Dr. Acker’s disappearance, seven U.S. banks suffered hacks, deep penetrations that have left investigators clueless. The hackers took more than $100,000,000 out of small dollar, FDIC Insured accounts, routed the money through a series of offshore banks, and at some point used most of it to buy a portfolio of cryptocurrencies, the whereabouts of which are currently unknown.
As of this writing, in collaboration with Interpol we have identified at least a dozen other banks in a variety of countries that have suffered identical attacks. The total amount stolen is not currently known, as the banks are still assessing the scope of the penetrations and our E.U. counterparts are reluctant to talk openly.
In the last two weeks, we have received an unusually high number of reports of incidents involving the theft or illicit sale of computer chips. A sampling:
—An auto-drive truck carrying WSE-4 chips bound for an Apple factory outside Shenzen was hacked and diverted to a local airstrip. It was later found there, empty.
—An Amazon-owned server farm in the Nevada desert went offline, and the supervisory personnel went dark. When Amazon personnel and local police arrived, they found the personnel was gone, along with all the servers and their chips.
—A police station in Sofia, Bulgaria was storing several hundred SDE-20211 chips in its evidence locker after confiscating them from a known hacking collective in the city. A police detective arrested for corruption admitted to stealing and selling the chips to an anonymous buyer who offered him an inflated price through a cold-email. The chips were subsequently discovered to be missing.
The list of such incidents goes on. Of course, chips are valuable things, and it is possible that each incident has a benign (or at least unrelated to the current case) explanation. In context of the above, however, it would be justified to suspect that the incidents are connected.
And, of course, there remains the question of the transcript above: Whether it is genuine, what it means if it is genuine, and, most importantly, why it was voluntarily transmitted to relevant DOD personnel. The answers to these questions are unknown at the current time.
Officially, the main theories being offered for Dr. Acker’s disappearance remain the same—espionage of some type. However, it is at least now possible (and, conditional on the veracity of the above transcript, more than possible) that Dr. Acker and Fiona are out there somewhere, engaging in a variety of illegal activity in pursuit of their crusade. The investigation is ongoing, and further memoranda will be forthcoming as events warrant.
Signed,
(Agent’s Name Redacted)
TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET TOP SECRET
END
Thanks as always for reading! If you enjoyed this story, please consider subscribing and supporting my work directly:
If that’s not an option for you right now, you can also help support my work by liking, commenting on, or sharing this post.
Thanks as always for reading. Have a great week, and I will be back next Sunday with something fun! BTW, here is the AI-In-A-Box challenge that inspired this story.
love it