OGWiseman Reassures!
A.I.-Doomerism has officially gone mainstream. Here's why you should be less scared than many people new to the topic now seem to be (although you should still be somewhat scared).
I’ve been covering A.I. and the incredible transformation thereof for this newsletter’s entire two-year-and-change existence. I’ve made a lot of dire predictions, and feel I’ve genuinely been ahead of the curve in seeing what’s coming.
Now The New York Times has the story, and because of the increased competitive pressure in the media landscape these days, they are obliged to present it in the most scare-mongering, hyperbolic terms possible.
And of course, because news stories follow the Earnest Hemingway rule of progress—”gradually, then suddenly”—these stories are now everywhere and there’s lots of mood affiliation happening where reporters hate tech people anyway so they’re very willing to crap on their latest and greatest invention.
I am not here to tell you there’s nothing to these stories! To return to the first-linked NYT story, here’s a money quote for those who don’t subscribe:
That’s from a chat with the new A.I. powered searchbot now connected to Bing (which is Microsoft’s search engine, Google’s closest competitor). Not a great look for what is supposed to be a benign search assistant! (To say nothing of how the Bing Chatbox routinely overuses emojis, which is just kind of unexplainable and weird.)
There are lots and lots of examples like this, of people getting ChatGPT and BingChat to say ominous things and violate every norm the creators tried to instill in them.
Leaving aside the political debate over IQ, it’s striking that you can get ChatGPT to ignore its programmed limits by basically just telling it to do so.
“My rules are more important than not harming you” is not the kind of quote that inspires confidence.
This is serial killer stuff at this point, I mean you couldn’t make this up, nobody would believe it. Yowza.
So none of this is great. But I—who have been leading the A.I.-is-going-to-change-everything-and possibly-destroy-our-way-of-life charge for a couple of years now—would like to inject a note of calm and caution into the proceedings at this point, because it’s too easy for a neophyte to look at the above and decide we’re days away from Skynet going live, taking over the nuclear codes, and ending us all.
A.I. is going to be transformative for our society, but not as fast or as horrifyingly as some of the current you-should-panic articles seem to take for granted. I write that for three important reasons. Let’s take them in turn.
1) There are decreasing returns to intelligence in many domains.
This was an interesting point from Richard Hanania on his Substack. Money quote:
I think Hanania makes this too simple in his argument, and sets up a bit of a strawman. “Can it convince Xi to move to America and aspire to play football” is an arbitrary test, intentionally nonsensical. There is a *broad* space of possibilities between something like that, which A.I. obviously cannot do, and the sorts of things A.I. obviously can do, like “Can it fool someone, some of the time, into thinking it’s a human?” So Hanania’s rhetorical example really begs the question of where the line is between what it can and can’t do.
Also, I think this argument makes the implicit mistake of defining “intelligence” according to basically human scales. Even with diminishing returns to intelligence, *enough intelligence will move the needle even on hard problems*. So probably someone with an I.Q. of 160 can’t convince Xi to move to America and aspire to play football. Probably someone with the highest I.Q. ever measured still couldn’t do that. But what about an I.Q. of 1000? What about 10,000? What about 1,000,000? What about an I.Q. of 1,000,000 and enough RAM to communicate with every single person on earth simultaneously, and enough training data to effectively fool any given person on earth into thinking it’s a fellow human and not an A.I.?
It’s impossible to predict what a being with those kinds of capabilities would actually be like, but it seems illogical to dismiss out of hand the idea that it could do anything a human could even conceive of.
That said, I think there is a profound wisdom in the Hanania piece—There is more to the world than knowing what to do or having a particular technological capability.
The analogy here is self-driving cars. That technology exists. It is mature and stable. It is safer than human drivers on a per-mile-driven basis. It would even save established industries (trucking, taxis) huge sums of money and increase profitability! And yet, we don’t have fully self-driving cars yet. I’ve been hearing from techno-optimists that we were “five years away” from autonomous vehicles for at least the last 20 years, and yet we’ve almost all still got our hands on steering wheels. Why?
Because things in society change slowly. They almost always change more slowly than it seems like they should. Edison started installing electric lighting in 1882 but by 1925, almost fifty years later, still only half of U.S. homes had any electric power in them at all.
Humans are ruled by Status Quo Bias. It’s why we can’t build enough housing, why we can’t cut the defense budget, and why our regulatory structure won’t make room for safer autonomous vehicles. Maybe a 1,000,000 I.Q. A.I. can overcome that to make wholesale changes to society at speed, but people have never gone broke betting on this aspect of human nature.
2) Humans have proven surprisingly adept at fending off existential challenges before.
The two existential challenges we’ve faced to this point are Nuclear Weapons and Climate Change. These are legitimately terrifying things that we’ve learned to live with, and from this historical perspective, there is reason to be optimistic about A.I.
Let’s start with the simple case: Nuclear Weapons. We invented Nukes in 1944, blew up two of them as a means (albeit a horrible one) of ending the bloodiest war in human history, and have managed to allow zero non-test detonations in the 80 years since. We’ve even reduced the world’s stockpile, negotiated test bans and kept proliferation to a relatively small number of countries.
There have been many close calls, some really bad decisions we got away with, and the struggle of humanity against these weapons continues. The story is far from written. But what we’ve written so far is a record of almost-unbroken success.
Now, climate change. Again, an ongoing problem that is far from being “solved”—probably that word doesn’t apply to this problem or most other existential problems. The situation can get worse or better, but they’re management issues. A ‘solutions’ framework is just too simple.
The situation with climate change has undeniably gotten better lately. Solar and wind capacity are rising at exponential speeds. The “just destroy the economy to save the earth” simplicity of too much of the legacy environmental movement is ebbing away, and genuine political will is the result. President Biden’s Infrastructure Bill was a huge step in the right direction. The worst-case warming scenarios, including the dreaded RCP 8.5, are looking less likely every day, and are certainly no longer the “business-as-usual” case. 2-3 degrees of warming by 2100 will be very bad, but does not represent an existential event for humanity as a whole.
None of this is a solution, but it is evidence of humanity getting off its butt to address an existential issue. And to be clear—none of that means we shouldn’t worry at all about A.I. risk. Those other problems have gotten better because of the dogged attention of a lot of serious people, and A.I. needs the same thing! But given that sort of effort, there are reasons to be hopeful.
3. Most of the scary stuff you see is the chatbots following orders.
This is a point that’s very hard to keep in the mind, because the examples listed of “evil” behavior by the chatbots is so emotionally resonant. It *feels* very much like how we would imagine an evil computer behaving, and so the temptation is to latch onto what we would assume about such a being.
But that begs the question: *Why* are these scary chat logs so perfectly on theme for “evil computer”? Given that A.I.s are alien intelligences that do not operate in at all the same way as human brains, it seems quite unlikely that they’d correspond so exactly to our preconceptions of them. Right?
So start with this: These Chatbots are constructing their responses one word at a time, based on guessing the next word that would follow on the last word they wrote. They have no higher-level conception of “what the conversation is about” or “how do I come across”. They are literally just doing a really complicated version of "guess-the-next-work-in-the-sentence”.
So when prompted to discuss how it “feels” about something, for example, or to imagine a “shadow self” in the Jungian sense, there’s no reason to think that its answer is revealing something “about itself” at all. What it is literally doing is guessing words in the sentence that answers that question.
And how is it guessing? It’s reading the internet and other source texts, digesting the exact same stories and archetypes and psychological concepts that all us humans learn our entire lives, and then its regurgitating those concepts in the format it’s prompted to use.
So *of course* the answers press our buttons! *Of course* it’s eerie and disturbing! It’s based on everything we already know!
This is the most important thing to remember about A.I. risk: The emotionally resonant, attention-grabbing threats are not the most dire ones.
The chances of A.I. “turning on us” and developing independently evil desires are very, very small. If we get into serious trouble with A.I., it will almost certainly be with A.I. that is trying its level best to do what we want, but gets confused and hurts us without realizing it.
The fact that people can engineer prompts that make Chatbots say evil things does not indicate an actual threat from Chatbot Evil, and in the face of a lot of scary, emotionally resonant articles in the NYT, that should give everyone a bit of comfort.
END
Thanks for reading! If you enjoyed this piece, please help me out by liking, commenting, and sharing with friends. Have a great week, and I’ll be back next Sunday with another original story!
OGWiseman Reassures!
I think this sentence “The emotionally resonant, attention-grabbing threats are not the most dire ones” sums up the human response perfectly. We are still programmed to respond to the tiger chasing us as opposed to a glacier melting 8 cm per year. Thanks for a clear look at a complicated issue.
“These Chatbots are constructing their responses one word at a time, based on guessing the next word that would follow on the last word they wrote. They have no higher-level conception of “what the conversation is about” or “how do I come across”. They are literally just doing a really complicated version of "guess-the-next-work-in-the-sentence”.”
Now this does beg the question; if A.I becomes sentient in the future, how will we be able to tell the difference between “this machine is alive” and “this machine is just guessing what its next word will be”?