OGWiseman Reminds!
It feels like A.I. hype has ebbed, but the next three years are going to be insane, plus other stuff!
For what I assume are the usual blistering-pace-of-media, constant-dimunition-of-attention-span, modern-society-is-a-money-flavored-soul-torturing-device type of reasons, it seems like the hype around A.I. has died down a little bit. After a flurry of headlines, reporters who have a passing familiarity with the issue have sort of run out of things to say.
And on one hand I get it—A.I. has not changed the world yet. ChatGPT is super impressive, but it doesn’t literally do your job better than you do. It’s a tool in a toolbox at this point. Better than Google for lots of applications, but nowhere near as good as having a competent human personal assistant. It can help you code, but only if you know how to code at least some.
But.
I was listening to the 80k hours podcast the other day, and the guest was Mustafa Suleyman, one of the smartest A.I. people in the world. He cofounded Deepmind, which is now a Google subsidiary; he’s now building a “Personal Intelligence”, billed as the ultimate personal assistant, based on the latest version of the GPT large-language model, called GPT-4. (For reference, the model behind ChatGPT is ‘GPT-3.5’, and no I don’t know why they did half-number versions); he also has a new book out that I’m anxious to read. All to say, he’s a heavy hitter, a very serious person.
Here’s the quote that literally gave me a flush of adrenaline when I heard it:
“We’re going to be training models that are 1000 times as large as the current ones within three years.”
That’s *three orders of magnitude*, or *one order of magnitude PER YEAR*. Nothing grows that fast. It’s never happened in the history of technology. The closest analog is atomic bombs:
—The first atomic bombs had 15-20 kilotons of explosive power, which was ~10x the power of the largest conventional ordinance available at the time. That’s one order of magnitude. Even if you estimate the dates conservatively, the Manhattan Project lasted 3 years to achieve that order of magnitude improvement.
—Tsar Bomba, the largest bomb ever, was exploded in 1961 and delivered 50,000 kilotons of force. That’s three additional orders of magnitude. But it took 16 years! That’s ~.2 orders of magnitude per year.
—So: A.I. is scaling ~5x as fast as the atomic bomb, the greatest scientific feat in human history.
Also note: Unlike predictions about future A.I. capabilities—which are always speculative and basically always wrong—Suleyman’s prediction is merely about hardware deployment. They’re going to chain more processors and run them for more time on more data, and all these technologies are mature and plentiful. Nothing needs to be invented; they buy chips, hire engineers, and it’s straightforward from there.
This has a good chance to actually happen.
And unlike atomic bombs, which (thank God) require a huge infrastructure to support that only large governments can afford, once these models are trained, anyone will be able to run them on a laptop. The magic trick of A.I. is that running a model requires much less computing power than training the model, and running the model does not destroy the model.
*
What will three orders of magnitude look like in practice? The short answer is nobody knows. The longer answer is to find a benchmark in what we know and extrapolate while maintaining a lot of epistemic humility. So let’s do that.
GPT-4 is ~5x as large as GPT-3.5.
The result, according to OpenAI (who, unlike most companies, is not known to exaggerate the power of their own products), is a model that is ~10x as “advanced” as its predecessor. Specifically, that means things like:
—GPT-4 is “multi-modal”. Example: GPT-3.5 can read a list of ingredients in your refrigerator and give you recipe suggestions. GPT-4 can study a *picture* of your refrigerator, make a list of what it sees, and give you recipe suggestions.
—The working/short-term memory of GPT-4 is 8x that of GPT-3.5, 64k words vs 8k words, meaning GPT-4 won’t get confused about what you’re talking about for 8x as long.
—GPT-4 can speak 25+ non-English languages, compared to English-only for GPT-3.5. Note that nobody “programmed” these languages into GPT-4, the ability to speak them simply emerged from its study of the training data.
—GPT-4 is “steerable”, meaning there is an input you can make where you give it instructions on *the manner in which it answers your questions*. If you want it to answer like a pirate, or a literature professor, or a talking dog, it can do that. If you want lots of detail or one-word answers only, it can do that too.
These are the results from a 5x improvement in model size and complexity. Suleyman predicts a 1000x expansion *starting from that 5x place* in the next three years.
What performance improvements (the word “improvements” feels inadequate here) can we expect from that 1000x increase? Here we get into the realm of guessing. The truth is I have no idea. Almost anything seems possible. Here’s a few things I’d love to see:
—You tell GPT-5 (hopefully they will drop the half-number thing) that you need a web site for your business. In seconds it scrapes all data about you from the web and your phone, including photos and information about your business. Seconds later, it presents ten options for fully-developed, custom-designed, ready-to-publish web sites. You pick the one you like best and tell it in natural language any changes you’d like made (although it’s such a better designer than you that you probably won’t have notes). It makes those changes and publishes your web site. What used to cost thousands of dollars and take weeks or months now takes five minutes and costs $20/month (current price of a ChatGPT subscription) for as much as you can use.
—You need to get some work done on your home, say electrical work. GPT-5—with zero supervision from you—calls every electrician in the area and gets quotes from each of them (or, in time, from their bid-giving A.I.), interviews each of them to determine which is the most trustworthy, aggregates every review that’s ever been left about them, then recommends the best professional for the job, pays them from your bank account, schedules them to come when you’re not home, and monitors their progress on your home security cameras. The electrical work gets done at high speed and with quality, and the only thing you did was authorize GPT-5 to get it done.
—You want to buy a piece of undeveloped land and build multiple homes and a business on it. (This is my wife and my long-term plan.) You have a multi-county range where you’re willing to consider living. The problem is, each county has different zoning codes, processes for permitting of housing and commercial uses, processes for securing water rights and a million other things, and different tools for finding all the relevant information about any individual parcel. Instead of taking days and days of your life making phone calls to the relevant county authorities and trying to get a handle on it, GPT-5 synthesizes 100% of the relevant laws and codes, spends hours on the phone with various authorities (or, eventually, their A.I.s), learns how to use all the different online tools for each county, and builds you a front-end application where you can enter the address of any parcel and instantly have user-friendly information about the development hurdles and allowed uses of that parcel.
And honestly, I’m not really even being science-fiction-y enough in those examples. It could learn to simulate human physiology well enough to conduct clinical drug trials without using actual humans, giving an order-of-magnitude speed boost to drug development. It could figure out how to generate 10%+ economic growth and thus transform the political landscape of our entire society. It could tell us how to turn CO2 in the atmosphere into fairy dust and send probes to faraway stars on a 10,000 year timeline. Nothing is impossible at that kind of scaling speed.
There are certainly non-technological hurdles that can (and will) stand in the way of lots of this progress. That’s baked into any new technology. (Example: We’ve had elevator technology for a long time, allowing dwellings to be stacked on top of each other at arbitrary scales, yet we still have a housing shortage and homeless people. Why? Politics.) So there will be implementation problems. But a 3-order-of-magnitude increase in three years is enough to absorb a whole lot of knee-capping blows and still change a whole lot of things.
And of course, it only begs the question: In three years, what then? After Tsar Bomba, nobody ever exploded a more powerful bomb, despite the theoretical ability to build much, much larger ones. The current U.S. arsenal contains an absurd amount of kil-tonnage, but no single bomb is even 10% of Tsar Bomba’s size. The world saw what they were doing, and… stopped. It’s kind of a miracle, really.
How many more orders of magnitude are we going to push A.I., and how quickly?
*
Some quick hitters to close. Here’s a sobering interview with Eric Schmidt, former CEO of Google. The video is five months old, which means it’s basically out of date at this point, but it points at a whole lot of horrible consequences of super-powered A.I. that could happen in the areas of politics, terrorism, etc., and also how important international regulation is in preventing that outcome.
At this point, the “A.I. turns on us” case is clearly overhyped, the “A.I. accidentally turns us all into paper clips” case is given more attention than its likelihood merits, and the “A.I. acts as a force multiplier for the absolute worst people among us and they gain the power to destroy civilization” case is problematically underrated. I’m not sure I buy that “International Atomic Energy Agency, but for A.I.” is going to prevent the proliferation of this technology, given it’s portability and durability, but we have to try.
*
Here’s a short and interesting essay exploring some of the political economy problems with the coming revolution. The key takeaway is that the short and long run are very different, and disturbing a stable cultural or political equilibrium (And we live in one, despite Trump, etc) is always painful! The printing press (kind of) caused the Enlightenment centuries later and thus invented the modern world, but in the 50 years after its widespread adoption, the printing press fomented the English Civil Wars, replete with so much bloodshed that it caused Thomas Hobbes to write “Leviathan”, one of the most misanthropic and pessimistic pieces of philosophy ever produced!
*
From the “humans-are-still-pretty-cool” files, here’s a Twitter thread with 14 lesser-known paintings from famous masters. My favorites are a masterpiece Picasso painted when he was 15 (!), and a tribute to Van Gogh’s “Starry Night” painted by Edvard Munch of “The Scream” fame. They’re all worth a look!
END
Thanks as always for reading! Have a great week, and I will be back next Sunday with another original story.
Wow. Every one of your examples is more jaw dropping than the last. Here is another video - a documentary - to add to the ones you linked to. It is not A.I., specifically, but more the long-term implications of the internet: https://www.youtube.com/watch?v=uaaC57tcci0.