Mark Zuckerberg has made outrageous offers to join Meta’s new “superintelligence” lab to artificial intelligence specialists from competing companies in recent weeks. Wondering what superintelligence is and why a social media guru would offer to pay a scientist hundreds of millions of dollars are valid questions. Insiders claim that the stakes are as high as humanity’s future. Ninja geeks are a logical choice at such stakes.
The narrative starts, as it always does, before contemporary times. As Christopher Summerfield points out in “These Strange New Minds: How AI Learned to Talk and What It Means,” the ancients regarded gods as the source of knowledge. The Greeks, in turn, disputed whether wisdom sprang from reason (Plato) or experience (Aristotle). That dispute was revived around two millennia later in the realm of computer science, when many early AI systems adhered to the logic meticulously instilled by experts. However, the world, being what it is, obstinately disregarded this reasoning, creating exceptions to every norm. With the advent of machine learning, experience-based methods progressively gained traction. Developers discovered that if you give a machine-learning system sufficient data, it will learn to recognize a cat or play chess, for example.
Or use language. The software engines that drive chatbots like ChatGPT, Claude, Gemini, and Grok are large language models (LLMs), which are the focus of contemporary AI frenzy. “The linguist Noam Chomsky has long maintained that humans rely on innate wiring for grammar,” Mr. Summerfield says. As it turns out, LLMs function quite well without it—though they need a lot more training than a child.
As a more sophisticated version of autocomplete, LLMs learn by first digesting large amounts of text and then training to anticipate the subsequent piece. They are then educated to mimic more carefully chosen facts, obey commands, or win over human or artificial intelligence evaluators. They have recently transcended language to become semiautonomous “agents” that can process and create multimedia and employ software tools.
In his friendly, humorous style, Mr. Summerfield, a cognitive neuroscientist at the University of Oxford and a staff research scientist at Google DeepMind, occasionally throws a jab. Others target critics who assert that LLMs don’t truly comprehend anything or that LLM progress is at a standstill. These critics draw attention to the models’ glaring flaws, such as inability to solve basic math issues, or point out that LLMs only think in terms of word frequencies and have no prior experience interacting with the real world.
Even specialists are still surprised by AI’s advancements, though, since models effectively perform jobs that many sane observers would assume call for logic and common sense. For example, an LLM told Mr. Summerfield that there are no plants or other types of life on the moon, which is why sunflowers cannot bloom there. The atmosphere, water, and constant temperature needed to sustain Earth-like creatures are absent on the moon.
The ambition of automating knowledge appears to be within reach at what the author refers to as “a watershed moment for humanity.” While some researchers have disparagingly referred to LLMs as parrots, the author’s preferred avian comparison is “If something swims like a duck and quacks like a duck, then we should assume that it probably is a duck.”
Mr. Summerfield admits AI’s significant distinctions from humans. LLMs do not have bodies, social ties, or (presumably) awareness. They can’t comprehend nonverbal clues or understand conversational context. Ludwig Wittgenstein viewed language as a succession of games, each with its own set of rules and objectives: to inform, hypothesize, and amuse. LLMs sometimes play the wrong game, inventing instead than informing.
Some of Mr. Summerfield’s barbs are directed at proponents of AI acceleration who downplay the numerous risks that might arise in the future. AIs can already create, disseminate misinformation, violate privacy, entice users, operate lethal drones, and motivate individuals to commit suicide, as the author explains. They may eventually force humanity to extinction or seize control of governments and companies (or tempt us to cede control). If disaster occurs, Mr. Summerfield believes it will be caused by a swarm of smaller agents rather than a single superintelligent one, which might result in flash crashes or other types of chaos. He says that organizations, rather than individuals, have nearly always had the power to alter the course of history. It is assumed that a comparable communal effort will be necessary to prevent disaster. Therefore, we have room for improvement.
Some people believe that replacing humans is not just inevitable but even welcome, such as renowned computer scientist Richard Sutton. Mr. Sutton has considered AI’s capacity for autonomous skill acquisition. Building AI on “how we think we think” hasn’t worked out in the long term, he wrote in 2019. AI has advanced instead by expanding the computing and emphasizing learning over preprogrammed logic. A reproduction of Mr. Sutton’s essay, “The Bitter Lesson,” may be found in the appendix of Dwarkesh Patel’s book, “The Scaling Era,” which he co-wrote with Gavin Leech.
The majority of Mr. Patel’s book is made up of excerpts from interviews he did on his own “Dwarkesh Podcast” with AI experts, including nineteen men and one women, including Mr. Zuckerberg. When reading “The Scaling Era,” three things in particular jump out. First, the key to enhancing AI, and LLMs in particular, is straightforward: scale. But simple does not imply easy. There are technological difficulties with training larger models on more data for longer periods of time. Google DeepMind co-founder and CEO Demis Hassabis says that every time you expand a model’s size tenfold, “you have to adjust the recipe”—that is, the “hyperparameters” like how much the model should learn from each training example—and that’s a bit of an art form. It also brings up issues with resources. More data, power, money, and computer chips are needed for better models. The prospect of data centers that require specialized nuclear reactors and cost trillions of dollars is discussed by Mr. Patel’s interviewees.
Artificial general intelligence is what a large portion of the industry is aiming toward. Although there are many different definitions of artificial general intelligence (AGI), it usually refers to software that might do a wide range of jobs just as well as humans, like managing a company or planning and carrying out scientific research. Presumably, AGI would then advance and become superintelligent very rapidly. AI is already being used to create new devices and algorithms, even in the absence of AGI.
The second thing that comes out of the book is that, even if scaling were straightforward, complicated things would still develop. In an analysis of interpretability (the goal of making AI comprehensible) and alignment (the task of making AI ethical), Dario Amodei, the CEO and co-founder of Anthropic (the company that created Claude), provides a shrewd viewpoint: “We truly don’t know a thing about what we’re discussing.” And those are the models of today. AI that is superintelligent would be more cryptic and able to create algorithms that are beyond human comprehension. “You don’t know if it’s hacking, exfiltrating itself, or trying to go for the nukes in those millions of lines of code,” says AI researcher Leopold Aschenbrenner.
Third, the issues swiftly go from engineering to geopolitics when you take into account AI’s capabilities, complexity, and resource consumption. Foreign governments may steal the hardware or replicate the trained models if you locate your data centers in the Middle East, Mr. Aschenbrenner points out. According to him, attacks can even target data centers in the United States. He also envisions a nation with a little advantage in AI creating tiny drones that destroy an enemy’s nuclear submarines.
“The Scaling Era” is ungainly in both its form and its content. There are endnotes, footnotes, and sidebar definitions, which create a staccato rhythm. The target audience is uncertain. Technical intricacies are discussed with simple words like “learning.” If you intend to read both volumes and are not an expert, start with Mr. Summerfield’s. Nonetheless, Mr. Patel is a sharp and knowledgeable interviewer who pushes his sources to the brink of what they know or can say publicly.
Several contributors provide estimates for the arrival of AGI, with a cluster of suggestions pointing to approximately 2028. I would have liked to learn more about how these individuals and Mr. Patel define and measure AGI. Otherwise, a date is worthless. Another nitpick: Both Mr. Patel and Mr. Summerfield depict our squishy brains using digital measures such as bits and operations per second, which are irrelevant and give a misleading sense of comparison. Fortunately, both writers explain how AI can be both supersmart and superstupid when compared to humans. Intelligence encompasses several aspects; IQ alone is not sufficient.
We previously looked to the gods for knowledge. We are now applying our expertise to create something approximating gods. It’s difficult to say whether we’ll create a monotheistic one—a single server farm to govern us all—or a slew of infighting demideities, each magnificent and imperfect in their own way. All Mr. Zuckerberg understands is that a high priesthood is priceless.
Mr. Hutson wrote the book “The 7 Laws of Magical Thinking: How Irrational Beliefs Keep Us Happy, Healthy, and Sane.” He is writing a book about intelligence.
