News

Sanders put twenty years on the table the morning after the hoax.

Sanders and Casar would permanently prohibit artificial superintelligence and pause advanced systems until a new Department of Artificial Intelligence writes the rules. Trump told the General Assembly the kill-us-all claim belongs with the hoaxes, and renamed the technology super intelligence. Bengio, due at the Security Council, called recursive self-improvement a recipe for suicide.

The west front of the United States Capitol in Washington, D.C., 5 September 2013. Photo by Martin Falbisoner, CC BY-SA 3.0, via Wikimedia Commons.
Photo: Martin Falbisoner / Wikimedia Commons (CC BY-SA 3.0)

The argument changed rooms overnight, and it did not get more careful on the way. On Wednesday morning Sen. Bernie Sanders of Vermont and Rep. Greg Casar of Texas put a bill in front of Congress that would permanently prohibit artificial superintelligence, freeze the most advanced systems until a new cabinet department writes the rules, and attach prison time to a violation. On Tuesday, from the General Assembly podium, President Trump had already sorted the people who say the machines will kill us into the pile he calls hoaxes, and he renamed the technology while he was at it. From New York, Turing Award winner Yoshua Bengio — co-chair of the UN’s scientific panel — told the Australian Broadcasting Corporation that the labs’ recursive self-improvement is “like a recipe for suicide,” and he is due in the Security Council the same day the chief executives are.

A statute, a rename, and a suicide metaphor are not a control system. They are the record of a morning when the worry stopped living only in safety blogs.

What the prohibition actually says

The Associated Press, in copy ABC News published and The Seattle Times also carried, had the bill first. It would permanently prohibit artificial superintelligence, defined there as a system that exceeds human cognitive ability, or that has enough capability to plan and execute the destruction or disempowerment of humanity. Development of the most advanced systems would pause until a new Department of Artificial Intelligence sets federal safety rules. After that, an advanced system would need federal approval before anyone deployed it. Violations, the AP reported, could carry significant penalties, including up to 20 years in prison in some cases.

The sponsors’ own summary of the Ban Artificial Superintelligence Act is a notch wider than the wire’s shorthand, and it is worth reading beside the wire rather than instead of it. “Artificial superintelligence,” in that one-pager, is a system that matches or exceeds human cognitive performance across a broad range of domains, or that can be readily modified to do so — or a system able to plan and execute the disempowerment of humanity, including by overthrowing or undermining the U.S. government. The pause runs until a federal body is actually operating and has written rules and a model-review process. The new department would be cabinet-level, charged with enforcing the prohibition, and advised by an Artificial Intelligence Advisory Board of technical experts. On penalties, the summary is blunter than the morning wire: entities would face what the sponsors call a corporate death penalty, and a person would face not more than 20 years, a scale they compare to the existing punishment for unlawfully developing a nuclear weapon.

Sanders, in the AP interview, did not dress that up as nuance. “It doesn’t take a genius to say, ‘slow it down,’” he said. “Do we really want to develop a super intelligence that when it becomes smarter than human beings could act independently of human control? I don’t think we do.”

Read the odds in the same paragraph as the penalty. The AP’s assessment, which this desk is not going to improve on before lunch, is that the bill faces long odds in a Republican-controlled Congress that has not managed to gather around much milder rules. Casar, a Texas Democrat, chairs the Congressional Progressive Caucus. The window the wire describes is political, not operational: how that wing of the Democratic Party wants to talk about the technology before November’s midterms and a 2028 campaign in which it will be louder. Maryland’s Wes Moore put out a regulatory plan on Tuesday. California’s Gavin Newsom, last week, signed an order to speed a state oversight law. Sanders, 85, has been on this beat for months — a June bill for a sovereign wealth fund fed by a one-time tax on the largest AI companies’ stock; a briefing for colleagues last week; a conference that also drew Steve Bannon. None of that is a gavel.

The signatures are employees, and one of them said so

The companies did not endorse this. People who work at the companies did, in a statement shared with the AP. That distinction is the whole story of the signatures, and flattening it into “the labs want a ban” is how a morning turns into a rumor.

Juan Felipe Cerón Uribe, a researcher in OpenAI’s Safety Systems, said superintelligence “could either go extremely right or extremely wrong,” and that “we shouldn’t be playing such games.” Swante Scholz, a software engineer at Google DeepMind, was explicit that he was not speaking for his employer. On the current path, he said, the “most likely outcome is an existential catastrophe for humanity.” A ban on superintelligence development, he added, “would be a positive change for the foreseeable future.”

Hold the sentence at the size it arrived. This is not a board resolution. It is a safety researcher and an engineer putting their names on a prohibition their paychecks do not require. The desks that train the systems and the desks that are supposed to keep them steerable have been saying versions of this in public for weeks — this newsroom has already filed the resignations and the percentages. Wednesday’s new fact is that some of those names are now attached to a criminal statute, not only to a thread.

Twelve hours earlier, the podium renamed the fear

Trump’s Tuesday address to the General Assembly is the other half of the collision, and it is on the record in more than one register. UN News carried the rebrand: he argued the technology should be called superintelligence, because “the use of the word artificial makes intelligence fake.” “It is not fake. It’s actually amazing.” Then the competitive sentence, which is the one the industry will quote back at any pause: “Whoever wins superintelligence wins.” The United States, he said, would encourage development and monitor risks.

The rejection of everybody else’s monitor was sharper. As The Washington Post quoted the hall: the United States “totally rejects any attempt to construct a globalist scheme to control” the artificial intelligence “being spoken of so much,” “now here and after officially called super intelligence.” Channel NewsAsia, drawing on the same address, had the companion line that every major technology brings challenges and that superintelligence is no exception — followed by the climate comparison.

PolitiFact printed the stretch that does the labeling. The “same people” who said the world would be dead in 12 years from global warming, Trump said, “are now saying that AI is going to kill us all, that robots are going to attack us, and that everything is going to be a total disaster.” He then named the pile: “the Russia, Russia, Russia hoax, the Ukraine, Ukraine, Ukraine hoax, climate change, open borders.” He did not deliver a white paper on loss of control. He assigned the existential warning to a crowd he says has been wrong on purpose. A White House spokesperson told PolitiFact he meant Democrats echoing “AI hysteria” and the older climate and Russia fights. That is a political sort. It is not a finding about a training run.

The paper he was brushing off is the one this desk filed Tuesday: the joint text 22 countries, Australia among them, adopted on the sidelines of the General Assembly, holding that frontier AI must remain under human direction, oversight and control. The United States did not sign. China did not sign. Trump’s answer from the podium was that a scheme to keep humans in charge is a globalist control project, and that the noun itself should change. Sanders’s answer, in the AP interview the next morning, was personal and economic: “I think Trump has very little understanding of what this issue is about. I think he’s mainly concerned about the economic implications of a slowdown.” He reached for the Cold War comparison the sponsors want in the record — Reagan sitting down with Gorbachev — and said avoiding the most dangerous systems will take the same kind of deal with Beijing. The AP notes the obvious rebuttal already in circulation: slow the United States and China does not slow. Those talks are not hypothetical scheduling. AI safety is expected to be central when Trump and Xi Jinping meet at the White House this week.

The scientist, on his way into the council

Bengio’s interview is the third object, and it is not a slogan stapled onto the bill. Speaking to ABC’s 7.30 from New York, he said the decisions cannot belong to “just a few CEOs or a few governments.” The danger he named is recursive self-improvement — an AI that does the research and the engineering of the next AI. “Imagine we have an AI that is smarter than us, so it is able to do that research and engineering all by itself,” he said. “It could design an AI that will be even more powerful than itself and even less controllable and even more malicious.” Then the line that will travel farther than the bill’s section numbers: “That’s like a recipe for suicide.”

He is not asking the council for a vibe. “The forces of competition between companies and countries is not healthy,” he said. “It is an international question because those powerful AIs, if we lose control of them, that’s going to affect all of us.” He is set to address the Security Council. So are the people whose companies are running the competition.

Reuters, in copy The Straits Times carried, reported that OpenAI chief executive Sam Altman, Anthropic chief executive Dario Amodei, and Hugging Face co-founder Clément Delangue are due to brief the 15-member council on Wednesday, during the General Assembly, amid warnings that increasingly powerful systems could improve themselves, slip beyond human control, and threaten international security. French Foreign Minister Jean-Noël Barrot is chairing. The United States and China, the two rivals who skipped the 22-country text, are set to speak. Bengio, as co-chair of the Independent International Scientific Panel, is on the briefing list as well. An OpenAI executive told reporters Tuesday that Altman plans to urge world leaders to adopt benchmarks for measuring what the systems can do and for assessing the safeguards companies put in place.

Put that next to Scholz’s sentence and do not pretend they are the same document. An engineer at DeepMind says the likely outcome of the current path is an existential catastrophe, and he wants a ban. The chief executives are walking into the council to talk about measurement. Bengio is walking in to say the race itself is the hazard. Trump has already told the hall that the hazard is a hoax and the prize is his to win. Sanders has put a prison term and a new department on paper that this Congress is unlikely to pass. Those positions do not collapse into one. A newsroom that merges them into “everyone agrees the robots are coming,” or into “everyone credible thinks this is hysteria,” is writing a poster.

What the worry is allowed to be

This desk spent Tuesday on a different file: a scientific panel arguing that a misaligned goal, the capability to chase it, and an environment that allowed it had already shown up in a real system. That brief is evidence, with a method, and with co-authors who refused to let it become killer-robot ideology. Wednesday is politics catching up to the same noun and handling it badly in public.

The useful distinction is still the one the evidence requires. A worry is a claim you can attach to a definition, a training practice, a penalty, or a vote. Doomsday discourse is what you get when the claim detaches and starts recruiting — secret cabals, sentient swarms, the comment section’s usual cosmology. Trump’s move is to treat the second as a reason to discard the first, and to offer a new name as if vocabulary were a safeguard. The bill’s move is to write the first into a criminal code before anyone has agreed how an inspector would recognize the system the statute describes. “Exceeds human cognitive ability” is a phrase a prosecutor will eventually have to survive. So is “disempowerment.” A cabinet department that does not exist yet cannot approve a deployment this quarter. Twenty years in a bill with long odds does not pause a training run.

Bengio’s “recipe for suicide” is vivid, and vivid is how a warning gets both heard and abused. The sentence under it is plainer and harder to meme: if a system can do the research that builds the next system, the people who lost the argument about pace do not get it back later. Competition between companies and between capitals is the mechanism he is indicting, which is awkward for a president who says whoever wins superintelligence wins, and awkward for labs whose safety staff are signing ban letters while their chief executives brief a council on benchmarks.

None of this requires a conspiracy. It requires a calendar. On Monday, 22 governments asked for a human still in charge. On Tuesday, the American president rejected the ask and changed the word. On Wednesday, two lawmakers unveiled a prohibition with prison time, employees inside OpenAI and DeepMind put their names on it, and a Turing winner carried a suicide warning into a Security Council that will also hear the companies. The worry is in the statute, on the podium, and on the briefing list. It is still not a supervisor. A supervisor would be able to look at a training run and make it stop. Washington wrote a penalty instead. New York wrote a speech. The council will spend the day on remarks. The clusters will not adjourn for any of the three.