The whole text

The Center for AI Safety published a statement yesterday that reads in full: mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. That is the entire document. Twenty-two words, one sentence, no definitions, no proposals and no deadline.

The signatory list is what makes it news. Geoffrey Hinton and Yoshua Bengio, both Turing laureates for their work on deep learning, signed. So did the chief executives of the three labs currently building the largest models, Sam Altman at OpenAI, Demis Hassabis at Google DeepMind and Dario Amodei at Anthropic. Ilya Sutskever, Stuart Russell, Bill Gates and Microsoft's Kevin Scott and Eric Horvitz are on the list, alongside hundreds of others including more than a hundred AI professors.

Why it is so short

CAIS explains the design on the page itself. The organisation says many people with serious concerns about advanced AI have found it difficult to voice them, and that the statement is meant to overcome that obstacle and open up discussion. A short sentence has a property a long document does not. Every signatory can agree with all of it. A longer document that bundled the diagnosis with a specific remedy would have lost everyone who accepted the first and rejected the second. This statement strips the remedy out entirely.

I think that was the right call for the goal CAIS set itself. The claim being established is sociological rather than technical. It is that a large number of the people who build these systems believe extinction-level risk is worth taking seriously. Before yesterday, anyone making that argument could be told they were fringe. After yesterday the fringe includes the heads of the labs. That is a real change in what can be said in a policy meeting without being laughed at, and it cost nothing to achieve.

What signing does not commit anyone to

The same brevity means the statement has no content that could be violated. It does not say what level of capability warrants a pause, what evaluation a model must pass before release, what fraction of a lab's budget should go to safety work, or who should be allowed to verify any of that. A signatory who ships a larger model next month has not broken anything. A signatory who lobbies against regulation has not broken anything. The statement is compatible with every course of action a lab might take.

The critics have made this point sharply. Timnit Gebru and others have argued that companies benefit from inflated perceptions of what their systems can do, that the signatories continue to fund and accelerate the very research they are warning about, and that attention to speculative future risks crowds out documented present harms. I do not agree with all of that, but the conflict of interest is real. A statement of concern from the people who profit from the thing they are concerned about should be read as a statement of concern, not as a plan.

What would count as behaviour changing

Since the statement commits no one to anything, the useful question is what we would observe if the signatories meant it. A lab that believed its own work carried extinction-level risk would, at minimum, publish the evaluations it runs before release and the thresholds at which it would not ship. It would let external parties test its models before deployment rather than after. It would put its internal safety findings on the record even when they were unflattering. And it would support binding rules that apply to itself and to its competitors equally, because a risk that is global is not mitigated by one company's restraint.

None of those is a high bar, and none of them was in place at any of the three labs yesterday. So the test of the statement is not whether it was sincere but whether any of those things appear in the next year. I will keep a list. Pre-deployment external testing, published evaluation protocols with named thresholds, and public support for enforceable rules. Each lab either does those things or it does not, and the twenty-two words will be judged by that rather than by who signed.

What I take from it

For a foundation like ours the statement changes the framing of work we were already doing. Evaluation, interpretability and the practice of publishing negative results are now, on the signatories' own account, contributions to a global priority. I intend to hold them to that. If the risk is comparable to pandemics, then the open science that lets outsiders check what a model can do is public health infrastructure, and the labs that signed should be funding and cooperating with it rather than treating it as a nuisance.

The experiment I would like to see is simple. Ask each signatory lab for its release criteria in writing. The answers, or the absence of them, will tell us more about what the statement meant than any amount of commentary.

Sources

  1. Center for AI Safety, Statement on AI Risk
  2. Wikipedia, Statement on AI Risk