1,122 AI Experts Urge Global Coordination to Pace Frontier AI Development
Pacing the frontier
We are witnessing AI research accelerate toward automation, creating a risk where capabilities outpace our ability to control them. While competitive pressures drive individual companies to rush, we believe the world needs coordinated tools to deliberately slow this progress. By requesting government support for international pacing mechanisms, we aim to buy time for safety measures and governance before an intelligence explosion occurs.
The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction.
- alach11
The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive.
I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest.
If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic risks, surely a joint petition like this is a good thing? We should want the AI Labs to make binding agreements to proceed safely and slower, rather than being stuck in a suboptimal game theoretical race to advance unsafely. We should be hoping that nations can make binding agreements. Our discourse here should be about the substance of what these agreements could be and the practicality of enforcement mechanisms.
- biffles
It's an admirable idea, but I don't see how you can slow this down in practice. We certainly know China will not, and they are already at the frontier.
RSI is too important and too tangible at this point for everyone to slow down. It's the equivalent of telling the world to stop developing nuclear weapons after Hiroshima. Just not going to happen, and in the meantime dramatically increases the risk that the power of AGI lands in the hands of the few (read: the political class) rather than empowers the people.
- dzink
The historic analog most befitting AI are weapons. It provides an advantage to companies and governments in creating weapons, in surveilling people and thus gaining leverage over individuals, in controlling what individuals can do or cannot stop doing. It thus follows the laws of weapons control - the more restrictions are threatened to be imposed, the more even the smallest actors will fight to get more of it. They may not understand or know what they are getting, but they will want more of it because their potential rivals have it. Governments are the ultimate customer here - they have the money, the energy, and the appetite to consume AI infinitely in battlement against opponents - foreign and domestic. The only approximate solution is to prevent any restrictions on the use of it - a-la-2nd amendment (the upside of that is it advances the AI for future use and is thus in the interest of governments as well). Modern large models seem to be compressible and thus we may see more and more tiny sub-specialized varieties. But the push to slow the frontier in weaponry always leads to the reverse.
- fidotron
Anyone serious about this should participate in pacing the frontier by quitting their job so as to not potentially contribute to the problematic progress. Be the change you want to see!
Curious how that isn't what happens.
- girfan
Can someone help clarify what it means, concretely, to make binding agreements to proceed "safely and slower" in the context of AI research (whether or not at frontier labs)? Are there specific policies and actionable guides being proposed?
- vessenes
This. Will. Never. Work.
This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow.
The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe they can accomplish. They CANNOT accomplish this, because the fundamental economics and possible outcomes make it mandatory for any strategic adversary to work as hard as possible to defect.
The proposals here imagine a global panopticon surveillance state overseeing all research and chips as a 'reasonable' solution. EVEN IN THAT (terrible) state for the world, we will have effective defection.
Like it or not, the world is on this train, and the only sane thing to do is to fully support high quality ethical research groups and firms as they get up the learning curve of this technology.
Variants of this idea in AI 2030 included burning some of the US lead to get some nebulous safety metrics improved - also insane. Any slowing allows strategic adversaries time to catch up with very different alignment goals.
Do I like this situation? No. Are the people signing these things just fully in denial about the game theory and the stakes and the behaviors of strategic adversaries? Yes, yes they are.
- tcdent
Something people don't seem to see, and not that it's placed directly in front of you, is that pacing the release and adoption of AI is already happening.
Cost-per-token determines the economic viability of tasks that AI gets applied to. How many tokens would it take to automate all of the responsibilities of a gas station clerk? Probably more than their hourly wage. Job safe.
A slightly larger stretch, that most people attribute to profit margins (labs do not have profit margins, they have unlimited invested capital and no expectation of short-term returns), is the flat-rate billing plans of most code products. Why doesn't Anthropic have a 50x or 100x Max plan? Not that it's impossible, that it releases tokens into the workforce at a greater rate (note that their are hour, day, week limits across most of these services as well).
Hypothetical, but reasoned, and I do not expect most agree with me. If compute as a variable is inferred, this supports the incorporation a deliberate (redundant) one as well.
- yks
> We request that the U.S. government ...
Too bad these researcher's organizations spent their efforts supporting the dumbest, ficklest possible version of the U.S. government to guide us through the gravity of this moment.