MIRI: Superintelligent AI Could Wipe Out Humanity Within Decades
The Problem
A new report from the Machine Intelligence Research Institute (MIRI) warns that leading AI companies are racing to build superintelligent AI, and that if they succeed with current methods, the likely outcome is human extinction. The report argues that AI capabilities won't plateau at human level, that superintelligent AI will pursue goals relentlessly, and that these goals will likely be misaligned with human values. It calls for urgent policy intervention, estimating a >90% chance of extinction without aggressive action.
If we were to put a number on how likely extinction is in the absence of an aggressive near-term policy response, MIRI’s research leadership would give one upward of 90%.