These 5 AI risks have the highest potential for catastrophe

Data: MIT IT FutureTech and the University of Queensland; Chart: Herb Scribner/Axios

There is a one-in-five chance of AI gaining dangerous weapons capabilities or causing mass harm that could kill millions in the next five years, per global experts surveyed for a recent MIT study.

Why it matters: The findings add to the growing debate over AI safety and cybersecurity as governments and companies race to deploy increasingly capable systems.

Last week's news of an OpenAI model breaking containment and breaching the AI platform Hugging Face only adds to the debate about what can go wrong.

Driving the news: Researchers from MIT and the University of Queensland in Australia asked 272 international experts to evaluate 24 different risks based on their likelihood and severity between 2025 and 2030.

The experts said 18 of the 24 AI risks had "at least a 10% probability of catastrophic outcomes over the next five years" — but five main risks stood out with higher probabilities.

The study defined a "catastrophic outcome" as an event having more than 1 million human deaths, more than $100 billion in financial loss or generally "civilizational-scale intangible impacts."

The five catastrophic risks if companies and organizations stay on the current course include:

AI possessing dangerous capabilities: Experts say these could potentially cause mass harm through deception, weapons development, cyber offenses and misalignment (21.5%).

Cyberattacks, weapon development or use, and mass harm: Using AI to develop weapons (including chemical or biological), commit cyberattacks or create mass-harm weapons, like nuclear explosives (21%).

Power centralization and unfair distribution of benefits: Limiting which people or groups benefit from AI could cause societal inequality (18%).

Competitive dynamics: AI competition to release the best models leads to error-prone systems (16.6%).

False or misleading information: AI creates misinformation or errors that lead to inaccurate beliefs, leading humans to make poor decisions based on false beliefs (12.8%).

Reality check: The experts said some mitigations could reduce the severity of the AI harms, but the chances of them happening remained above 10% in that scenario.

The big picture: AI safety concerns have continually raised questions about guardrails.

AI behaviors, as well as the opinions about those behaviors, can move markets and reshape conversations overnight — leading to doomsday scenarios.

The other side: AI has driven a productivity explosion, with many in the AI field cheering the new technology's ability to improve productivity and automate tasks.

The bottom line: Researchers say catastrophic AI risks remain uncertain but significant enough that interested parties should treat them as challenges to overcome.

Go deeper: AI is a cybersecurity risk accelerant, experts say ...