A chief AI scientist has warned that superintelligence could kill the population. Geoffrey Irving, the former chief scientist at the Government-run AI Security Institute, believes that AI systems can, in principle, align with human intentions, but he fears we are moving too fast.
The expert previously warned that AI capabilities are advancing much faster than our ability to ensure safety, creating a serious risk of losing control. The AI-safety researcher warned on X today: “I think we have a ~50% chance of all dying as a result of superintelligence, mostly due to actions in the next few to 10 years. I don’t expect to have that resolved to below 10% or above 90% before we either make it through, or we don’t. We will have to act despite uncertainty.”
When asked how he arrived at that percentage, the scientist explained that he had settled on a range of 10-90% for years and now takes the average as the “honest” answer.
He responded: “As to why 50%, despite thinking about this for years I still think there is a ton of model uncertainty of a variety of types, and a few years ago converged on ’10-90%, but I don’t feel calibrated within that range’. Then I realized that the honest move was just to take the mean.”
Having previously worked at DeepMind, OpenAI and Google Brain, Irving explained his move to the Government-run AISI by stating “I worry that progress on safety is too slow relative to the rate of advancement in AI”.
He added: “Most employees and leaders at AI labs mean well, but market forces mean that there is constant pressure to prioritise speed over caution.”
If you take writing as an example, models went from producing fairly simple text, to writing essays, coding, translating, and holding coherent conversations.
This story originally appeared on Express.co.uk
