Prominent artificial intelligence researchers have warned about the possibility of human extinction and the inability to control superintelligence.
The probability of human extinction due to the emergence of artificial superintelligence is, in the view of some senior researchers, akin to a coin toss. Warnings that were until recently labeled as apocalyptic or pessimistic are now being voiced directly by the scientists who are personally building and upgrading the world's most advanced AI models.
The non-profit organization Palisade Research has published a series of video interviews with current and former researchers from major organizations such as OpenAI, Google, and Anthropic on a website called frominside.ai. Geoffrey Irving, a former researcher at OpenAI and Google DeepMind, estimates the chance of human destruction by superintelligent systems at about 50 percent and believes such a threat should not be underestimated.
Neel Nanda, a senior research scientist at Google DeepMind, puts the probability of the destruction of human civilization at at least 10 percent, describing such a probability as ridiculously and unacceptably high for an event of such catastrophic magnitude.
Discussions regarding the safety and containment of autonomous systems have always been a subject of debate, but recent interviews indicate that even at high technical levels, there is no comprehensive or reliable solution for containing next-generation models, as reported by TahaSite.
Daniel Kokotajlo, a former member of OpenAI, assesses the potential power of super-artificial intelligence as god-like. In part of his remarks, he emphasizes: "Unfortunately, we currently have absolutely no idea how to control such a system, and in all likelihood, no one will be able to contain it. The impending danger is just as dire as it sounds, and it must never be allowed to happen."
The researchers featured in these videos have answered numerous questions regarding their personal motivations, the hype surrounding the projects, and why they are working on such an incredibly dangerous system. Mary Phuong, a researcher at Google, notes with rare honesty: "I think you are completely right to be skeptical of what I say, because I draw a salary from this very company."
Neel Nanda, pointing to the ethical challenges of working in such centers, considers his motivation to be an effort to make the tools safe. Nanda believes that the resignation of concerned scientists will not hinder the speed of the development race: "If I didn't believe that my research was directly reducing the existential risks of these systems, I would resign; but companies will not stop building models just because I leave."
The recorded conversations show that the existential risks arising from the automation of thought and decision-making are no longer the subject of science fiction, but are considered an objective concern among the world's frontline technology engineers, as reported by TahaSite.
Comments