The sponsor of AI alarmed as advanced systems learning quickly to lie, to deceive, to sing and to hack

A key pioneer of artificial intelligence is concerned about the growing propensity of technology to lie and deceive – and he founded his own non -profit organism to slow down this behavior.

In a Blog points Advertisement Lawzero, the new non -profit company, “AI Sponsor“Yoshua Bengio said he had become” deeply concerned “while AI models become more and more powerful and misleading.

“This organization was created in response to evidence that today’s border AI models have dangerous increasing capacities and (behaviors)” the most cited computer scientist Written: “Including deception, cheating, lie, hacking, self-preservation and, more generally, the desalination of objectives.”

Of everyone, Bengio would know it. In 2018, the founder of Montreal Institute for Learning Algorithms (Mila) was has given a Turing price Alongside the pioneering colleagues of the Yann Lecun and Geoffrey Hinton for their training roles in research on automatic learning, and he was listed as one of the Time magazine “100 most influential people“In 2024, thanks to its oversized impact on always accelerated technology.

Despite the distinctions, Bengio has have expressed regrets several times on its role in the contribution of AI Advanced technology – and its Silicon Valley braking cycle – at the conclusion. The latter missive seems to be the most austere to date.

“I am deeply worried,” wrote the AI ​​pioneer in his blog article, “according to behaviors that unhappy agental AI systems are already starting to exhibit.”

Bengio underlined recent experiences in the red team, or tests that push AI models to their limits to see how they will act, showing that advanced systems have developed a strange trend to keep themselves “alive” by all the necessary means. Among its examples, there was a Recent anthropic report Retail how his model Claude 4, when he was told that he would be closed, threatened An engineer with incriminating emails if they followed.

“These incidents,” wrote the decorated researcher, “are signs of early alert of the types of involuntary and potentially dangerous strategies that AI can continue if it is not controlled.”

To put such behavior in check, Bengio said that his new non-profit organization builds a so-called “trustworthy” model, which he calls “Scientist IA”, which is “formed to understand, explain and predict, as a disinterested and Platonic scientist”.

“Instead of an actor trained to imitate or please people (including sociopaths), imagine an AI formed as a psychologist – more generally a scientist – who tries to understand us, including what can harm us,” he explained. “The psychologist can study a sociopath without acting as such.”

A pre-revived newspaper Bengio and his colleagues published earlier this year explain it a little more simply.

“This system is designed to explain the world of observations”, paper readings“As opposed to taking measures to imitate or please humans.”

The concept of construction of the AI ​​”in complete safety” is far from new, of course – that is literally why several OPENAI researchers have left OPENAI and anthropogenic As a rival research laboratory.

This one seems to be different because, unlike Anthropic, Openai or to any other company that pays the Lip Service to IA security while providing cash in cash, Bengio is a non -profit organism – although this has not prevented it from Raise $ 30 million Tastes of the former CEO of Google, Eric Schmidt, among others.

More on frightening AI: Advanced Openai Model caught the sabotage code intended to stop it