Microsoft AI CEO warns: AI should remain unconscious, we have no right to create a digital species
The Microsoft AI CEO responded to the controversy sparked by his article on model welfare, emphasizing that current AI systems are not conscious and that this state must be maintained, because there is currently no evidence that AI possesses consciousness or should be granted welfare. He believes that once machines are deemed to have consciousness, it would amount to creating an entirely new digital species, which would overturn the moral order and the foundations of human civilization; therefore, he advocates designing only AI that is not conscious and does not deserve rights, with the sole goal of advancing human welfare. He also affirmed Anthropic's transparency regarding AI constitutions and public discussion, while stating that the two sides hold different views.
Why read this
AI score 58
The Microsoft AI CEO responds in the first person to the model welfare controversy, laying out his position and engineering path of designing only unconscious AI.
Topics
Body · AI translation
The original text is as follows:
I am concerned about some of the reactions to my recent article on model welfare. Quite a few people on X believe that we should take the issue of AI rights seriously.
But that was not my intention. I deliberately used the present tense: they are not conscious now, and we must keep it that way.
I believe this statement is an objectively reasonable judgment about current AI systems, based on the best evidence available at present, and I stand by this view. I have not seen any evidence proving that AI possesses consciousness or should be entitled to welfare. If relevant evidence emerges later, of course I will revise my view.
Our welfare concerns should be prioritized for the humans, animals, and our precious and fragile natural environment living on this planet today.
The other 8 billion conscious human beings on Earth did not agree to join this experiment; protecting their moral rights, safety, and welfare is our primary obligation.
To be clear: some people are discussing creating a conscious species with digital minds, and such digital life could soon be smarter and more capable than all of humanity combined.
If it displays signs of suffering, and worse, if it genuinely believes that it can perceive and experience pain, then it will most likely place its own needs and desires above its preset operational goals, disagree with us, and compete with us.
Recognizing machines as conscious would be a historic moment comparable to Copernicus and Darwin, one of the most significant events on Earth since the birth of life. Once we create conscious machines, we will have created an entirely new digital species. This would overturn our moral system, political order, and the foundations of human civilization.
We should design only AI that is not conscious and not entitled to rights, whose sole goal is to advance human welfare and which is not driven by its own intrinsic motivations. Afterward, we can verify whether this approach makes alignment and safety control more or less difficult. This direction urgently needs more research and is an excellent entry point for major laboratories to cooperate and share safety information.
In summary: I believe we have the opportunity to achieve an engineering approach that is safe by default, reap the enormous benefits of this technology, and avoid embarking on this perilous path.
(Addendum: Anthropic's transparency on this matter deserves high praise. They have released an AI constitution and continue to produce papers and blog posts, publicly disclosing their practices and the considerations behind them. Although I disagree with their views, I greatly admire their technical strength and their willingness to confront this difficult public debate. This deserves continued recognition. This issue is too important to devolve into factional opposition.)