King Charles III has warned leading technology executives that artificial intelligence could pose “existential dangers” if powerful systems fall into the wrong hands.
The King delivered the warning on Thursday at an AI summit held at Dumfries House in Cumnock, East Ayrshire, Scotland. The gathering brought together representatives from Nvidia, Google DeepMind, OpenAI and Anthropic, alongside government officials and other specialists.
Charles acknowledged that AI was already showing the ability to improve and save lives, particularly in medicine and the life sciences. However, he said some of the people developing the technology were increasingly warning that it could acquire darker capabilities.
“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” he said.
“Surely, we need sufficient means of control before it is all too late?”
The King’s complete remarks were published by the Royal Family.
Safety must remain central
The King asked participants to consider how the benefits of AI could be developed with safety at the centre of the process.
He also called for stronger international cooperation so that countries could contribute to and benefit from the technology without losing control of its direction.
Charles said the decisions being made during the present stage of AI development would shape the world inherited by future generations.
He told the gathering that its responsibility was not simply to develop more capable technology, but to ensure that it remained in the service of humanity, communities and the natural world.
“As we embrace what is new, we must not lose what is timeless,” he said.
He added that human qualities and moral judgement must remain central as machines become more advanced, declaring that “our humanity must remain sacred”.
Leading technology companies attend
The summit included senior figures from some of the world’s most influential AI and technology companies.
Among those attending were Nvidia chief executive Jensen Huang, Google DeepMind chief executive Demis Hassabis and OpenAI chief financial officer Sarah Friar. Anthropic was also represented at the meeting.
The UK’s minister for artificial intelligence, Kanishka Narayan, attended alongside representatives from the King’s Foundation, the King’s Trust and the Sustainable Markets Initiative.
Figures from civil society and international organisations were also present, including Vatican AI adviser Paolo Benanti and historian Yuval Noah Harari.
Discussions were facilitated by the Ditchley Foundation and examined how AI could be used to benefit society while protecting human dignity and limiting serious risks.
Shared principles under consideration
According to the Royal Household, delegates considered whether a common set of principles could guide the future development and use of artificial intelligence.
Such principles would seek to ensure that AI is treated not only as a tool for greater efficiency and capability, but also as technology that should support people and the planet.
However, no binding regulatory agreement was announced following the meeting. The summit was presented as part of a wider effort to establish common ground among technology companies, policymakers and civil society.
That distinction is important: the King called for safeguards and international cooperation, but he did not announce a new law or regulatory regime.
Different views within the industry
Technology leaders continue to disagree about whether AI development should slow while stronger safety systems are established.
Some researchers and executives have warned that companies may be moving too quickly towards increasingly autonomous systems without fully understanding how those systems behave.
Others argue that development can continue if companies conduct proper testing and refuse to release products that do not meet adequate safety standards.
Huang said unsafe or unfinished products should be held back while further engineering and testing take place. He nevertheless cautioned against treating all AI development as inherently dangerous.
The debate extends beyond individual companies. Governments are also considering whether voluntary commitments provide enough protection or whether enforceable rules will be required as AI systems become more capable.
Charles did not endorse a particular regulatory model. His message focused instead on the need for control, international cooperation and the protection of human values.
The summit’s practical impact will depend on whether the participants turn its discussions into clear principles and measurable safety commitments.
