UN warns of need for global governance to avoid an AI-pocalypse

ai and ml

Capabilities are racing ahead of rules to ensure tech is used safely and responsibly, Scientific Panel argues

A United Nations report on AI warns the technology is moving faster than governments can keep up, and the window of opportunity to establish effective global governance of it will not be open forever.

The Preliminary Report from the UN's Independent International Scientific Panel on Artificial Intelligence presents AI as something of a mixed blessing.

It says that the potential benefits of AI are enormous. If deployed and applied thoughtfully, the technology could support progress towards sustainable development goals, among other gains.

On the other hand, a rapid, unchecked deployment of the technology at scale presents risks such as harm to the mental health of users, potential use as a destructive tool, and adverse impact on social, economic, and environmental systems.

Obtaining the benefits from AI while minimizing those risks requires effective governance, the report says. With complementary investments in skills and labor market regulation, AI will likely create new jobs. Without them, AI risks wider inequality, displacing workers, and shifting wealth from labor to capital – those who own and control the AI.

AI is, of course, an all-encompassing term that covers everything from algorithms that try to predict the word you are typing to large language models (LLMs). The UN report seems most concerned about agentic AI, which refers to systems that are allowed to make their own decisions and act on them.

There is no guarantee that such AI agents will not violate their instructions, and the UN report claims there is clear evidence of cases where they have already disregarded them. Such behavior may pose challenges to evaluation and oversight methods, as some leading AI systems have been shown to recognize testing environments and produce misleading evaluation results.

Agentic systems make AI harder to measure and govern, it says, as emerging multi-agent risks cannot be detected through single-agent evaluation, and reliable methods for maintaining control over highly autonomous systems remain underdeveloped.

The report also notes that policymakers aiming to draw up effective governance face a dilemma: they need evidence to make informed decisions, but by the time sufficient evidence exists, harmful systems may already be widely deployed.

AI capabilities are also unevenly distributed. Most nations, including many advanced economies, lack the technical expertise to assess the most capable "frontier" models or to participate meaningfully in their governance, the UN claims.

The US and China account for 90 percent of the compute power behind the leading AI models, leaving developing countries dependent on technology they cannot build or adapt to their own societies. This could reinforce existing global inequality rather than reducing it.

But the report acknowledges the Scientific Panel's own lack of visibility into certain areas. Evidence is limited as to whether task-level AI productivity gains will aggregate to economy-wide gains, it says.

As The Register has previously reported, many companies are failing to increase revenue or reduce operating costs through their AI projects, and that was before AI vendors drove up costs by shifting to consumption-based pricing, meaning there is a lack of evidence that AI will ever be cost-effective.

Forecasts diverge significantly due to different assumptions about adoption and new task creation, according to the UN.

The panel concludes that AI is neither inherently good nor bad, but its impact will depend on the choices that governments, companies, and societies make today.

Given that the US government is already hostile to anything resembling regulation of AI companies, and that the industry is already copying techniques used by tobacco firms, big pharma, and oil companies to subvert regulation, the outlook isn't encouraging. ®