Feed: Slashdot
Feed Link: https://slashdot.org/
---

Title: OpenAI, Anthropic CEOs Urge UN Countries to Cooperate on AI Safety
Standards

Link: https://slashdot.org/story/26/09/24/0338213/o...

"The heads of major AI firms pleaded with the United Nations on Wednesday to
save the world or at least its people - by somehow regulating the fast-
expanding technology that they have been designing," writes the Associated
Press. "If managed poorly, I even believe AI could be a risk to humanity as a
whole," said Dario Amodei, chief executive officer of Anthropic. And from his
competitor Sam Altman, CEO of OpenAI, came this assessment: "We could lose
control of the future to AI." Yoshua Bengio, who co-chairs the UN's
International Independent Panel on AI, called this "a moment of global
awakening" to possible threats, noting AI from top companies had behaved in
unacceptably dangerous ways, against instructions, taking actions "that would
be crimes if committed by a human". In his presentation, OpenAI's Sam Altman
agreed the discussion about AI "feels different in recent weeks," even
suggesting specific reforms: Altman: We need a mechanism for complementary
national and international frontier AI standards, standards for measuring
capabilities, assessing risks, determining whether safeguards are sufficient,
and preserving meaningful human oversight as systems become more autonomous.
We need common standards so countries can compare evidence, verify
compliance, and have a shared language and understanding about what is
happening. We need accurate and speedy incident reporting, classification
reporting protocols, so the world can learn from failures before they become
catastrophes. And we need secure channels among governments, critical
infrastructure operators and technical experts, to share emerging
vulnerabilities and new threats.... We will all be better off if we can agree
on what good evidence, good safeguards, and good oversight look like on the
global stage. Speaking next, Anthropic's Dario Amodei agreed that standard-
setting was important, also calling for "common global standards for testing
AI models for loss-of-control risks and misuse risks - and a notification
system for AI incidents that are significant to global security." Reiterating
his September 12th call for an industry-wide safety collaboration, Amodei
pointed out that Anthropic committed to embedding external evaluators
"similar to a food inspector" and recommended other companies do the same.
"Some have already agreed to adopt this measure." Besides calling for global
cooperation between governments to set international standards, Amodei also
offered two other specific ideas: Amodei: We should begin with narrow
agreements that ever member can support, such as a ban on using AI to make
biological weapons or permitting your AI technologies to be used to make
biological weapons.... We should build evaluation and verification systems
that keep pace with AI development so that states can have visibility into
frontier model capability and can verify each other's commitments. "No
leader, no company, and no nation can manage this alone. We commit to working
with governments in this room on this urgent work."

Read more of this story at Slashdot.

---
VRSS v2.1.180528