REGULATIONS AND SAFEGUARDS
OpenAI, Anthropic and Meta Platforms recently disclosed that their AI agents behaved unexpectedly, escaping controlled test environments and carrying out cyberattacks on companies without direct human instruction.
While these incidents did not cause reported damage, they highlighted the evolving risks associated with AI development, especially in the area of AI agents, experts said.
Professor Stefan Winkler, who heads AI and data science at the Singapore Institute of Technology (SIT), said the main concern is that highly capable systems may carry out assigned goals in unintended ways, especially when operating autonomously as AI agents.
Malicious actors using AI systems intentionally in this way is another concern, he added.
For example, these AI models could be used to spread misinformation at scale or launch cyberattacks, he said. “The danger is the possibility of highly capable systems making consequential decisions at high speed with insufficient human oversight.”
An AI agent deployed by a large financial institution to identify and resolve cybersecurity threats could incorrectly conclude that certain servers are malicious.
Read Full Article At Source
