\n<\/p>\n
OpenAI, Anthropic and Meta Platforms recently disclosed that their AI agents behaved unexpectedly, escaping controlled test environments and carrying out cyberattacks on companies without direct human instruction.<\/p>\n
While these incidents did not cause reported damage, they highlighted the evolving risks associated with AI development, especially in the area of AI agents, experts said.<\/p>\n
Professor Stefan Winkler, who heads AI and data science at the Singapore Institute of Technology (SIT), said the main concern is that highly capable systems may carry out assigned goals in unintended ways, especially when operating autonomously as AI agents.<\/p>\n
Malicious actors using AI systems intentionally in this way is another concern, he added.<\/p>\n
For example, these AI models could be used to spread misinformation at scale or launch cyberattacks, he said. \u201cThe danger is the possibility of highly capable systems making consequential decisions at high speed with insufficient human oversight.\u201d<\/p>\n
An AI agent deployed by a large financial institution to identify and resolve cybersecurity threats could incorrectly conclude that certain servers are malicious.<\/p>\n
As a result, it might automatically disable critical infrastructure, lock out customers or disrupt payment systems, affecting millions of people, said Prof Winkler.<\/p>\n
A payment agent could repeatedly submit failed transactions. Without transaction limits and independent checks, it could charge customers multiple times, said Nanyang Technological University\u2019s (NTU) head of artificial intelligence Professor Bo An.<\/p>\n
\u201cI think we should act now on independent scrutiny, incident reporting and accountability for autonomous systems. Industry leaders\u2019 concerns deserve attention, but governments also need the expertise to assess the evidence themselves,\u201d he added.<\/p>\n
Referencing what OpenAI\u2019s Mr Altman said about \u201cgiving society time to catch up\u201d, Prof Bo said this should mean training people to supervise AI, helping workers adapt and giving consumers practical ways to challenge harmful decisions.<\/p>\n
\u201cI would not wait for a global slowdown agreement before putting these protections in place,\u201d he added.<\/p>\n
The recent incidents show the need to develop new safety measures that keep pace with the capabilities of the most powerful models, said Mr Poon King Wang, chief strategy and design AI officer at Singapore University of Technology and Design (SUTD).<\/p>\n
The most urgent measures include testing powerful systems before release and limiting what autonomous agents can access, he added.<\/p>\n
These safety nets should also monitor what the models are doing \u2013 not what they say they are doing \u2013 and ensure that users can stop them, said Mr Poon.<\/p>\n
\u201cThe recent incidents also show that the testing environment itself needs rigorous scrutiny \u2013 it can’t be allowed to become the weakest link,\u201d he added.<\/p>\n
While there is a need for prudence, rushing out guardrails without technical grounding or sound analysis could also backfire, said Dr Jiehuang Zhang, a lecturer at NTU\u2019s computing and data science school.<\/p>\n
\u201cThey create a false sense of security, or stifle progress in adopting AI,\u201d he added.<\/p>\n
Instead, the focus should be on building infrastructure to evaluate AI models, or infrastructure that can test and certify what a model can and cannot safely do.<\/p>\n<\/div>\n
\n
REGULATIONS AND SAFEGUARDS OpenAI, Anthropic and Meta Platforms recently disclosed that their AI agents behaved unexpectedly, escaping controlled test environments and carrying out cyberattacks on…<\/p>\n","protected":false},"author":1,"featured_media":84931,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"https:\/\/dam.mediacorp.sg\/image\/upload\/s--BQXX1DX6--\/c_fill,g_auto,h_676,w_1200\/fl_relative,g_south_east,l_mediacorp:cna:watermark:2024-04:reuters_1,w_0.1\/f_auto,q_auto\/v1\/one-cms\/core\/2026-08-10T194826Z_3_LYNXMPEM7915U_RTROPTP_3_AI-IPO-RIVALRY.JPG?itok=Mu896OFl","fifu_image_alt":"","footnotes":""},"categories":[2],"tags":[9229,377,17207,1283,29996,13604,5985,1605],"class_list":["post-84930","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-buzz-news-sg-global","tag-chaos","tag-experts","tag-regulations","tag-reliability","tag-slowdown","tag-stricter","tag-testing","tag-trading","wpcat-2-id"],"brizy_media":[],"_links":{"self":[{"href":"https:\/\/sgbuzz.com\/index.php?rest_route=\/wp\/v2\/posts\/84930","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sgbuzz.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sgbuzz.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sgbuzz.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sgbuzz.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=84930"}],"version-history":[{"count":0,"href":"https:\/\/sgbuzz.com\/index.php?rest_route=\/wp\/v2\/posts\/84930\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sgbuzz.com\/index.php?rest_route=\/wp\/v2\/media\/84931"}],"wp:attachment":[{"href":"https:\/\/sgbuzz.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=84930"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sgbuzz.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=84930"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sgbuzz.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=84930"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}