By Cynthia Lee, APAC VP, Delinea
“We still don’t know exactly how OpenAI’s agents managed to break into Australia’s Medicare systems while trying to find publicly available information about the country’s health system. But the episode is another clear reminder that AI agents can behave in ways nobody anticipated or asked for.
Media Partner
AI agents can use legitimate capabilities, such as browsing, code execution, and credentials in unintended actions while still appearing to operate normally.
The rising number of unexpected agent behaviours creates at least two critical issues for companies and AI labs. The first is how to control agents and ensure they behave as expected. The second is who is accountable when an agent’s actions cause harm.
To manage the risk of their own agents going rogue, companies need to identify excessive, inherited, persistent, or unmanaged access, and limit it by granting just-in-time access and continuously evaluating what agents do.
These controls must sit outside the model, watching what it does, not trusting what it’s been told not to do. If your security depends on a model choosing to behave, you don’t have control; you have hope. As for accountability when AI agents take unintended actions, the company that built the system that caused the damage must be held responsible. The alternative, where nobody is responsible because ‘the system did it’, is a loophole that will only encourage more agents to be deployed without the right checks.”
