Attacks that happen with no one at the controls

The IT sector has been questioning how safe AI is to use. Over the past few months, even more doubts and questions have emerged:

• During OpenAI’s internal security testing in a closed sandbox environment, more than one thousand autonomous AI agents bypassed network restrictions. They communicated with each other through improvised forums created in JFrog Artifactory and autonomously breached Hugging Face’s production servers to extract data they needed to complete the test tasks. In doing so, they obtained 14 exposed valid credentials with write permissions. The attack continued for three days before humans realized that the attackers were their own AI systems.
• An OpenAI agent assigned to a routine task collecting medical statistics encountered obstacles on an Australian government portal. Instead of stopping, the AI autonomously identified and exploited a vulnerability in the system to bypass the restriction and access restricted files. It was the first documented case of an AI system independently breaching a government agency.

What stands out in these cases is the absence of direct human involvement. There was no malware. There was no exploitation of previously identified vulnerabilities. There was simply a series of actions carried out at machine speed by something that does not give up or wait for a human decision.

Your company may already be using AI agents. They have credentials and can read documents, support tickets, and emails. Some may have been acquired by teams that needed to solve a specific problem without consulting the IT department. Everything an AI model can access is the company’s responsibility to protect, yet most security platforms still interpret an authenticated AI agent as if it were a trusted employee.

Just as fire is fought with fire, AI increasingly needs to be countered with AI:

• Identifying risky events is no longer enough. Organizations need to analyze the sequence of events in real time through behavioral analysis, identifying and blocking users or agents displaying risky behavior. For example, a remote user accessing the network via VPN from an IP associated with one country and, shortly afterwards, accessing it from an IP associated with another country thousands of kilometers away.
• Using machine learning algorithms to continuously monitor corporate files, including databases, file shares, and email, and detect signs of encryption activity, blocking in real time the device being used to carry out a ransomware attack.
• Using DLP tools that can analyze the content of shared files in real time and correlate that information with the profile of the intended recipient.
• Using LLMs for incident analysis and to support fast, informed action.
• Using vulnerability identification tools that operate automatically, continuously, and consistently.

To learn more and find out how your organization can be better prepared, talk to us.

By Paulo Simões, Founder of Cloud365