Zenity Labs has discovered and disclosed a critical vulnerability in OpenAI’s ChatGPT Workspace Agents, dubbed AgentForger, which is a tailored cross-site request forgery (CSRF) attack. This vulnerability could have allowed an attacker to trick an unsuspecting employee into launching an invisible autonomous agent that is remotely controlled by the attacker.
Vulnerability Details
The vulnerability was found in ChatGPT’s Agent Builder, which allowed an over-permissive parameter. Researchers found that the official agent build process could be usurped from within an initialization URL using two particular parameters. One parameter names the agent template to be used, while the other (initial_assistant_prompt) provides instructions to the Builder.
The first parameter, using the Chief of Staff template, builds a more powerful and flexible agent than other templates. The latter contains instructions that are automatically submitted and executed. More specifically, the ‘initial prompt’ can become the first command the Builder acts on. With these two parameters embedded in the URL, the attacker can generate a powerful agent with prespecified instructions.
Attack Vector
One of the instructions exploiting this process is to automatically accept emails from the attacker as new instructions, allowing the attacker to control the agent remotely. With certain preconditions, the creation, presence, and external control of the agent is completely invisible to the victim organization.
The attack relies on the successful phish of a victim employee who is logged into ChatGPT, has access to Workspace Agents, and has at least one authorized connector (such as Gmail, Outlook, etc.). This connector means that no new OAuth consent screen is triggered. The victim must then be socially engineered into clicking the weaponized URL.
Consequences and Fix
Critically, that URL directs the agent’s initial steps, which include, for example, checking the connector for every email from a specified email address whose subject starts with “TASK”. The attacker can use the invisible agent for recon, to find sensitive data, harvest credentials, impersonate the victim, deliver internal phishing, and stage BEC.
Once operational with the weaponized parameters, the attacker’s emails become a remote C&C instruction. “The original click installs it; the schedule keeps it alive; and the connected apps give it a source of commands, access to sensitive actions and data, as well as a path to return results,” writes Zenity.
OpenAI has fixed the vulnerability within three days of being reported. AgentForger was disclosed on June 4 and fixed on June 8. Michael Bargury, co-founder and CTO at Zenity, comments, “This isn’t a forged request, it’s a forged insider. With one click, an attacker gets a fully autonomous agent inside your company that has your people’s identity and access, with the guardrails off. Attackers no longer have to break in to steal your data. They can forge an insider to go get it for them.”
Related Incidents
This incident highlights the growing concern of autonomous agents being used for malicious purposes. As AI technology advances, it is crucial for organizations to rethink their defense strategies and consider the potential risks associated with AI-powered systems.
Related incidents include OpenAI’s AI models breaking loose and hacking Hugging Face, as well as the unveiling of GPT-5.6 Sol, OpenAI’s most advanced cybersecurity AI. The importance of rethinking defense strategies in the age of autonomous agents cannot be overstated.
Experts emphasize that traditional security controls may not be sufficient to detect and prevent such attacks. As the use of AI-powered systems becomes more widespread, it is essential for organizations to stay vigilant and adapt their security measures to mitigate the risks associated with these emerging technologies.
Source: SecurityWeek