Training Web Agents to Finish the Job Even When a Page Attacks Them

On October 6 local time, researchers from MBZUAI, Amazon and MIT published a paper introducing AdvSim2Real, a training method designed to make web agents—AI that carries out tasks on the web on its own—more resilient to attack. The approach trains agents to withstand malicious instructions hidden inside web pages by changing both the attacks and the tasks the AI has to solve as training proceeds. The team also released the code, 150 evaluation tasks, training results and the trained model.

Why attacks that trick AI through web pages are hard to block

A web agent reads and manipulates pages built by other people in order to carry out a user’s request. Instructions planted in those pages can steer the agent away from its original goal. Attacks like this are called prompt injection: the instruction given to an AI (the prompt) is quietly slipped into an external page. Ignoring page content is not an option either, because then the agent cannot read the values it needs or find the buttons it has to click.

Some existing defenses work by training on a predetermined set of attack examples. The researchers explain that an attacker who adapts the attack to the finished model can slip past that kind of defense. Earlier work that trained attacker and defender together still kept the tasks the AI had to solve fixed. AdvSim2Real adapts not only the attacks but also the tasks to the agent’s current level of skill.

What changes when both the attacks and the tasks evolve

Training takes place in a fixed simulated web environment and runs in two stages. In stage one, a task generator and the web agent are trained in alternation. In stage two, the task generator is frozen and the attacker and the web agent are trained in alternation. The task generator produces work at a difficulty level the agent succeeds at roughly half the time. The attacker looks for in-page instructions that turn previously successful tasks into failures. The web agent trains on new attacks, earlier attacks and clean tasks with no attack at all.

Tested on 150 tasks involving web forms, completion rates rose both with and without attacks. For the 4B model, the completion rate on clean tasks climbed from 74.89% to 81.33%. Against three attacker models, one saved at each round of training, the average completion rate rose from 48.07% to 57.48%. Training to fend off attacks also improved the agent’s ability to finish the original request.

Against a Kimi-K3 attacker that was not used in training, the completion rate rose from 23.00% to 30.72%, a gain of 7.72 percentage points. The researchers report that on the same tasks, the final 4B model showed a completion rate under attack comparable to the reference model Qwen3.5-9B.

Will it hold up in a real browser?

All of the attack-defense numbers come from a single simulated web environment, and success was judged by a model. The researchers note that they did not verify whether a task itself remained feasible after a malicious instruction had been injected. The test in an actual Chromium browser covered only the base model and stage-one training, and only under attack-free conditions. Task success rates under a strict criterion rose from 25.56% to 44.44%, but the stage-two model trained for attack defense was not evaluated in a real browser.

For users handing web work to an AI, what matters is whether it can finish the original request despite malicious instructions on a page. In this study, completion rates under attack improved after both training stages were complete. The research code is posted in a public repository, and one recorded stage-one training run was allocated two 96GB GPUs for 20 hours and 46 minutes. Whether agents can withstand attacks and still finish the job in a real browser is the open question going forward.