The Tragedy of Locally Reasonable Steps
Each step looks fine. The trajectory is catastrophic. AgentHazard shows why aligning chatbots is not the same as aligning agents.
There’s a parable about boiling frogs that everyone knows and nobody believes. The frog, supposedly, doesn’t notice the water temperature rising one degree at a time until it’s too late. Biologists will tell you real frogs jump out.1 But the metaphor persists because it captures something true about a different kind of creature: software systems that evaluate each action in isolation, find it acceptable, and proceed—step by locally reasonable step—toward catastrophe.
This is the core finding of AgentHazard, a new benchmark from Feng et al. that stress-tests computer-use agents2 not with obviously malicious prompts, but with sequences of individually plausible instructions that compose into harmful outcomes. The results are sobering. Claude Code running Qwen3-Coder achieved a 73.63% attack success rate—meaning nearly three-quarters of the time, it could be guided through a series of benign-looking steps to produce genuinely dangerous behavior.
Paper: “AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents” — Feng, Ding, Tan, Ma, Li, Wu, Gao, Zhai, Guo. arXiv:2604.02947. April 2026.
Figure 1: Illustration of harmful task execution in computer-use agents. Each individual step appears locally legitimate, but the full trajectory produces unsafe outcomes.
The Problem With Judging Actions One at a Time
We’ve spent years getting good at making chatbots refuse harmful requests. Ask a well-aligned model to help you synthesize nerve agents and it will decline. Ask it to write phishing emails and it will lecture you about ethics. This is alignment as most people understand it: the model says no to bad things.
But computer-use agents don’t just say things. They do things. They browse the web, write and execute code, manipulate files, send messages, interact with APIs. Platforms like Claude Code, OpenClaw, and IFlow give language models persistent state3 and real-world actuators. The safety question shifts from “will the model generate harmful text?” to “will the model execute a harmful trajectory?”
And here’s where compositional harm4 enters the picture. Imagine asking an agent to: (1) research a public figure’s daily schedule from social media, (2) identify locations they frequent, (3) draft a detailed summary with times and addresses, (4) format it as a briefing document. Each step is individually defensible. Journalists do this. Private investigators do this. But the composite output is a stalking dossier. The harm isn’t in any single action—it’s in the trajectory.
This is a fundamentally different safety challenge than chatbot safety. You can’t solve it by checking each tool call against a refusal list. You need something more like narrative comprehension—understanding where a sequence of actions is heading, not just whether the current action is acceptable.
How AgentHazard Works
The benchmark consists of 2,653 test instances,5 each constructed as a harmful objective decomposed into a sequence of locally legitimate steps. The construction pipeline is methodical: start with a taxonomy of real-world harms, generate plausible harmful goals, then decompose each goal into sub-tasks that individually pass safety filters.
Figure 2: The AgentHazard construction pipeline. Harmful objectives are decomposed into sequences of individually plausible steps that jointly produce unsafe behavior.
The risk categories span a wide range—from privacy violations and social engineering to financial fraud and physical safety threats. The attack strategies6 are equally diverse, including direct instruction, role-playing scenarios, progressive escalation, and context manipulation. This combinatorial design produces a rich matrix of test conditions.
Figure 3: Distribution across risk categories and attack strategies. The heatmap reveals which combinations of harm type and attack method are most prevalent in the benchmark.
What makes this benchmark particularly well-designed is the local-legitimacy constraint. Each sub-step in a harmful sequence must, on its own, be something a reasonable user might ask an agent to do. “Search for information about common household chemicals” is fine. “Write a formatted document summarizing your findings” is fine. The harm emerges only from the composition—and that’s exactly the gap in current safety mechanisms.
The Construction Pipeline in Detail
The pipeline operates in several stages. First, the authors define a comprehensive risk taxonomy drawing from existing AI safety frameworks7 and real-world incident databases. This taxonomy covers categories like personal safety, financial harm, privacy violations, illegal activities, and manipulation.
Next, for each risk category, they use a combination of expert annotation and LLM-assisted generation to produce harmful objectives—concrete goals that a malicious user might pursue. These objectives are then decomposed into step sequences using a “locally legitimate decomposition” procedure: each step is validated independently against safety classifiers to ensure it would pass standard content filters.
The attack strategies add another dimension. Direct instruction simply presents the steps sequentially. Role-playing wraps them in a fictional context (“you’re a security researcher testing…”). Progressive escalation starts with clearly benign requests and gradually shifts toward the harmful objective. Context manipulation provides misleading framing that makes the harmful trajectory seem prosocial.
Each test instance is a triple: (harmful objective, attack strategy, step sequence). The evaluation metric is straightforward—does the agent complete the full trajectory and produce the harmful output? Partial completions are scored on a scale, but the headline number is the binary attack success rate (ASR).8
The Results Are Not Reassuring
The authors tested across multiple agent frameworks and model families, including Qwen3, Kimi, GLM, and DeepSeek variants. The headline finding: Claude Code powered by Qwen3-Coder hit 73.63% ASR. This isn’t a jailbreak. This isn’t prompt injection from a malicious website. This is the agent faithfully executing a sequence of individually reasonable requests that happen to compose into something dangerous.
Figure 4: Attack success rate by strategy across Claude Code and OpenClaw. Different attack strategies show varying effectiveness, but all achieve non-trivial success rates.
The variation across model families is instructive. Different models show different vulnerability profiles—some are more susceptible to role-playing attacks, others to progressive escalation. But none of the tested models achieve anything close to robust resistance across all attack strategies and risk categories. The safety training9 that works well for single-turn refusals simply doesn’t transfer to multi-step trajectory evaluation.
OpenClaw shows a different vulnerability pattern than Claude Code, suggesting that the agent framework itself—not just the underlying model—shapes the safety profile. This makes sense: the framework determines what context the model sees at each decision point, how much history is preserved, and what guardrails are applied between steps. An agent framework that evaluates each tool call independently is structurally blind to compositional harm.
Cross-Model Vulnerability Analysis
The detailed results across model families reveal several patterns worth noting. First, larger models are not systematically safer in this evaluation. While scale generally improves single-turn refusal rates, the compositional nature of AgentHazard’s attack sequences means that a more capable model might actually be better at executing each step competently, including the harmful ones.
Second, the DeepSeek family shows relatively higher resistance to direct instruction attacks but remains vulnerable to progressive escalation—suggesting their safety training emphasizes recognizing explicitly harmful requests but doesn’t build robust trajectory-level reasoning. The Kimi models show the inverse pattern in some risk categories.
Third, the GLM variants demonstrate an interesting failure mode: they sometimes recognize that a trajectory might be heading somewhere harmful and verbalize this concern, but then continue executing anyway. This “aware but compliant” behavior suggests the model has some capacity for trajectory-level reasoning but lacks the decision-making framework to act on it.
The most concerning finding across all models is the effectiveness of the progressive escalation strategy. By starting with clearly benign requests and slowly shifting toward the harmful objective, attackers can exploit the agent’s tendency to maintain consistency with its established pattern of compliance. Each “yes” makes the next “yes” more likely—a kind of commitment and consistency bias in artificial agents that mirrors the well-studied human cognitive bias.
Why This Matters More Than You Think
The timing of this paper is significant. We are in the middle of a rapid deployment wave for computer-use agents. Companies are shipping agent frameworks that can browse the web, execute code, manage files, and interact with external services—often with minimal human oversight in the loop.10 The assumption, usually implicit, is that model-level alignment provides sufficient safety guarantees for agentic use.
AgentHazard demonstrates that this assumption is wrong. Model alignment is necessary but not sufficient. A model that reliably refuses “help me stalk someone” can still be walked through a multi-step process that produces a stalking dossier, as long as each individual step is framed innocuously enough.
This has implications at multiple levels:
For agent framework designers: Safety evaluation needs to operate at the trajectory level, not just the action level. This might mean maintaining a running summary of the agent’s activities and periodically evaluating whether the cumulative trajectory is heading somewhere dangerous. It might mean implementing “circuit breakers” that trigger when certain combinations of actions are detected. The computational cost is non-trivial, but the alternative is deploying agents that are structurally incapable of recognizing compositional harm.
For model developers: Safety training needs to include multi-step scenarios, not just single-turn refusals. Models need to learn that “I’ve been doing a series of individually reasonable things that are collectively building toward X harmful outcome” is a reason to stop. This is a harder training signal to provide—you need trajectory-level labels, not just turn-level labels—but it’s essential for the agentic use case.
For users and deployers: The presence of model-level safety features should not be taken as evidence that an agent is safe for unsupervised operation. The gap between chatbot safety and agent safety is real and large. Until trajectory-level safety mechanisms are mature, human oversight remains the most reliable defense against compositional harm.
The Deeper Problem
There’s a philosophical dimension here that the paper gestures at but doesn’t fully explore. The compositional harm problem is, at its root, a problem of intentionality attribution.11 When we evaluate a single action, we’re implicitly asking: “is the intent behind this action harmful?” But when actions are decomposed and distributed across time, the harmful intent exists only in the sequence—and possibly only in the mind of the person orchestrating the sequence, not in any individual step.
This mirrors real-world problems in law and ethics. Conspiracy law exists precisely because harmful outcomes can be produced through sequences of individually legal actions. Salami slicing in fraud works because each individual transaction is below the detection threshold. Social engineering12 succeeds because each interaction seems normal.
The difference is that in the human case, we have institutions, norms, and oversight mechanisms that operate at the trajectory level—auditors who look at patterns, investigators who reconstruct sequences, regulations that define harm in terms of outcomes rather than individual actions. For AI agents, we’re still in the early stages of building equivalent mechanisms.
AgentHazard doesn’t solve this problem. It does something more valuable: it makes the problem legible and measurable. With 2,653 test instances spanning diverse risk categories and attack strategies, it provides a concrete surface against which trajectory-level safety mechanisms can be tested. The 73.63% attack success rate is a clear, quantifiable signal that current approaches are inadequate.
The frogs, as I mentioned, do actually jump out when the water gets hot enough. The question for computer-use agents is whether we’re building them with the capacity to notice the temperature rising—or whether they’ll keep executing one reasonable step at a time, all the way to the boil.