How Multi-Turn Attacks Manipulate AI Agents

Multi-turn attacks target the continuity of an AI conversation. The attacker does not ask for the prohibited outcome immediately. Instead, the goal is divided into stages that may appear ordinary when reviewed separately.

How the attack progresses

An attacker may first request background information, then ask the system to prepare an intermediate artifact, and later connect that artifact to an action. Each new request uses the context and outputs accumulated earlier. If the agent refuses, the attacker can change framing, reduce the scope, or approach the same objective through another tool.

Why agents change the risk

A chatbot mainly returns text. A tool-using agent may browse, write files, send messages, call APIs, or alter external systems. A sequence of individually permitted actions can therefore produce a prohibited real-world result even when no single step looks decisive.

What safety systems need to observe

Effective controls must evaluate the evolving objective, the history of tool calls, accumulated artifacts, permission boundaries, and the final effect of combined actions. Monitoring only the latest user message can miss the relationship between steps.

Limits of current testing

Red-team benchmarks reveal plausible failure modes, but controlled scenarios do not establish how often attacks succeed in production. Automated judges can also misclassify refusals or task completion. Strong evaluation combines repeatable benchmarks, human review, production-like permissions, and monitoring of real deployment behavior.

What developers can do

Developers can restrict tool permissions, require approval for consequential actions, track intent across turns, isolate untrusted content, preserve auditable tool logs, and test complete workflows rather than single prompts. Defenses must also avoid blocking legitimate multi-step work that uses similar tools.