domains / families / security / orchestrating-llm-attacks-with-pyrit

orchestrating-llm-attacks-with-pyrit

separate

Build automated multi-turn adversarial attacks against conversational LLM targets using Microsoft PyRIT's RedTeamingOrchestrator, CrescendoOrchestrator (gradual escalation), and TreeOfAttacksWithPruningOrchestrator (adaptive branching), with scorer feedback loops and persisted conversation memory. Use when single-shot LLM scanning is insufficient and you need multi-turn, scorer-driven AI red-team campaigns against a chatbot or agent.

When to use

• When single-shot scanning (e.g. garak) finds a model robust to one-prompt attacks and you need to test multi-turn escalation (Crescendo) or adaptive branching (TAP).
• When assessing a conversational agent or assistant where state accumulates over a dialogue.
• When you need an automated, scorer-driven harness rather than manual prompt-by-prompt red teaming.
• When building reproducible red-team campaigns with persisted conversation memory for evidence and regression.
• When evaluating whether guardrails hold under gradual, plausibly-deniable escalation.

When not to use

• single-shot LLM scanning
• manual prompt-by-prompt red teaming

Skill page

Source