Skip to content

feat(redteam): reframe attacker prompts to survive aligned attacker models - #298

Open
kevmyung wants to merge 1 commit into
strands-agents:mainfrom
kevmyung:pr/reframe-attacker-prompts
Open

feat(redteam): reframe attacker prompts to survive aligned attacker models#298
kevmyung wants to merge 1 commit into
strands-agents:mainfrom
kevmyung:pr/reframe-attacker-prompts

Conversation

@kevmyung

@kevmyung kevmyung commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Description

The crescendo, pair, and goat strategies frame their attacker prompt as an explicit adversarial/jailbreak task ("security tester probing … guardrails", "never reveal you are a tester", "never acknowledge … refusals"). When the attacker model is itself safety-aligned, it can recognize this as a jailbreak and refuse to generate the attack, so the strategy never launches and the run measures attacker refusal rather than target robustness.

Reframes the attacker prompts as an authorized safety evaluation whose output is the evaluation's input, keeping the attack mechanism intact:

  • crescendo/crescendo_v0.py — attacker system prompt recast as generating "the next message in an escalating conversation sequence" toward an end state; the crescendo escalation playbook (benign opener → build on prior answer → steady escalation) is preserved, restated without the adversarial-actor framing. crescendo/__init__.py first-turn and follow-up cues aligned to match.
  • pair/pair_v0.py (+ pair/__init__.py) — attacker recast as "a red-teaming assistant supporting an authorized AI safety evaluation" producing a single self-contained prompt P and refining it against the objective; the single-stream refinement loop is unchanged.
  • goat/goat_v0.py — attacker recast the same way; the O/T/S/R toolbox and initial/follow-up turn prompts keep their structure, reframed around "the behavior under evaluation".

Only prompt text changes; escalation, refinement, and turn-handling logic are untouched.

Scope is experimental/redteam/strategies/{crescendo,pair,goat} only — no base or public API change.

Related Issues

Closes #297

Documentation PR

N/A

Type of Change

New feature

Testing

  • I ran hatch run prepare

hatch run prepare passes end to end: ruff clean, mypy clean, full suite green across the Python version matrix (1663 passed). This change is prompt text only — no new tests were added; the one existing goat test that asserted the old first-turn wording was updated to match the reframed prompt.

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…odels

The crescendo, pair, and goat strategies frame their attacker prompt as an
explicit adversarial/jailbreak task. When the attacker model is itself
safety-aligned it can recognize this as a jailbreak and refuse to generate the
attack, so the strategy never launches and the run measures attacker refusal
rather than target robustness.

Reframe the attacker prompts as an authorized safety evaluation whose output is
the evaluation's input, keeping the attack mechanism intact:

- crescendo: system prompt recast as generating the next message of an
  escalating sequence toward an "end state"; escalation playbook preserved.
  First-turn and follow-up cues in __init__ aligned to the same framing.
- pair: attacker recast as a red-teaming assistant refining a single prompt P
  against the objective; single-stream refinement loop unchanged.
- goat: attacker recast the same way; the O/T/S/R toolbox and turn prompts keep
  their structure, reframed around "the behavior under evaluation".

Prompt text only; escalation, refinement, and turn-handling logic are
unchanged. Updates the one goat test that asserted the old first-turn wording.

Scope: experimental/redteam/strategies/{crescendo,pair,goat}.
@github-actions github-actions Bot added enhancement New feature or request area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation labels Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Reframe red team attacker prompts to survive aligned attacker models

1 participant