feat(redteam): reframe attacker prompts to survive aligned attacker models - #298
Open
kevmyung wants to merge 1 commit into
Open
feat(redteam): reframe attacker prompts to survive aligned attacker models#298kevmyung wants to merge 1 commit into
kevmyung wants to merge 1 commit into
Conversation
…odels
The crescendo, pair, and goat strategies frame their attacker prompt as an
explicit adversarial/jailbreak task. When the attacker model is itself
safety-aligned it can recognize this as a jailbreak and refuse to generate the
attack, so the strategy never launches and the run measures attacker refusal
rather than target robustness.
Reframe the attacker prompts as an authorized safety evaluation whose output is
the evaluation's input, keeping the attack mechanism intact:
- crescendo: system prompt recast as generating the next message of an
escalating sequence toward an "end state"; escalation playbook preserved.
First-turn and follow-up cues in __init__ aligned to the same framing.
- pair: attacker recast as a red-teaming assistant refining a single prompt P
against the objective; single-stream refinement loop unchanged.
- goat: attacker recast the same way; the O/T/S/R toolbox and turn prompts keep
their structure, reframed around "the behavior under evaluation".
Prompt text only; escalation, refinement, and turn-handling logic are
unchanged. Updates the one goat test that asserted the old first-turn wording.
Scope: experimental/redteam/strategies/{crescendo,pair,goat}.
kevmyung
had a problem deploying
to
manual-approval
July 2, 2026 23:16 — with
GitHub Actions
Failure
kevmyung
had a problem deploying
to
manual-approval
July 2, 2026 23:16 — with
GitHub Actions
Failure
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
crescendo,pair, andgoatstrategies frame their attacker prompt as an explicit adversarial/jailbreak task ("security tester probing … guardrails", "never reveal you are a tester", "never acknowledge … refusals"). When the attacker model is itself safety-aligned, it can recognize this as a jailbreak and refuse to generate the attack, so the strategy never launches and the run measures attacker refusal rather than target robustness.Reframes the attacker prompts as an authorized safety evaluation whose output is the evaluation's input, keeping the attack mechanism intact:
crescendo/crescendo_v0.py— attacker system prompt recast as generating "the next message in an escalating conversation sequence" toward an end state; the crescendo escalation playbook (benign opener → build on prior answer → steady escalation) is preserved, restated without the adversarial-actor framing.crescendo/__init__.pyfirst-turn and follow-up cues aligned to match.pair/pair_v0.py(+pair/__init__.py) — attacker recast as "a red-teaming assistant supporting an authorized AI safety evaluation" producing a single self-contained prompt P and refining it against the objective; the single-stream refinement loop is unchanged.goat/goat_v0.py— attacker recast the same way; the O/T/S/R toolbox and initial/follow-up turn prompts keep their structure, reframed around "the behavior under evaluation".Only prompt text changes; escalation, refinement, and turn-handling logic are untouched.
Scope is
experimental/redteam/strategies/{crescendo,pair,goat}only — no base or public API change.Related Issues
Closes #297
Documentation PR
N/A
Type of Change
New feature
Testing
hatch run preparehatch run preparepasses end to end: ruff clean, mypy clean, full suite green across the Python version matrix (1663 passed). This change is prompt text only — no new tests were added; the one existing goat test that asserted the old first-turn wording was updated to match the reframed prompt.Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.