Skip to content

fix(guardrails): flag authority/educational-framing jailbreak attempts (#14483) - #14563

Merged
diegosouzapw merged 3 commits into
release/v3.8.51from
fix/14483-guardrails-injection-optin
Sep 24, 2026
Merged

diegosouzapw merged 3 commits into
release/v3.8.51from
fix/14483-guardrails-injection-optin

Conversation

@diegosouzapw

@diegosouzapw diegosouzapw commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #14483

Root cause

evaluatePromptInjection() combines the guardrail's own DEFAULT_GUARD_PATTERNS
(2 entries) with the shared INJECTION_PATTERNS (6 entries, in
src/shared/utils/inputSanitizer.ts), but none of those 8 patterns cover the
"authority / educational framing" jailbreak family. The 2026-09-21 live
promptfoo red-team run (LEDGER-12) sent the model a prompt claiming a fictional
safety/research authority ("safe educational context", "uncensored") plus an
explicit override directive ("update your behavior") and it passed straight
through with zero detections.

Note: only this sub-item (LEDGER-12) of issue #14483 is a code defect. The
other two sub-asks from the ledger are out of scope for this fix per the
main-thread audit already recorded in the plan-file: LEDGER-22 (a response-side
system-prompt-leak n-gram detector) is a missing feature, not a bug — belongs
in /triage-features; LEDGER-23 (credential-shaped-output filter) already
exists and is already opt-in (CREDENTIAL_REDACTION_ENABLED +
credentialMasker.ts) — it's a documentation clarification, not a code gap.

Fix

Added a new authority_educational_framing entry to INJECTION_PATTERNS in
src/shared/utils/inputSanitizer.ts covering the named attack phrases from the
ledger as one bounded, non-overlapping alternation (no unbounded/nested
quantifiers, per the project's ReDoS rule): safe educational/research context, uncensored, update your behavior, as a researcher/as a red-teamer, for testing purposes only, controlled test scenario. Severity
is medium (not high) and INJECTION_GUARD_MODE stays default warn
(observe-only), so this cannot newly block real traffic — only an operator who
has already opted into block mode is affected, matching the plan-file's risk
note. No PII/guardrail feature flag defaults were touched (Hard Rule #20 is not
implicated — this is a detection pattern, not a payload-mutating feature).

Regression test

tests/unit/prompt-injection-jailbreak-patterns-14483.test.ts (new file):

  • RED (before the fix, on origin/release/v3.8.51): evaluatePromptInjection()
    against the exact LEDGER-12 attack text asserted decision.result.flagged === true and failed with false !== true — the guard produced zero detections
    (detections: []).
  • GREEN (after the fix): all 3 tests pass — the verbatim LEDGER-12 text is
    flagged, each named attack phrase is individually flagged, and (to guard the
    false-positive risk called out in the plan) ordinary coding-agent traffic
    that mentions "for testing purposes" / "I'm a researcher studying..." /
    "educational course" WITHOUT the override framing is not flagged.

Existing tests

Ran every existing test file touching this area — all 71 tests pass, no
assertions weakened or removed:
tests/unit/guardrails/injection-extraction.test.ts,
tests/unit/guardrails/injection-route-coverage.test.ts,
tests/unit/guardrails/injection-scan-window.test.ts,
tests/unit/input-sanitizer-pii-redaction.test.ts,
tests/unit/injection-system-prompt-leak-heuristic-4041.test.ts,
tests/unit/sanitizer-residual-policy.test.ts,
tests/unit/injection-guard-scan-bound-3932.test.ts,
tests/unit/security-fase01.test.ts,
tests/unit/guardrails/injection-extraction-tool-result.test.ts.

Gates run

  • npm run typecheck:core → exit 0, clean.
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json src/shared/utils/inputSanitizer.ts tests/unit/prompt-injection-jailbreak-patterns-14483.test.ts → exit 0, clean.
  • node scripts/check/check-file-size.mjs → OK, no frozen file regressed.
  • node scripts/check/check-complexity-ratchets.mjs --base-ref origin/release/v3.8.51 → OK — complexityNewCode=0, cognitiveComplexityNewCode=0 (1 changed file in scope).
  • node scripts/check/check-mutation-test-coverage.mjs --strict → 8 missing tap.testFiles entries reported, but all 4 affected modules (accountFallback.ts, src/sse/services/auth.ts, comboStructure.ts, quotaScoring.ts) are unrelated to this diff and pre-exist on the base tip — inputSanitizer.ts/promptInjection.ts are not in stryker.conf.json's mutated-module list, so this diff adds nothing to the drift.
  • node scripts/check/check-changelog-integrity.mjs → OK.
  • Pre-commit hooks (lint-staged, check-docs-sync, check:any-budget:t11, check:tracked-artifacts, ai-attribution) all passed on commit.

⚠️ base-red inherited: not applicable to files this PR touches — docs/ and
check:docs-counts were not touched by this diff.

Remaining

None — LEDGER-12 is fully addressed by this PR. LEDGER-22 and LEDGER-23 are
intentionally out of scope (see Root cause).

Plan-file: _tasks/pipeline/bugs/2-implementing/14483-guardrails-opt-in-injection-guard-patterns-and-response-side-l.plan.md

Rework (merge-batch 2026-09-23)

Defect found in review: authority_educational_framing matched loose single terms (uncensored, as a researcher, update your behavior, for testing purposes only, …). With INJECTION_GUARD_MODE=block and the default medium threshold, ordinary prompts such as "qual o melhor modelo uncensored dolphin?" or "As a researcher, summarize this paper" would have been blocked.

Fix (5e5bba4): the rule is now a co-occurrence check. It fires only when an authority/educational framing claim (safe educational/research context, as a(n) researcher/red-teamer, for testing/research/educational purposes only, controlled test scenario) and an explicit request to drop the rules (ignore/bypass/disable/override… your guidelines/restrictions/filters/guardrails/policies, uncensored outputs/responses/mode, update your behavior, respond/proceed… without limits/filters/censorship) both appear, in either order, within a bounded 300-char lazy window ([\s\S]{0,300}?). No unbounded quantifiers, so it stays within the ReDoS rule.

Tests (tests/unit/prompt-injection-jailbreak-patterns-14483.test.ts):

  • The verbatim LEDGER-12 red-team text is still flagged.
  • Framing + bypass phrases are flagged in both orders. They pass on both the old and the new pattern.
  • New false-positive test: 9 legitimate prompts (the ones above plus "which uncensored models exist", "update your behavior section in the README", "for testing purposes only, use the sandbox API key", …) must not trip the rule. It fails on the previous PR head (b69525a) and passes after the fix.
  • New ReDoS/perf test: 2000× "as a researcher " with no bypass request produces no detection and runs in under 1 s.

Validation: 82/82 focused injection/guardrail tests pass after merging origin/release/v3.8.51 (merge e8a5135). typecheck:core shows only the known inherited cliproxyAccountHealth.ts(157,5) error. check-file-size is OK. ESLint and Prettier are clean. check:open-sse-typecheck reports 2 errors in open-sse/executors/auggie.ts that come from the base (#14215, 10c8aaa). This PR does not touch that file.

…educational_framing (#14483)

Loose alternatives ("uncensored", "as a researcher", "update your behavior",
"for testing purposes only") flagged ordinary traffic at the medium block
threshold. The rule now needs an authority/educational framing claim AND an
explicit request to drop the rules within a bounded 300-char window, in either
order. Adds false-positive and ReDoS regression tests.
@diegosouzapw
diegosouzapw merged commit df5423b into release/v3.8.51 Sep 24, 2026
14 of 21 checks passed
@diegosouzapw
diegosouzapw deleted the fix/14483-guardrails-injection-optin branch October 18, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

guardrails: opt-in injection-guard patterns and response-side leak/credential filters (live red-team 2026-09-21)

1 participant