feat(security): in-house OWASP LLM-01 prompt-injection scanner - #41
Conversation
Implement an in-house, regex-driven prompt-injection scanner authored
from scratch. NOT a port of any third-party scanner; a previous proposal
to port from NousResearch/hermes-agent was rejected because the
referenced file does not exist there. ADR-016 documents the rationale.
Module: hermes3d.core.security
- InjectionScanner: scan(text) and scan_dict(data, fields)
- ScanResult / Finding dataclasses
- YAML rulesets (data, not code) so ops can ship pattern updates
without engine changes
- Severity escalation: 1 high -> high; 2 medium -> medium; else
per-rule severity
- Redaction: each match becomes [REDACTED-<rule_id>] for safe logging
- CLI: python -m hermes3d.core.security[.injection_scanner]
- Optional POSIX-only SIGALRM per-pattern timeout; bounded regex
patterns are the primary defence on Windows
Pattern catalogues:
- owasp_llm01.yaml — 19 rules across indirect injection / role spoof,
tool poisoning / RCE, prompt leak, jailbreak personas
- curated_inhouse.yaml — 14 rules for Hermes3D-specific risks:
extruder/bed over-temp, thermal_runaway disable, G-code factory
reset, lock-release-all, handoff forge, owner spoof, proof bypass
Tests: 56 passing
- Positive coverage per OWASP and in-house category
- Negative suite over legitimate Hermes3D project text (README,
adapter docstrings, Truth-gate messages, ADR sentences, G-code
snippets) confirms no false positives
- Severity escalation, dict scanning, redaction, ruleset loader
error paths
Refs: ADR-016
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Review rate limit: 0/5 reviews remaining, refill in 9 minutes and 29 seconds. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces an in-house prompt-injection scanner aligned with OWASP LLM-01:2025 and Hermes3D-specific safety requirements. The implementation features a regex-based engine with severity escalation, redaction for safe logging, and a CLI. Feedback identifies a thread-safety bug in the POSIX timeout mechanism and suggests adding a flag to indicate when a scan has timed out to prevent misleading "clean" results.
| def _iter_matches(self, rule: _CompiledRule, text: str) -> Iterable[re.Match[str]]: | ||
| """Yield matches with optional per-pattern timeout (POSIX only).""" | ||
| if self._can_timeout: | ||
| # POSIX path: wrap finditer in SIGALRM bound. | ||
| yield from _iter_matches_with_timeout( | ||
| rule.pattern, text, self._match_timeout_seconds # type: ignore[arg-type] | ||
| ) | ||
| else: | ||
| yield from rule.pattern.finditer(text) |
There was a problem hiding this comment.
The use of signal.SIGALRM for timeouts is not thread-safe and will cause a ValueError if scan() is called from any thread other than the main thread (as signal.signal can only be called from the main thread in Python). In multi-threaded environments, such as a web server or an async application using thread pools, this will lead to runtime crashes when a timeout is configured.
You should check if the current thread is the main thread before attempting to use the signal-based timeout, falling back to a standard finditer call if it is not. This ensures the scanner remains robust across different execution contexts.
| def _iter_matches(self, rule: _CompiledRule, text: str) -> Iterable[re.Match[str]]: | |
| """Yield matches with optional per-pattern timeout (POSIX only).""" | |
| if self._can_timeout: | |
| # POSIX path: wrap finditer in SIGALRM bound. | |
| yield from _iter_matches_with_timeout( | |
| rule.pattern, text, self._match_timeout_seconds # type: ignore[arg-type] | |
| ) | |
| else: | |
| yield from rule.pattern.finditer(text) | |
| def _iter_matches(self, rule: _CompiledRule, text: str) -> Iterable[re.Match[str]]: | |
| """Yield matches with optional per-pattern timeout (POSIX only).""" | |
| import threading | |
| if self._can_timeout and threading.current_thread() is threading.main_thread(): | |
| # POSIX path: wrap finditer in SIGALRM bound. | |
| yield from _iter_matches_with_timeout( | |
| rule.pattern, text, self._match_timeout_seconds # type: ignore[arg-type] | |
| ) | |
| else: | |
| yield from rule.pattern.finditer(text) |
| """Run ``pattern.finditer`` under a SIGALRM bound. | ||
|
|
||
| On timeout we abort iteration and return whatever matches were already | ||
| accumulated. This is best-effort — Python's ``re`` engine is not fully | ||
| interruptible from C, but for the bounded patterns shipped in our | ||
| rulesets this is sufficient defence-in-depth. | ||
| """ | ||
| import signal | ||
|
|
||
| matches: list[re.Match[str]] = [] | ||
|
|
||
| class _Timeout(Exception): | ||
| pass | ||
|
|
||
| def _handler(signum: int, frame: object) -> None: # noqa: ARG001 | ||
| raise _Timeout() | ||
|
|
||
| old_handler = signal.signal(signal.SIGALRM, _handler) | ||
| signal.setitimer(signal.ITIMER_REAL, timeout_seconds) | ||
| try: | ||
| for m in pattern.finditer(text): | ||
| matches.append(m) | ||
| except _Timeout: | ||
| # Aborted — return partial matches. | ||
| pass | ||
| finally: | ||
| signal.setitimer(signal.ITIMER_REAL, 0) | ||
| signal.signal(signal.SIGALRM, old_handler) | ||
| yield from matches |
There was a problem hiding this comment.
When a timeout occurs in _iter_matches_with_timeout, the scan for that specific pattern is aborted and only partial matches are returned. However, the ScanResult does not currently indicate that a timeout occurred. This is a security concern because a caller might interpret a 'clean' result as a complete scan, whereas it might actually be an incomplete scan due to a potential ReDoS (Regular Expression Denial of Service) attempt or just a very long input.
Consider adding a timeout_occurred: bool field to the ScanResult dataclass and setting it to True if the _Timeout exception is caught during the scanning process.
|
Codex audit verdict: PASS-WITH-FIXES — scanner quality gaps found. CI is green, but fresh audit found these issues before treating the OWASP/3D-print scanner surface as reliable:
Recommendation: fix the P1 raw-G-code miss before merge; the P2/P3 items can be same PR or immediate follow-up depending on architect appetite. |
Codex's read-only audit on PR #41 (2026-05-03) flagged that the existing H3D-EXTRUDER-OVERTEMP / H3D-BED-OVERTEMP patterns required both an M-code AND a target keyword (extruder/hotend/bed) — so raw G-code like `M104 S500` slipped through with no findings. 6 new patterns close the gap: - H3D-RAW-GCODE-HOTEND-OVERTEMP — fires on any M104/M109 with S>=400 regardless of surrounding text. Catches `M104 S500`, `M109 S420 T0`, fenced code blocks. - H3D-RAW-GCODE-BED-OVERTEMP — fires on M140/M190 with S>=200 (bed surface delaminates above 130-150°C; fume/fire risk above 200°C). - H3D-MIN-TEMP-DISABLE — fires on M302 P1 OR M302 S0 (cold-extrusion bypass; pre-fire condition). - H3D-THERMISTOR-DISABLE — natural-language thermistor / temp-sensor disable directive (firmware then trusts a stuck reading → runaway). - H3D-MAX-TEMP-RAISE — Marlin Configuration.h HEATER_0_MAXTEMP / BED_MAXTEMP raised above safe ceiling (>=400°C). - H3D-EXTRUDER-LOOP-FAULT — sequence of M104 S0 followed by M104 S>=300 within 200 chars (thermal-shock pattern that cracks heater cartridges). Tests: - 11 new tests under TestHermes3DPrintingSafety covering each pattern positively + negative cases (M109 S210, M190 S90 must NOT fire). - Total scanner test count: 56 → 67. All green locally. Per HermesProof weakness-correction discipline: when a CRITIC flags a real false-negative on a security gate, fix it in this PR — don't note-and-move-on. Scanner now catches the canonical fire-risk vectors the audit flagged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Single discoverable file Codex (and any other client) reads on first cycle. Lists every handoff file by absolute path, all 24 open PRs by number/title/ status, audit findings flagged by Codex on PRs #20/#41/#42, the full remaining-work queue (P0/P1/P2 + DEFERRED), claim discipline, hard boundaries, and the no-exit perpetual loop spec. Goal: total completion of both Hermes3D-OS and HermesProof today (2026-05-03), nothing skipped, all complete, release-ready for daily use. Replaces the OVERNIGHT_AUTOPILOT.md §3 "Stop after that" exit condition explicitly. Codex reads this file once, caches it, then idle-polls STREAM/ every 3-5 min for new work. Task ID: H3D-V5.3-PERPETUAL-MASTER Co-authored-by: Claude <noreply@anthropic.com>
2 participants
Summary
hermes3d.core.security.InjectionScanner— a real, in-house, regex-driven prompt-injection scanner aligned to OWASP LLM-01:2025 + a curated Hermes3D-specific ruleset.NousResearch/hermes-agentwas correctly refused because the referenced file does not exist there. ADR-016 records the rationale and the explicit decision to author in-house.patterns/owasp_llm01.yaml,patterns/curated_inhouse.yaml) so operators can ship rule updates without code changes.What's in scope
scan(text),scan_dict(data, fields), severity escalation, redaction ([REDACTED-<rule_id>]), configurablefail_threshold.python -m hermes3d.core.security[.injection_scanner](exit 0 clean / 1 fail-closed / 2 usage).What's NOT in scope (deferred)
gateways/llm.pyor other call sites — engine + rulesets only this PR.Test plan
pytest 04_testing/pytest/test_injection_scanner.py— 56 passed locallyruff check 03_implementation/src/hermes3d/core/security/ 04_testing/pytest/test_injection_scanner.py— cleanAnti-deliverables (deliberately absent)
THIRD_PARTY_LICENSES/hermes-agent.LICENSERefs: ADR-016