fix: populate req_url_pattern before event creation

[Ice3man543]

Proposed changes

Populate the req_url_pattern in internalEvent to fix timing issue causing interactsh events to miss this causing wrong vulnhash calculation

Example:

d3t35d0ngn4503g699909td9q7j9jboit.oast.site'","matcher-status":true,"req_url_pattern":"/models?url=http%3a//{{interactsh-url}}"}

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Summary by CodeRabbit

• Refactor
  • Restructured request URL pattern handling to improve result processing efficiency. The ReqURLPattern field is now properly exposed in result events when the export option is enabled, with optimized population logic during operator execution.

[coderabbitai]

Walkthrough

This change refactors how HTTP request URL patterns are exported. Instead of propagating the URL pattern to each individual result, it now sets the pattern once in a shared interim event before operator execution, where operators can access it during processing.

Changes

Cohort / File(s)	Summary
HTTP URL Pattern Export Refactoring pkg/protocols/http/operators.go, pkg/protocols/http/request.go	Moved ReqURLPattern initialization from per-result propagation to interim event setup. Added public ReqURLPattern string field to output.ResultEvent struct. Operators now read the pattern from the shared interim event via ReqURLPatternKey and populate result items accordingly.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

The changes involve logic reorganization across two files with modifications to data flow and struct exposure. While the pattern remains consistent, careful verification is needed to ensure the interim event properly carries the URL pattern through operator execution and that all result items receive the intended value.

Poem

🐰 A pattern takes a grander stage,
From result to interim's page,
Where operators dance and flow,
One source of truth, a steady glow,
Refactored paths now cleanly show! 🌟

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The PR title "fix: populate req_url_pattern before event creation" directly and accurately describes the main change in the changeset. The modifications move the population of the req_url_pattern field from being handled per-result (after event creation) to being inserted into the interim event before operator execution (before event creation). The title is specific, clear, and uses the exact field name involved, making it easy for someone reviewing commit history to understand the primary change without ambiguity.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix-req-url-pattern

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 71293f2 and ff56e64.

📒 Files selected for processing (2)

• pkg/protocols/http/operators.go (2 hunks)
• pkg/protocols/http/request.go (1 hunks)

🧰 Additional context used

📓 Path-based instructions (2)

**/*.go

📄 CodeRabbit inference engine (CLAUDE.md)

**/*.go: Format Go code using go fmt
Run static analysis with go vet

Files:

• pkg/protocols/http/request.go
• pkg/protocols/http/operators.go

pkg/protocols/**/*.go

📄 CodeRabbit inference engine (CLAUDE.md)

Each protocol implementation must provide a Request interface with methods Compile(), ExecuteWithResults(), Match(), and Extract()

Files:

• pkg/protocols/http/request.go
• pkg/protocols/http/operators.go

🧬 Code graph analysis (2)

pkg/protocols/http/request.go (1)

pkg/protocols/http/build_request.go (1)

• ReqURLPatternKey (37-37)

pkg/protocols/http/operators.go (2)

pkg/output/output.go (1)

• InternalEvent (93-93)

pkg/protocols/http/build_request.go (1)

• ReqURLPatternKey (37-37)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Lint

🔇 Additional comments (2)

pkg/protocols/http/request.go (1)

1111-1111: Verify consistency with the interim event population.

This unconditionally sets req_url_pattern in the event, which is correct for ensuring it's always available internally. However, this should be consistent with the population in the interim event (line 1049). If the conditional check at line 1049 is removed (as suggested in the previous comment), both locations will correctly set the pattern unconditionally for internal use, while MakeResultEventItem in operators.go controls whether it's exported to results.

pkg/protocols/http/operators.go (1)

176-181: Implementation looks correct.

The code properly reads req_url_pattern from the internal event and conditionally includes it in the result event based on the ExportReqURLPattern option. The safe handling of the missing key and type conversion is appropriate.

Note: This implementation depends on the pattern being populated in the interim event at request.go:1049. Ensure that location correctly populates the pattern unconditionally (as suggested in the review comment for that file) so the value is always available in the internal event for operators and interactsh to use.

Also applies to: 206-206

[coderabbitai]

⚠️ Potential issue | 🟠 Major

The conditional check appears inconsistent with the PR objectives.

Based on the PR description and inline comment, the pattern needs to be available in the interim event for interactsh and operators to use during execution. However, the conditional if request.options.ExportReqURLPattern on line 1048 means the pattern won't be added to interimEvent when this option is false, preventing interactsh from accessing it for vulnhash calculation.

Consider making this unconditional so interactsh and operators always have access to the pattern:

 interimEvent := generators.MergeMaps(generatedRequest.dynamicValues, finalEvent)
 // add the request URL pattern to the event BEFORE operators execute
 // so that interactsh events etc can also access it
-if request.options.ExportReqURLPattern {
-	interimEvent[ReqURLPatternKey] = generatedRequest.requestURLPattern
-}
+interimEvent[ReqURLPatternKey] = generatedRequest.requestURLPattern

The ExportReqURLPattern check in MakeResultEventItem (operators.go:177) already ensures it's only exported to result events when enabled, so this change maintains the correct export behavior while fixing the availability for interactsh.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	// add the request URL pattern to the event BEFORE operators execute
	// so that interactsh events etc can also access it
	if request.options.ExportReqURLPattern {
	interimEvent[ReqURLPatternKey] = generatedRequest.requestURLPattern
	}
	// add the request URL pattern to the event BEFORE operators execute
	// so that interactsh events etc can also access it
	interimEvent[ReqURLPatternKey] = generatedRequest.requestURLPattern

🤖 Prompt for AI Agents

In pkg/protocols/http/request.go around lines 1046-1050, the code currently only
adds generatedRequest.requestURLPattern to interimEvent when
request.options.ExportReqURLPattern is true, which prevents interactsh and
operators from accessing the pattern during execution; remove the conditional
and always set interimEvent[ReqURLPatternKey] =
generatedRequest.requestURLPattern so the pattern is available to
interactsh/operators for vulnhash calculation, leaving ExportReqURLPattern in
MakeResultEventItem (operators.go:177) to control final result export.