Skip to content

Reporting validation #6456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ehsandeep merged 5 commits into from
Sep 5, 2025
Merged

Reporting validation #6456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6b635be
add custom validator for reporting issues
mkrs2404 Aug 29, 2025
1425fdd
Merge branch 'dev' into reporting-validation
mkrs2404 Sep 1, 2025
c7e149d
Merge branch 'dev' into reporting-validation
mkrs2404 Sep 2, 2025
0b6ea85
use httpx dev branch
mkrs2404 Sep 4, 2025
01854c1
remove yaml marshal/unmarshal for validator callback
mkrs2404 Sep 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ require (
github.com/projectdiscovery/gologger v1.1.54
github.com/projectdiscovery/gostruct v0.0.2
github.com/projectdiscovery/gozero v0.1.0
github.com/projectdiscovery/httpx v1.7.1-0.20250901093029-3909667cbaa6
github.com/projectdiscovery/httpx v1.7.1-0.20250902174407-8d6c2658663f
Copy link
Contributor

@coderabbitai coderabbitai bot Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

httpx pseudo-version bump — verify build/tidy and compatibility

Looks fine, but please run mod hygiene and confirm no breakages in dependents.

Suggested checks:

🏁 Script executed:

#!/bin/bash
# Refresh sums and vendor if used
go mod tidy
# Surface potential API changes
go list -m -json github.com/projectdiscovery/httpx | jq '.Version, .Time'
# Quick compile of reporting pkg (touches Options/CreateIssue paths)
go build ./pkg/reporting/...

Length of output: 20309

The directory listing will confirm whether regexes.json is missing in pkg/output/stats/waf. If it’s absent, add the file or adjust the //go:embed pattern accordingly before merging.

🤖 Prompt for AI Agents 
In go.mod around line 98, the reviewer notes that builds referencing
pkg/output/stats/waf expect regexes.json to be embedded; verify whether
pkg/output/stats/waf/regexes.json exists in the repo and if missing either add
the file at that path or update the //go:embed pattern in the Go source to match
the actual filename/location (or include a wildcard) so the file is correctly
embedded during build.

github.com/projectdiscovery/mapcidr v1.1.34
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5
github.com/projectdiscovery/networkpolicy v0.1.21
Expand Down Expand Up @@ -352,7 +352,7 @@ require (
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
github.com/ysmood/fetchup v0.2.3 // indirect
github.com/ysmood/got v0.40.0 // indirect
github.com/yuin/goldmark v1.7.8 // indirect
github.com/yuin/goldmark v1.7.11 // indirect
github.com/yuin/goldmark-emoji v1.0.5 // indirect
github.com/zcalusic/sysinfo v1.0.2 // indirect
github.com/zeebo/blake3 v0.2.3 // indirect
Expand Down
8 changes: 4 additions & 4 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1263,8 +1263,8 @@ github.com/projectdiscovery/gozero v0.1.0 h1:QC+WPEsEVFtPmAm9FiIVT/obv9rF/pS2mna
github.com/projectdiscovery/gozero v0.1.0/go.mod h1:gJUNa8eQgMxLaa0UiLChPTV71/BLLrlPAaUp1C2mrhs=
github.com/projectdiscovery/hmap v0.0.93 h1:iIRdioT4byGJ4Hz5cOjo1fd3HFFi1MUFPv+EGYc1yng=
github.com/projectdiscovery/hmap v0.0.93/go.mod h1:oKgtWo2QMD7BkW25ezYbFCqKO3IctZ2ByEaG2XWW0t0=
github.com/projectdiscovery/httpx v1.7.1-0.20250901093029-3909667cbaa6 h1:LrO3ryyJikuyZmHk/vWlQGnio65gVIqkgQK1+2L80+Q=
github.com/projectdiscovery/httpx v1.7.1-0.20250901093029-3909667cbaa6/go.mod h1:lV4GZJb6TArwq/VT+AdZbysG+Ukneyfr9cXsYTrLbbE=
github.com/projectdiscovery/httpx v1.7.1-0.20250902174407-8d6c2658663f h1:Z73qF1ilD5RUsZP4Px4JImFkCTRlQ5K8uR2gm9eakw8=
github.com/projectdiscovery/httpx v1.7.1-0.20250902174407-8d6c2658663f/go.mod h1:hGAOnHyiq80ASpwbycv8Q32wsfcI8xtADEDw/mKPqTs=
github.com/projectdiscovery/interactsh v1.2.4 h1:WUSj+fxbcV53J64oIAhbYzCKD1w/IyenyRBhkI5jiqI=
github.com/projectdiscovery/interactsh v1.2.4/go.mod h1:E/IVNZ80/WKz8zTwGJWQygxIbhlRmuzZFsZwcGSZTdc=
github.com/projectdiscovery/ldapserver v1.0.2-0.20240219154113-dcc758ebc0cb h1:MGtI4oE12ruWv11ZlPXXd7hl/uAaQZrFvrIDYDeVMd8=
Expand Down Expand Up @@ -1535,8 +1535,8 @@ github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1
github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/goldmark v1.7.1/go.mod h1:uzxRWxtg69N339t3louHJ7+O03ezfj6PlliRlaOzY1E=
github.com/yuin/goldmark v1.7.8 h1:iERMLn0/QJeHFhxSt3p6PeN9mGnvIKSpG9YYorDMnic=
github.com/yuin/goldmark v1.7.8/go.mod h1:uzxRWxtg69N339t3louHJ7+O03ezfj6PlliRlaOzY1E=
github.com/yuin/goldmark v1.7.11 h1:ZCxLyDMtz0nT2HFfsYG8WZ47Trip2+JyLysKcMYE5bo=
github.com/yuin/goldmark v1.7.11/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/yuin/goldmark-emoji v1.0.5 h1:EMVWyCGPlXJfUXBXpuMu+ii3TIaxbVBnEX9uaDC4cIk=
github.com/yuin/goldmark-emoji v1.0.5/go.mod h1:tTkZEbwu5wkPmgTcitqddVxY9osFZiavD+r4AzQrh1U=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
Expand Down
3 changes: 3 additions & 0 deletions pkg/reporting/options.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package reporting

import (
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/exporters/es"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/exporters/jsonexporter"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting/exporters/jsonl"
Expand All @@ -23,6 +24,8 @@ type Options struct {
AllowList *filters.Filter `yaml:"allow-list"`
// DenyList contains a list of denied events for reporting module
DenyList *filters.Filter `yaml:"deny-list"`
// ValidatorCallback is a callback function that is called to validate an event before it is reported
ValidatorCallback func(event *output.ResultEvent) bool `yaml:"-"`
// GitHub contains configuration options for GitHub Issue Tracker
GitHub *github.Options `yaml:"github"`
// GitLab contains configuration options for GitLab Issue Tracker
Expand Down
4 changes: 4 additions & 0 deletions pkg/reporting/reporting.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,10 @@ func (c *ReportingClient) CreateIssue(event *output.ResultEvent) error {
return nil
}

if c.options.ValidatorCallback != nil && !c.options.ValidatorCallback(event) {
return nil
}

Comment on lines +291 to +294
Copy link
Contributor

@coderabbitai coderabbitai bot Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Harden validator callback against panics and add trace on rejection

A faulty user callback can crash reporting. Wrap it with recover and optionally emit a debug when it rejects.

Apply:

-    if c.options.ValidatorCallback != nil && !c.options.ValidatorCallback(event) {
-        return nil
-    }
+    if v := c.options.ValidatorCallback; v != nil {
+        valid := false
+        func() {
+            defer func() {
+                if r := recover(); r != nil {
+                    gologger.Warning().Msgf("reporting: validator panicked: %v", r)
+                    valid = false
+                }
+            }()
+            valid = v(event)
+        }()
+        if !valid {
+            // rejected by validator; skip reporting
+            // gologger.Debug().Msg("reporting: event rejected by ValidatorCallback")
+            return nil
+        }
+    }

Also, double-check that skipping dedupe on validator rejection is intentional (events will be re-evaluated every run).

🏁 Script executed:

#!/bin/bash
file="pkg/reporting/reporting.go"
if [ ! -f "$file" ]; then
  echo "$file not found"
  exit 1
fi

echo "=== Imports ==="
sed -n '1,80p' "$file"

echo "=== Validator snippet context ==="
sed -n '260,330p' "$file"

echo "=== Searching for dedupe in pkg/reporting ==="
grep -R -n "dedupe" pkg/reporting

Length of output: 5797

Harden validator callback against panics and add trace on rejection
Wrap the ValidatorCallback invocation in a defer-recover closure to catch panics (logging with gologger.Warning) and emit a debug log on rejection. The current check at lines 291–294 runs before the dedupe.Index call (around line 297), so rejected events are never indexed—confirm that this ordering is intentional, or move the validation below the dedupe step if you need to dedupe even rejected events.

🤖 Prompt for AI Agents 
In pkg/reporting/reporting.go around lines 291-294, the ValidatorCallback
invocation should be hardened and its rejection traced: wrap the call in a small
defer/recover so any panic is caught and logged via gologger.Warning with the
recovered error and stack, and when the callback returns false emit a debug log
noting the event was rejected (include identifying info). Also review ordering:
currently the validation runs before the dedupe.Index call at ~297 — either keep
that intentional ordering or move the validated call below the dedupe step if
you must dedupe rejected events; implement the panic protection and debug log
regardless of the final placement.

var err error
unique := true
if c.dedupe != nil {
Expand Down
Loading