Skip to content

fix(interactsh): skip DNS lookups on interactsh domains#6614

Merged
ehsandeep merged 2 commits intodevfrom
dwisiswant0/fix/interactsh/skip-DNS-lookups-on-interactsh-domains
Nov 21, 2025
Merged

fix(interactsh): skip DNS lookups on interactsh domains#6614
ehsandeep merged 2 commits intodevfrom
dwisiswant0/fix/interactsh/skip-DNS-lookups-on-interactsh-domains

Conversation

@dwisiswant0
Copy link
Member

@dwisiswant0 dwisiswant0 commented Nov 16, 2025

Proposed changes

fix(interactsh): skip DNS lookups on interactsh domains

to prevent false positives.

Prevents nuclei from resolving interactsh domains
injected in "Host" headers, which would cause
self-interactions to be incorrectly reported as
matches.

Changes:

  • Add GetHostname() method to interactsh.Client
    to expose active server domain.
  • Skip CNAME DNS lookups in
    (*http.Request).addCNameIfAvailable when
    hostname matches the
    (*interactsh.Client).GetHostname.

Fixes #6613

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Summary by CodeRabbit

  • Performance Improvements

    • DNS handling optimized to skip unnecessary CNAME resolution when targeting the configured Interactsh domain, reducing lookup overhead.
  • API Enhancements

    • Exposed a new accessor to retrieve the currently configured Interactsh server hostname.

to prevent false positives.

Prevents nuclei from resolving interactsh domains
injected in Host headers, which would cause
self-interactions to be incorrectly reported as
matches.

Changes:
* Add `GetHostname()` method to `interactsh.Client`
  to expose active server domain.
* Skip CNAME DNS lookups in
  `(*http.Request).addCNameIfAvailable` when
  hostname matches the
  `(*interactsh.Client).GetHostname`.

Fixes #6613

Signed-off-by: Dwi Siswanto <git@dw1.io>
@auto-assign auto-assign bot requested a review from dogancanbakir November 16, 2025 07:48
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 16, 2025

Walkthrough

Adds a public GetHostname() accessor for the Interactsh client and modifies CNAME extraction to short‑circuit DNS lookups when the target hostname is an Interactsh domain, preventing DNS-based CNAME resolution in that case.

Changes

Cohort / File(s) Summary
Interactsh hostname accessor
pkg/protocols/common/interactsh/interactsh.go
Adds public GetHostname() string on Client that delegates to internal getHostname() to expose the configured Interactsh server hostname.
CNAME resolution short-circuit
pkg/protocols/http/request.go
Updates addCNameIfAvailable() to obtain the Interactsh hostname and return early (skip DNS CNAME resolution) when the target hostname ends with the Interactsh domain.

Sequence Diagram(s)

sequenceDiagram
    participant Req as Request flow
    participant Logic as addCNameIfAvailable
    participant Interact as Interactsh client
    participant DNS as DNS resolver

    Req->>Logic: prepare hostname for target
    Note right of Logic: New: call GetHostname()
    Logic->>Interact: GetHostname()
    Interact-->>Logic: interactsh.example.com
    alt target ends with interactsh.example.com
        Logic-->>Req: short-circuit (skip DNS/CNAME)
        Note right of Logic `#CDE5D8`: No DNS lookup performed
    else otherwise
        Logic->>DNS: resolve CNAMEs / perform DNS lookup
        DNS-->>Logic: CNAME/A records
        Logic-->>Req: attach discovered CNAMEs
    end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

  • Areas to check:
    • Correctness of hostname suffix comparison (case sensitivity, trailing dots).
    • That GetHostname() is exported safely and returns expected configured value in all client states (nil/default).
    • Ensure early return does not skip needed CNAME handling for non-Interactsh domains.

Poem

🐰 I hop and fetch the hostname bright,

I show the host so tests see light.
When callbacks wear an Interact hat,
I skip the DNS — no false‑positive chat.
🥕

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding a mechanism to skip DNS lookups when dealing with Interactsh domains.
Linked Issues check ✅ Passed The PR addresses issue #6613 by adding GetHostname() to expose the Interactsh server domain and modifying addCNameIfAvailable to skip DNS lookups for Interactsh domains, directly preventing false positive external-service-interaction detections.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the Interactsh false positive issue: GetHostname() accessor method and DNS lookup short-circuit logic are both necessary for the stated objective.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch dwisiswant0/fix/interactsh/skip-DNS-lookups-on-interactsh-domains

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 009850b and db2e77e.

📒 Files selected for processing (1)
  • pkg/protocols/http/request.go (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/protocols/http/request.go

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 141f34a and 009850b.

📒 Files selected for processing (2)
  • pkg/protocols/common/interactsh/interactsh.go (1 hunks)
  • pkg/protocols/http/request.go (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Lint
🔇 Additional comments (1)
pkg/protocols/common/interactsh/interactsh.go (1)

458-462: LGTM! Clean accessor method.

The new GetHostname() method properly exposes the interactsh server hostname via the existing thread-safe getHostname() method. This enables external components (like the HTTP request handler) to check whether a hostname is an interactsh domain.

Signed-off-by: Dwi Siswanto <git@dw1.io>
@ehsandeep ehsandeep merged commit d5d8d50 into dev Nov 21, 2025
50 of 52 checks passed
@ehsandeep ehsandeep deleted the dwisiswant0/fix/interactsh/skip-DNS-lookups-on-interactsh-domains branch November 21, 2025 15:42
@dwisiswant0 dwisiswant0 mentioned this pull request Dec 4, 2025
dwisiswant0 added a commit that referenced this pull request Dec 4, 2025
* Multi Port Support Added - JS

* minor -changes

* restoring basic sequential multiport support

* better error handling

* feat(openapi/swagger): direct fuzzing using target url

* fix (openapi/swagger): improve error handling and tmpDir cleanup

* fix(openapi/swagger): err shadowing on write failure

* fix(openapi/swagger): remove discarded error in defer

* fix(openapi/swagger): linter and url validation

* fix(openapi/swagger): remove code duplication

* reusing dialer

* removing debug log

* fix: restore parallel processing in workflow & file proto

add missing `go` keyword to anonymous funcs that
were intended to run as goroutines but were
executing synchronously instead.

Fixes #6492

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test: adds `Test(FileProtocol|Workflows)ConcurrentExecution` tests

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(file): satisfy lints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(integration-test): enhance debug mode detects

* replace hardcoded `DEBUG` env var check with
  extensible helper func.
* add support for GitHub Actions Runner env var.
* accept multiple truthy value variants.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(core): race cond in workflow execution

caused by shared context callbacks.

it was exposed after adding concurrent exec to
workflow processing and occurred when multiple
goroutines attempted to write to the same
`ctx.OnResult` callback field simultaneously,
causing data races during workflow template exec.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* introducing workflow sequential mode

* Revert "introducing workflow sequential mode"

This reverts commit 1093bbc.

* refactor(core): keep workflow exec seq

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(core): rm unused tests

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(sdk): configure tmpDir for SDK

Closes #6595.

* docs(sdk): update comment to more accurately reflect purpose

* feat(sdk): add tmpDir configuration option for SDK users

* fix(sdk): init default engine tmpDir when unconfigured

* style(sdk): remove unnecessary else block

* feat(sdk): create parent & tmp dir in WithTemporaryDirectory

* test(cmd): enable `BenchmarkRunEnumeration/Default` bench

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(cmd): collect CPU & heap profiles

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(cmd): satisfy lints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* Merge pull request #6610 from projectdiscovery/feat-result-upload

allow custom id for upload

* feat: write resume file specified by flag

* updating docs

* chore(deps): bump the modules group with 6 updates

Bumps the modules group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger) | `1.1.59` | `1.1.60` |
| [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) | `1.7.2-0.20250911192144-fc425deb041a` | `1.7.2` |
| [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.27` | `0.1.28` |
| [github.com/projectdiscovery/utils](https://github.com/projectdiscovery/utils) | `0.6.1-0.20251030144701-ce5c4b44e1e6` | `0.6.1` |
| [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.54` | `0.2.55` |
| [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.9` | `1.2.10` |


Updates `github.com/projectdiscovery/gologger` from 1.1.59 to 1.1.60
- [Release notes](https://github.com/projectdiscovery/gologger/releases)
- [Commits](projectdiscovery/gologger@v1.1.59...v1.1.60)

Updates `github.com/projectdiscovery/httpx` from 1.7.2-0.20250911192144-fc425deb041a to 1.7.2
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Changelog](https://github.com/projectdiscovery/httpx/blob/dev/.goreleaser.yml)
- [Commits](https://github.com/projectdiscovery/httpx/commits/v1.7.2)

Updates `github.com/projectdiscovery/networkpolicy` from 0.1.27 to 0.1.28
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases)
- [Commits](projectdiscovery/networkpolicy@v0.1.27...v0.1.28)

Updates `github.com/projectdiscovery/utils` from 0.6.1-0.20251030144701-ce5c4b44e1e6 to 0.6.1
- [Release notes](https://github.com/projectdiscovery/utils/releases)
- [Changelog](https://github.com/projectdiscovery/utils/blob/main/CHANGELOG.md)
- [Commits](https://github.com/projectdiscovery/utils/commits/v0.6.1)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.54 to 0.2.55
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](projectdiscovery/wappalyzergo@v0.2.54...v0.2.55)

Updates `github.com/projectdiscovery/cdncheck` from 1.2.9 to 1.2.10
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml)
- [Commits](projectdiscovery/cdncheck@v1.2.9...v1.2.10)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/gologger
  dependency-version: 1.1.60
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/httpx
  dependency-version: 1.7.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/networkpolicy
  dependency-version: 0.1.28
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/utils
  dependency-version: 0.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.2.55
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-version: 1.2.10
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <support@github.com>

* refactor(sdk): don't create parentDir when configuring tmpDir

* adding test case

* lint

* removing unused check

* adding multiport template

* refactor test

* chore(deps): bump golang.org/x/crypto

Bumps the go_modules group with 1 update in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto).


Updates `golang.org/x/crypto` from 0.43.0 to 0.45.0
- [Commits](golang/crypto@v0.43.0...v0.45.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.45.0
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat(variables): check for undefined params for lazy eval (#6618)

* feat(variables): check for undefined params for lazy eval

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(variables): add TestCheckForLazyEval

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(variables): fail safe on err compile expr

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(deps): bump github.com/projectdiscovery/fastdialer@v0.4.16

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(interactsh): skip DNS lookups on interactsh domains (#6614)

* fix(interactsh): skip DNS lookups on interactsh domains

to prevent false positives.

Prevents nuclei from resolving interactsh domains
injected in Host headers, which would cause
self-interactions to be incorrectly reported as
matches.

Changes:
* Add `GetHostname()` method to `interactsh.Client`
  to expose active server domain.
* Skip CNAME DNS lookups in
  `(*http.Request).addCNameIfAvailable` when
  hostname matches the
  `(*interactsh.Client).GetHostname`.

Fixes #6613

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(http): prevent false `interactshDomain` matches

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat: bump dsl with deserialization helpers

* chore: omit unnecessary reassignment (#6622)

Signed-off-by: ledigang <shuangcui@msn.com>

* disable stale workflow for enhancements

* ci: cache go-rod browser (#6640)

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(deps): bump actions/checkout from 5 to 6 in the workflows group

Bumps the workflows group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 5 to 6
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: workflows
...

Signed-off-by: dependabot[bot] <support@github.com>

* do not exempt abandoned issues and prs

* ci: apply free-disk-space on tests

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: bump PD modules & update `httputil` calls (#6629)

* chore(deps): bump the modules group across 1 directory with 11 updates

Bumps the modules group with 11 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.4.16` | `0.4.17` |
| [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) | `0.0.95` | `0.0.96` |
| [github.com/projectdiscovery/retryabledns](https://github.com/projectdiscovery/retryabledns) | `1.0.108` | `1.0.109` |
| [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.0.131` | `1.0.132` |
| [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger) | `1.1.60` | `1.1.61` |
| [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.28` | `0.1.29` |
| [github.com/projectdiscovery/tlsx](https://github.com/projectdiscovery/tlsx) | `1.2.1` | `1.2.2` |
| [github.com/projectdiscovery/useragent](https://github.com/projectdiscovery/useragent) | `0.0.102` | `0.0.103` |
| [github.com/projectdiscovery/utils](https://github.com/projectdiscovery/utils) | `0.6.1` | `0.7.1` |
| [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.55` | `0.2.56` |
| [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.10` | `1.2.11` |



Updates `github.com/projectdiscovery/fastdialer` from 0.4.16 to 0.4.17
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](projectdiscovery/fastdialer@v0.4.16...v0.4.17)

Updates `github.com/projectdiscovery/hmap` from 0.0.95 to 0.0.96
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](projectdiscovery/hmap@v0.0.95...v0.0.96)

Updates `github.com/projectdiscovery/retryabledns` from 1.0.108 to 1.0.109
- [Release notes](https://github.com/projectdiscovery/retryabledns/releases)
- [Commits](projectdiscovery/retryabledns@v1.0.108...v1.0.109)

Updates `github.com/projectdiscovery/retryablehttp-go` from 1.0.131 to 1.0.132
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](projectdiscovery/retryablehttp-go@v1.0.131...v1.0.132)

Updates `github.com/projectdiscovery/gologger` from 1.1.60 to 1.1.61
- [Release notes](https://github.com/projectdiscovery/gologger/releases)
- [Commits](projectdiscovery/gologger@v1.1.60...v1.1.61)

Updates `github.com/projectdiscovery/networkpolicy` from 0.1.28 to 0.1.29
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases)
- [Commits](projectdiscovery/networkpolicy@v0.1.28...v0.1.29)

Updates `github.com/projectdiscovery/tlsx` from 1.2.1 to 1.2.2
- [Release notes](https://github.com/projectdiscovery/tlsx/releases)
- [Changelog](https://github.com/projectdiscovery/tlsx/blob/main/.goreleaser.yml)
- [Commits](projectdiscovery/tlsx@v1.2.1...v1.2.2)

Updates `github.com/projectdiscovery/useragent` from 0.0.102 to 0.0.103
- [Release notes](https://github.com/projectdiscovery/useragent/releases)
- [Commits](projectdiscovery/useragent@v0.0.102...v0.0.103)

Updates `github.com/projectdiscovery/utils` from 0.6.1 to 0.7.1
- [Release notes](https://github.com/projectdiscovery/utils/releases)
- [Changelog](https://github.com/projectdiscovery/utils/blob/main/CHANGELOG.md)
- [Commits](projectdiscovery/utils@v0.6.1...v0.7.1)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.55 to 0.2.56
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](projectdiscovery/wappalyzergo@v0.2.55...v0.2.56)

Updates `github.com/projectdiscovery/cdncheck` from 1.2.10 to 1.2.11
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml)
- [Commits](projectdiscovery/cdncheck@v1.2.10...v1.2.11)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-version: 0.4.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/hmap
  dependency-version: 0.0.96
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryabledns
  dependency-version: 1.0.109
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-version: 1.0.132
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/gologger
  dependency-version: 1.1.61
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/networkpolicy
  dependency-version: 0.1.29
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/tlsx
  dependency-version: 1.2.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/useragent
  dependency-version: 0.0.103
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/utils
  dependency-version: 0.7.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.2.56
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-version: 1.2.11
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update utils.httputil calls

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(deps): bump github.com/projectdiscovery/utils => v0.7.3

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dwi Siswanto <git@dw1.io>

* chore(deps): bump the modules group with 11 updates

Bumps the modules group with 11 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.4.17` | `0.4.18` |
| [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap) | `0.0.96` | `0.0.97` |
| [github.com/projectdiscovery/retryabledns](https://github.com/projectdiscovery/retryabledns) | `1.0.109` | `1.0.110` |
| [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.0.132` | `1.0.133` |
| [github.com/projectdiscovery/dsl](https://github.com/projectdiscovery/dsl) | `0.8.5` | `0.8.6` |
| [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger) | `1.1.61` | `1.1.62` |
| [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.29` | `0.1.30` |
| [github.com/projectdiscovery/uncover](https://github.com/projectdiscovery/uncover) | `1.1.0` | `1.2.0` |
| [github.com/projectdiscovery/useragent](https://github.com/projectdiscovery/useragent) | `0.0.103` | `0.0.104` |
| [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.56` | `0.2.57` |
| [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.11` | `1.2.12` |


Updates `github.com/projectdiscovery/fastdialer` from 0.4.17 to 0.4.18
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](projectdiscovery/fastdialer@v0.4.17...v0.4.18)

Updates `github.com/projectdiscovery/hmap` from 0.0.96 to 0.0.97
- [Release notes](https://github.com/projectdiscovery/hmap/releases)
- [Commits](projectdiscovery/hmap@v0.0.96...v0.0.97)

Updates `github.com/projectdiscovery/retryabledns` from 1.0.109 to 1.0.110
- [Release notes](https://github.com/projectdiscovery/retryabledns/releases)
- [Commits](projectdiscovery/retryabledns@v1.0.109...v1.0.110)

Updates `github.com/projectdiscovery/retryablehttp-go` from 1.0.132 to 1.0.133
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](projectdiscovery/retryablehttp-go@v1.0.132...v1.0.133)

Updates `github.com/projectdiscovery/dsl` from 0.8.5 to 0.8.6
- [Release notes](https://github.com/projectdiscovery/dsl/releases)
- [Commits](projectdiscovery/dsl@v0.8.5...v0.8.6)

Updates `github.com/projectdiscovery/gologger` from 1.1.61 to 1.1.62
- [Release notes](https://github.com/projectdiscovery/gologger/releases)
- [Commits](projectdiscovery/gologger@v1.1.61...v1.1.62)

Updates `github.com/projectdiscovery/networkpolicy` from 0.1.29 to 0.1.30
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases)
- [Commits](projectdiscovery/networkpolicy@v0.1.29...v0.1.30)

Updates `github.com/projectdiscovery/uncover` from 1.1.0 to 1.2.0
- [Release notes](https://github.com/projectdiscovery/uncover/releases)
- [Commits](projectdiscovery/uncover@v1.1.0...v1.2.0)

Updates `github.com/projectdiscovery/useragent` from 0.0.103 to 0.0.104
- [Release notes](https://github.com/projectdiscovery/useragent/releases)
- [Commits](projectdiscovery/useragent@v0.0.103...v0.0.104)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.56 to 0.2.57
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](projectdiscovery/wappalyzergo@v0.2.56...v0.2.57)

Updates `github.com/projectdiscovery/cdncheck` from 1.2.11 to 1.2.12
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Commits](projectdiscovery/cdncheck@v1.2.11...v1.2.12)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-version: 0.4.18
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/hmap
  dependency-version: 0.0.97
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryabledns
  dependency-version: 1.0.110
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-version: 1.0.133
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/dsl
  dependency-version: 0.8.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/gologger
  dependency-version: 1.1.62
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/networkpolicy
  dependency-version: 0.1.30
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/uncover
  dependency-version: 1.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/useragent
  dependency-version: 0.0.104
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.2.57
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-version: 1.2.12
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <support@github.com>

* feat(loader): implement persistent metadata cache (#6630)

* feat(loader): implement persistent metadata cache

for template filtering optimization.

Introduce a new template metadata indexing system
with persistent caching to dramatically improve
template loading perf when filters are applied.
The implementation adds a new index pkg that
caches lightweight template metadata (ID, tags,
authors, severity, .etc) and enables filtering
templates before expensive YAML parsing occurs.

The index uses an in-memory LRU cache backed by
`otter` pkg for efficient memory management with
adaptive sizing based on entry weight, defaulting
to approx. 40MB for 50K templates.
Metadata is persisted to disk using gob encoding
at "~/.cache/nuclei/index.gob" with atomic writes
to prevent corruption. The cache automatically
invalidates stale entries using `ModTime` to
detect file modifications, ensuring metadata
freshness w/o manual intervention.

Filtering has been refactored from the previous
`TagFilter` and `PathFilter` approach into a
unified `index.Filter` type that handles all basic
filtering ops including severity, authors, tags,
template IDs with wildcard support, protocol
types, and path-based inclusion and exclusion. The
filter implements OR logic within each field type
and AND logic across different field types, with
exclusion filters taking precedence over inclusion
filters and forced inclusion via
`IncludeTemplates` and `IncludeTags` overriding
exclusions.

The `loader` integration creates an index filter
from store configuration via `buildIndexFilter`
and manages the cache lifecycle through
`loadTemplatesIndex` and `saveTemplatesIndex`
methods. When `LoadTemplatesOnlyMetadata` or
`LoadTemplatesWithTags` is called, the system
first checks the metadata cache for each template
path. If cached metadata exists and passes
validation, the filter is applied directly against
the metadata without parsing. Only templates
matching the filter criteria proceed to full YAML
parsing, resulting in significant performance
gains.

Advanced filtering via "-tc" flag
(`IncludeConditions`) still requires template
parsing as these are expression-based filters that
cannot be evaluated from metadata alone. The
`TagFilter` has been simplified to handle only
`IncludeConditions` while all other filtering ops
are delegated to the index-based filtering system.

Cache management is fully automatic with no user
configuration required. The cache gracefully
handles errors by logging warnings & falling back
to normal op w/o caching. Cache files use schema
versioning to invalidate incompatible cache
formats across nuclei updates (well, specifically
`Index` and `Metadata` changes).

This optimization particularly benefits repeated
scans with the same filters, CI/CD pipelines
running nuclei regularly, development and testing
workflows with frequent template loading, and any
scenario with large template collections where
filtering would exclude most templates.

* test(loader): adds `BenchmarkLoadTemplates{,OnlyMetadata}` benchs

Signed-off-by: Dwi Siswanto <git@dw1.io>

* ci: cache nuclei-templates index

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore(index): satisfy lints

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(index): correct metadata filter logic

for proper template matching.

The `filter.matchesIncludes()` was using OR logic
across different filter types, causing incorrect
template matching. Additionally, ID matching was
case-sensitive, failing to match patterns like
'CVE-2021-*'.

The filter now correctly implements: (author1 OR
author2) AND (tag1 OR tag2) AND (severity1 OR
severity2) - using OR within each filter type and
AND across different types.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(index): resolve test timing issue

in CI environments.

Some test was failing in CI due to filesystem
timestamp resolution limitations. On filesystems
with 1s ModTime granularity (common in CI),
modifying a file immediately after capturing its
timestamp resulted in identical ModTime values,
causing IsValid() to incorrectly return true.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* ci: cache nuclei with composite action

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(index): file locking issue on Windows

during cache save/load.

Explicitly close file handles before performing
rename/remove ops in `Save` and `Load` methods.

* In `Save`, close temp file before rename.
* In `Load`, close file before remove during error
  handling/version mismatch.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(index): flaky index tests on Windows

Fix path separator mismatch in `TestCacheSize`
and `TestCachePersistenceWithLargeDataset` by
using `filepath.Join` consistently instead of
hardcoded forward slashes.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* test(cmd): init logger to prevent nil pointer deref

The integration tests were panicking with a nil
pointer dereference in `pkg/catalog/loader`
because the logger was not init'ed.

When `store.saveMetadataIndexOnce` attempted to
log the result of the metadata cache op, it
dereferenced the nil logger, causing a crash.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* fix(loader): resolve include/exclude paths

for metadata cache filter.

The `indexFilter` was previously init'ed using raw
relative paths from the config for
`IncludeTemplates` and `ExcludeTemplates`.
But the persistent metadata cache stores templates
using their absolute paths. This mismatch caused
the `matchesPath` check to fail, leading to
templates being incorrectly excluded even when
explicitly included via flags
(e.g., "-include-templates
loader/excluded-template.yaml").

This commit updates `buildIndexFilter` to resolve
these paths to their absolute versions using
`store.config.Catalog.GetTemplatesPath` before
creating the filter, ensuring consistent path
matching against the metadata cache.

Signed-off-by: Dwi Siswanto <git@dw1.io>

* feat(index): adds `NewMetadataFromTemplate` func

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(index): return metadata when `(*Index).cache` is nil

Signed-off-by: Dwi Siswanto <git@dw1.io>

* refactor(loader): restore pre‑index behavior semantics

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>

* chore: bump version

Signed-off-by: Dwi Siswanto <git@dw1.io>

---------

Signed-off-by: Dwi Siswanto <git@dw1.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: ledigang <shuangcui@msn.com>
Co-authored-by: pussycat0x <65701233+pussycat0x@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: tvroi <roy.oswaldha@traveloka.com>
Co-authored-by: Niek den Breeje <n.denbreeje@guardian360.nl>
Co-authored-by: circleous <circleousdev@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Co-authored-by: ledigang <shuangcui@msn.com>
Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] external-service-interaction false positive

3 participants