Skip to content

Comments

Fix data race caused by concurrent access to the request URL during scans#448

Merged
Mzack9999 merged 2 commits intoprojectdiscovery:mainfrom
hdm:bug/url-clone-race
Jul 15, 2025
Merged

Fix data race caused by concurrent access to the request URL during scans#448
Mzack9999 merged 2 commits intoprojectdiscovery:mainfrom
hdm:bug/url-clone-race

Conversation

@hdm
Copy link
Contributor

@hdm hdm commented Jul 15, 2025

A data race was identified when the Nuclei engine calls request.dump() when the net/http client transport for the request is still active (as part of a scan goroutine). This PR changes the Update() function to clone and replace the URL, avoiding the race. This can be tricky to reproduce (might require a HTTP/2 target) and unfortunately I misplaced the original data race log that led to this fix, but it hasn't popped up again since I made this change.

@hdm
Copy link
Contributor Author

hdm commented Jul 15, 2025

Doh, this is still triggering and will need more work, here's race log:

==================
WARNING: DATA RACE
Write at 0x00c064436540 by goroutine 867838:
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).executeRequest()
      project/nuclei/pkg/protocols/http/request.go:869 +0x7170
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1()
      project/nuclei/pkg/protocols/http/request.go:508 +0xa04
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults()
      project/nuclei/pkg/protocols/http/request.go:585 +0x786
  github.com/projectdiscovery/nuclei/v3/pkg/tmplexec/generic.(*Generic).ExecuteWithResults()
      project/nuclei/pkg/tmplexec/generic/exec.go:61 +0x595
  github.com/projectdiscovery/nuclei/v3/pkg/tmplexec.(*TemplateExecuter).Execute()
      project/nuclei/pkg/tmplexec/exec.go:216 +0x876
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2.1()
      project/nuclei/pkg/core/executors.go:138 +0x415
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2.gowrap1()
      project/nuclei/pkg/core/executors.go:145 +0x61

Previous read at 0x00c064436540 by goroutine 868391:
  net/http.newTransferWriter()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/transfer.go:93 +0x3b3
  net/http.(*Request).write()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/request.go:706 +0x1011
  net/http.(*persistConn).writeLoop()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/transport.go:2593 +0x364
  net/http.(*Transport).dialConn.gowrap3()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/transport.go:1945 +0x33

Goroutine 867838 (running) created at:
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2()
      project/nuclei/pkg/core/executors.go:114 +0xd6a
  github.com/projectdiscovery/nuclei/v3/pkg/input/provider.(*SimpleInputProvider).Iterate()
      project/nuclei/pkg/input/provider/simple.go:38 +0x8a
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets()
      project/nuclei/pkg/core/executors.go:79 +0x85b
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateSpray.func1()
      project/nuclei/pkg/core/execute_options.go:137 +0x10a
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateSpray.gowrap2()
      project/nuclei/pkg/core/execute_options.go:138 +0x41

Goroutine 868391 (running) created at:
  net/http.(*Transport).dialConn()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/transport.go:1945 +0x2dd9
  net/http.(*Transport).dialConnFor()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/transport.go:1615 +0x11d
  net/http.(*Transport).startDialConnForLocked.func1()
      /root/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.5.linux-amd64/src/net/http/transport.go:1597 +0x3c
==================

@hdm hdm marked this pull request as draft July 15, 2025 07:13
@hdm
Copy link
Contributor Author

hdm commented Jul 15, 2025

This PR helps, but requires a separate PR in nuclei to resolve:

@hdm hdm marked this pull request as ready for review July 15, 2025 07:44
@Mzack9999 Mzack9999 self-requested a review July 15, 2025 13:14
@Mzack9999 Mzack9999 merged commit 1e3d56a into projectdiscovery:main Jul 15, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants