fix race condition

[dogancanbakir]

Proposed changes

Closes #5534

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

[dogancanbakir]

perf test passes https://github.com/projectdiscovery/nuclei/actions/runs/10460146436

[Mzack9999]

I've tried to execute locally and it seems like we have other few race conditions, for example:

==================
WARNING: DATA RACE
Write at 0x000111024d50 by goroutine 51:
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate.Close()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/common/protocolstate/state.go:217 +0x84
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolinit.Close()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/common/protocolinit/init.go:41 +0x1c4
  github.com/projectdiscovery/nuclei/v3/internal/runner.(*Runner).Close()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/internal/runner/runner.go:380 +0x1bc
  main.main.func4()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/cmd/nuclei/main.go:165 +0x11c

Previous read at 0x000111024d50 by goroutine 71255:
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool.wrappedGet()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/http/httpclientpool/clientpool.go:236 +0x764
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool.Get()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/http/httpclientpool/clientpool.go:154 +0x124
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).executeRequest()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/http/request.go:787 +0x28da
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1()

I think we should also consider in a follow up issue to disable the panic-recover mechanism in GitHub Actions, where not necessary. Otherwise, errors are not detected. The app should just crash to ensure failures are visible.

[dogancanbakir]

I've tried to execute locally and it seems like we have other few race conditions, for example:

==================
WARNING: DATA RACE
Write at 0x000111024d50 by goroutine 51:
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate.Close()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/common/protocolstate/state.go:217 +0x84
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolinit.Close()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/common/protocolinit/init.go:41 +0x1c4
  github.com/projectdiscovery/nuclei/v3/internal/runner.(*Runner).Close()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/internal/runner/runner.go:380 +0x1bc
  main.main.func4()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/cmd/nuclei/main.go:165 +0x11c

Previous read at 0x000111024d50 by goroutine 71255:
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool.wrappedGet()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/http/httpclientpool/clientpool.go:236 +0x764
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool.Get()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/http/httpclientpool/clientpool.go:154 +0x124
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).executeRequest()
      /Users/user/go/src/github.com/projectdiscovery/nuclei/pkg/protocols/http/request.go:787 +0x28da
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1()

I think we should also consider in a follow up issue to disable the panic-recover mechanism in GitHub Actions, where not necessary. Otherwise, errors are not detected. The app should just crash to ensure failures are visible.

I couldn't repro this, but I removed the redundant nil assignment here: 35a0d67. Could you re-run and see if it happens again?