add max-os-threads flag

[dogancanbakir]

Proposed changes

Closes #5605

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

[Mzack9999]

What do you think if we make this automatic to a reasonable larger amount than the estimated maximum parallelism, for example something like the result of 3 x (concurrency x bulk-size x payload-concurrency + js-concurrency + probe-concurrency)

[dogancanbakir]

We can do that, but I believe overriding it should be an option.

[Mzack9999]

Since this is something valid for all the tools I think we should consider overidding the setting via environment variable and move this logic to the utils package that will evaluate within an init() function in any existing package providing similar functionalities, or eventually creating a new one named global, for example:

$ OS_THREADS=xxx nuclei

What do you think?

[tarunKoyalwar]

if possible i think we should skip adding this flag, don't exactly have a link to quote at the moment but i did read in golang repo or google form that if max threads threshold is being breached either something went wrong at go level (cgo or something else) or a usecase that might not be suitable for go , but almost all cases this setting would not be required to tweak ( even db's do not start 10k threads if i am not wrong )

also recently a contributor reported same issue when i asked if he was able to repro this on different machine it turned out the bug was in the machine / setup and not nuclei ( see: https://github.com/orgs/projectdiscovery/discussions/4767 )

[dogancanbakir]

@Mzack9999 makes sense.

@tarunKoyalwar We focus only on maximizing limits. If we allow users to configure this, they will have the flexibility to increase the maximum limit and set it to a lower value if desired.