[MongoDB] Creds not getting detected in latest version 3.78.1 & in older versions, getting verification error: context deadline exceeded

[k-sau]

TruffleHog Version

3.78.1
3.78.0

Trace Output

https://gist.github.com/k-sau/e44f532bd423776ff0c964fe150d2ec1

Expected Behavior

MongoDB creds should be detected

Actual Behavior

In 3.78.1 in mac sillicon chips based machine, it wasn't detecting at all but it is working fine in ubuntu.
In 3.78.0, it's detecting the credentials but failing to validate the credential which are on different region other than yours.

Steps to Reproduce

1. Create dummy MongoDb credential of region which have ideally high latency from your region.
2. Run the trufflehog scan on it from v3.78.1 & v3.78.0 on apple silicon chips machine.
3. In v3.78.1, trufflehog will not even detect the mongodb pattern.
4. And in v3.78.0, you will get: Verification issue: context deadline exceeded.

Environment

• OS: Sonama
• Version 14.5

[yilmi]

@ahrav @dustin-decker - We also have the same issue, this is generating false negatives.

I tested the same set of valid credentials (10 credentials) and scanned them 5 times with the latest version of trufflehog (3.82.6). Each time a different set of credentials came back as verified, depending on which ones encountered the error: Verification issue: context deadline exceeded

MongoDB verification doesn't seem to have an explicit "context deadline" (unless I missed it) as the postgres verifier has, which leads me to think that this may affect other detectors.

This is a problem because this can lead folks to believe that a secret has been successfully rotated while this is not the case.

[rgmz]

MongoDB verification doesn't seem to have an explicit "context deadline" (unless I missed it)

It inherits the timeout from the context, although this wasn't enforced until #3320. FWIW I often find the timeouts too short and manually increase them.

trufflehog/pkg/engine/engine.go

Lines 1051 to 1055 in 5f3b452

	ctx, cancel := context.WithTimeout(ctx, detectionTimeout)
	t := time.AfterFunc(detectionTimeout+1*time.Second, func() {
	ctx.Logger().Error(nil, "a detector ignored the context timeout")
	})
	results, err := data.detector.Detector.FromData(ctx, data.chunk.Verify, matchBytes)

[ahrav]

I initially thought 10 seconds was enough for verification, but this logic likely fails with multiple credentials in a chunk. It would be better to give each detector a configurable context timeout. While it's more complex than using a single timeout across the detector fleet, I prefer moving towards more independent and configurable detectors.

@yilmi, after modifying the detectionTimeout as @rgmz suggested, are you still seeing the context timeout?

@zricethezav, what are your thoughts? @abasit-folio3 @kashifkhan0771, I’d also like your input.

[rgmz]

initially thought 10 seconds was enough for verification, but this logic likely fails with multiple credentials in a chunk.

A single url can easily exhaust 10s. I think 10s per attempt with a much more generous timeout per chunk/detector would reduce the number of false negatives.

[yilmi]

@rgmz / @ahrav - Is the only way to configure this timeout to compile the project myself or can I somwhow override this value?

[ahrav]

@rgmz / @ahrav - Is the only way to configure this timeout to compile the project myself or can I somwhow override this value?

I would like to be able to provide an override. @zricethezav @abasit-folio3 @kashifkhan0771 Could we get your opinions here.

[kashifkhan0771]

I agree with this approach @ahrav

[ankushgoel27]

i am seeing this error even after increasing the timeout to 100s(engine.go) and only having one MongoDB valid credential in file.