[WARN] Unable to download pom.xml for *jar from Central; this could result in undetected CPE/CVEs.

[proo4509]

Precondition

• [x ] I checked the issues list for existing open or closed reports of the same problem.

Describe the bug
Since today I see this warning/error:
Unable to download pom.xml *.jar from Central
this occurs for all jars and takes for ever (about 10 minutes for each jar)

I did had to change yesterday my config and added -retireJSURL "https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json"
-ossIndexUsername AND --ossIndexPassword

Version of dependency-check used
The problem occurs using version 12.1.0 of the CLI but have been using this versions since it came out. No problems until today.

Log file
When reporting errors, 99% of the time log file output is required. Please post the log file as a gist and provide a link in the new issue.
I will do that when my scan is finished (extremely slow)

To Reproduce
Steps to reproduce the behavior:
running dependency-check-12.1.0\dependency-check\bin\dependency-check.bat

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

[readonlyuser1]

also #7989

[chadlwilson]

Perhaps Maven Central was having issues, although nothing noted on https://status.maven.org/ just yet

You can temporarily disable the analyzer.

[xtermi2]

We see the same issue. Our CI Pipeline was canceled after 200 Minutes. In this time I see 27 Warnings regarding Unable to download pom.xml .... This means there is a timeout of ~9.5 Minutes!

Is there a option to configure the timeout of this analyzer? If not, there should be one, so my pipeline will succeed and the dependency check is maybe not perfect because of missing information, but better than nothing!

[chadlwilson]

If you shorten the timeout, it will simply make it fail faster - but dont see how that helps your pipeline succeed?

If you don't mind the (potential) missing information, disable the analyzer?

[petergphillips]

I raised a similar issue about the timeouts for the gradle plugin in dependency-check/dependency-check-gradle#468.

Our issue is that we use four github action runners to process all of our security jobs. This is needed as each one caches the NVD feed locally. When each one takes ~3h to process a single job then jobs are left waiting and never run. I don't mind the jobs failing if search.maven.org is down, but would prefer them to fail after a few minutes instead of a few hours.

[chadlwilson]

There is a global connection.read.timeout setting which might help, but its default value is 60s so doesn't necessarily explain what's happening here.
edit: looks like there are 7 or 8 attempts defined by analyzer.central.retry.count. Which adds up fast when each is timing out after 60s.

But changing that might affect other analyzers and there is still a multiplicative effect.

At root, what is needed here is proper circuit breaking logic that fast fails every external call for a given host after n consecutive failures.

I can see if it's easy enough to introduce. I fear not consistently as many analyzers use client libraries that hide some of this stuff, or present a crap API (e.g OSSIndex)

[jeremylong]

Currently, the default retry count is 7:

analyzer.central.retry.count=7

The analyzer also does not disable itself after failures - which, due to the current stability, it probably should...

[milco-dot-numan-at-joulz-dot-nl]

We're having the same issue, fortunately we use the Azure Task in a pipeline where we timeout in an hour. Disabling central circumvents the problem for now.

[chadlwilson]

Folks, “me too” comments don’t really add anything to the discussion. Please subscribe to the issue if you want to be notified of replies.

Otherwise check https://status.maven.org/

[ThomasKl-Sasis]

I have the same problem. (me too)

I traced it down to a 502 error when the dependency checker tries to download the artifact from https://search.maven.org/remotecontent?filepath=.

Now if there only was a way to change that URL...

Actually, there is a property that can be used (central.content.url, with a default value of "https://search.maven.org/remotecontent?filepath=") which can be changed to "https://repo1.maven.org/maven2/".

For me changing the URL would work (tested by manually appending ).

But there is another problem: I'm working with the gradle plugin, but I did not find a way to change that property when the dependency checker is running from the gradle plugin.

[chadlwilson]

See #7686. I am dubious about the value of doing so personally, but you could try -Dcentral.content.url=blah (if it works it may be an unsupported mechanism, I dunno)

If you start varying the URLs independently (and randomly) you allow for situations where the search index returns a result but the other place you pointed it to does not have the POM (because it is actually a different store, or behind a different CDN with different cache semantics) and all sorts of other complexities that might evolve as Sonatype evolve the infrastructure.

IMHO this is a rather limited use case where the search is working but the content download is not. Given Central is completely changing at the moment, it probably doesn't make sense to do anything until it's clearer where Sonatype are going?

If Central isn't working for a while, disable it? If that's not good enough for your SLAs, run your own Nexus or Artifactory as a pull-through artifact cache (perhaps paying Sonatype some money for improved reliability in your own Nexus) and enable one of those Analyzers instead of Central.

Even better, avoid scanning jars directly and use Maven/Gradle plugins where there should be no need to search for/download POMs from Central/Artifactory/Nexus (because your build tool has already done so) and which have their own robust local and remote caching approaches for POMs/jars/artifacts.

[proo4509]

If you shorten the timeout, it will simply make it fail faster - but dont see how that helps your pipeline succeed?

If you don't mind the (potential) missing information, disable the analyzer?

No, i really don't want to miss information, but on the other hand, it takes 20 minutes per project now :-(