[JENKINS-72469] Avoid repeated tool downloads from misconfigured HTTP servers

[MarkEWaite]

[JENKINS-72469] Avoid repeated tool downloads from misconfigured HTTP servers

The Azul Systems content delivery network stopped providing the last-modified header in their URL responses. They only provide the ETag header.

Add ETag support (based on the ETag specification) to the Jenkins FilePath URL download method so that if ETag is provided, we use the ETag value. If last-modified is provided and matches, we continue to honor it as well.

JENKINS-72469 has more details.

The community post also includes more details.

Testing done

• Automated test added to FilePathTest for code changes on the controller. The automated test confirms that even without a last-modified value, the later downloads are skipped if a matching ETag is received. The automated test also confirms that download is skipped if OK is received with a matching ETag. No automated test was added to confirm download on the agent because that path is not tested by any of the other test automation of this class.
• Interactive test with the Azul Systems JDK installer on the controller. I created a tool installer for the Azul JDK. I verified that before this change it was downloaded each time the job was run. I verified that after the change it was downloaded only once.
• Interactive test with the Azul Systems JDK installer on an agent. I created a tool installer for the Azul JDK. I verified that before this change it was downloaded each time the job was run. I verified that after the change it was downloaded only once.
• Interactive test on the controller with a file download from an NGINX web server confirmed that the tool is downloaded once and then later runs of the job did not download the file again.

Proposed changelog entries

• Avoid repeated tool downloads from misconfigured HTTP servers.

Proposed upgrade guidelines

N/A

Submitter checklist

Beta Give feedback

1. The Jira issue, if it exists, is well-described.
2. The changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developers, depending on the change) and are in the imperative mood (see examples). Fill in the Proposed upgrade guidelines section only if there are breaking changes or changes that may require extra steps from users during upgrade.
3. There is automated testing or an explanation as to why this change has no tests.
4. New public classes, fields, and methods are annotated with @Restricted or have @since TODO Javadocs, as appropriate.
5. New deprecations are annotated with @Deprecated(since = "TODO") or @Deprecated(forRemoval = true, since = "TODO"), if applicable.
6. New or substantially changed JavaScript is not defined inline and does not call eval to ease future introduction of Content Security Policy (CSP) directives (see documentation).
7. For dependency updates, there are links to external changelogs and, if possible, full differentials.
8. For new APIs and extension points, there is a link to at least one consumer.

Desired reviewers

N/A

Before the changes are marked as ready-for-merge:

Maintainer checklist

Beta Give feedback

1. There are at least two (2) approvals for the pull request and no outstanding requests for change.
2. Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
3. Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
4. Proper changelog labels are set so that the changelog can be generated automatically.
5. If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
6. If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

[mawinter69]

Looks good now

[timja]

/label ready-for-merge

---

This PR is now ready for merge, after ~24 hours, we will merge it if there's no negative feedback.

Thanks!

[MarkEWaite]

Thanks for the approval @timja.

I plan to merge this pull request shortly before I go to sleep tonight so that it will be included in the weekly build tomorrow. I'd like it in at least one weekly before we select the next LTS baseline.