Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JENKINS-68122] Avoid deadlock involving RingBufferLogHandler.LogRecordRef class loading (III) #6449

Merged
merged 1 commit into from
Apr 9, 2022

Conversation

jglick
Copy link
Member

@jglick jglick commented Apr 7, 2022

See JENKINS-68122. Amends #6018 + #6044. Alternative to #6444 & #6446.

Effectiveness still being validated. Variant of suggestion by @roband7 (hope the GitHub and Jira ids align). I think the problem is in

/**
* Puts the {@link #SLAVE_LOG_HANDLER} into a separate class so that loading this class
* in JVM doesn't end up loading tons of additional classes.
*/
static final class LogHolder {
/**
* This field is used on each agent to record logs on the agent.
*/
static RingBufferLogHandler SLAVE_LOG_HANDLER;
}
private static class SlaveInitializer extends MasterToSlaveCallable<Void, RuntimeException> {
final int ringBufferSize;
SlaveInitializer(int ringBufferSize) {
this.ringBufferSize = ringBufferSize;
}
@Override
@SuppressFBWarnings(value = "ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD", justification = "field is static for the reason explained in the Javadoc for LogHolder")
public Void call() {
SLAVE_LOG_HANDLER = new RingBufferLogHandler(ringBufferSize);
// avoid double installation of the handler. Inbound agents can reconnect to the controller multiple times
// and each connection gets a different RemoteClassLoader, so we need to evict them by class name,
// not by their identity.
for (Handler h : LOGGER.getHandlers()) {
if (h.getClass().getName().equals(SLAVE_LOG_HANDLER.getClass().getName()))
LOGGER.removeHandler(h);
}
LOGGER.addHandler(SLAVE_LOG_HANDLER);

jenkinsci/remoting#527 might offer a hotfix for those who cannot easily change the core (controller) version.

Proposed changelog entries

  • Avoid a deadlock between agent class loading and logging.

Maintainer checklist

Before the changes are marked as ready-for-merge:

  • There are at least 2 approvals for the pull request and no outstanding requests for change
  • Conversations in the pull request are over OR it is explicit that a reviewer does not block the change
  • Changelog entries in the PR title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood
  • Proper changelog labels are set so that the changelog can be generated automatically
  • If the change needs additional upgrade steps from users, upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the PR title. (example)
  • If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

@jglick jglick added regression-fix Pull request that fixes a regression in one of the previous Jenkins releases major-bug For changelog: Major bug. Will be highlighted on the top of the changelog labels Apr 7, 2022
@jglick jglick requested review from basil and timja April 7, 2022 13:48
@timja timja added the on-hold This pull request depends on another event/release, and it cannot be merged right now label Apr 7, 2022
@timja
Copy link
Member

timja commented Apr 7, 2022

marked as on-hold till confirmed it works :)

@roband7
Copy link

roband7 commented Apr 8, 2022

I can confirm this patch works for us. Been running 2.332.2 + this patch on our main Jenkins server today. No hangs, and no other issues.

@jglick jglick added ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback and removed on-hold This pull request depends on another event/release, and it cannot be merged right now labels Apr 8, 2022
@basil
Copy link
Member

basil commented Apr 8, 2022

This PR is now ready for merge. We will merge it after approximately 24 hours if there is no negative feedback. Please see the merge process documentation for more information about the merge process. Thanks!

@basil basil merged commit 442673b into jenkinsci:master Apr 9, 2022
@basil basil deleted the preload branch April 9, 2022 15:22
NotMyFault pushed a commit to NotMyFault/jenkins that referenced this pull request Apr 13, 2022
…ordRef` class loading (jenkinsci#6449)

Co-authored-by: Basil Crow <[email protected]>
(cherry picked from commit 442673b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
major-bug For changelog: Major bug. Will be highlighted on the top of the changelog ready-for-merge The PR is ready to go, and it will be merged soon if there is no negative feedback regression-fix Pull request that fixes a regression in one of the previous Jenkins releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants