Skip to content

fix: quiesce peer discovery before shutdown - #9809

Closed
GrapeBaBa wants to merge 2 commits into
ChainSafe:unstablefrom
GrapeBaBa:fix/graceful-network-shutdown
Closed

fix: quiesce peer discovery before shutdown#9809
GrapeBaBa wants to merge 2 commits into
ChainSafe:unstablefrom
GrapeBaBa:fix/graceful-network-shutdown

Conversation

@GrapeBaBa

Copy link
Copy Markdown
Contributor

Motivation

Lodestar can hang during shutdown when new peer discovery or dial work overlaps libp2p connection teardown. This change quiesces peer management before sending goodbye messages and closing libp2p.

Issue reproduction: https://gist.github.com/nflaig/5f41cfc50f38baf5046a034162943dc3

Description

  • Add an idempotent PeerManager.quiesce() that clears peer-management intervals and stops discovery.
  • Call quiesce() before taking the goodbye peer snapshot and disconnecting peers.
  • Keep PeerManager.close() responsible for calling quiesce() so direct close callers retain the same cleanup.
  • Add a shutdown-order unit test.
  • Log remaining TCP handles after libp2p shutdown. This diagnostic is intentionally included because it is part of the exact version under mainnet soak testing.

The direct socket reset/shutdown race is addressed by libp2p/js-libp2p#3596. Both draft PR heads correspond exactly to the image currently under test.

Validation

  • Tested Lodestar commit: d8ab5dbb3eb8623ed0d12319b77d75b707cce490
  • Tested js-libp2p commit: c20c70314f5328ef240f6b061242a313eb38d860
  • Image: lodestar:shutdown-fix-v6-reset-close-race
  • Image ID: sha256:2c2bcfbe1a9adf72cb9f73dfc605fb3a149e283d8aeae6b0325bbefd5e1b13f9
  • Mainnet reproduction loop uses the same container with docker stop -t 60 followed by docker start, with about six minutes of synced soak before shutdown.
  • At PR creation: 45 effective shutdowns completed cleanly, 0 hangs; round 46 is in progress toward 50.
  • pnpm lint
  • pnpm check-types
  • pnpm test:unit (339 files passed, 1 skipped; 3487 tests passed, 9 skipped, 1 todo)

Local checks ran under Node.js 25.2.1 and emitted the repository engine warning requesting Node.js ^24.13.0.

AI Assistance Disclosure

  • External Contributors: I have read the contributor guidelines and disclosed my usage of AI below.

This PR was written primarily by Codex.

@GrapeBaBa

Copy link
Copy Markdown
Contributor Author

Closing in favor of libp2p/js-libp2p#3597. The controlled minimal candidate used unmodified Lodestar at de5f89e with only the js-libp2p socket reset guard and completed 50 clean mainnet shutdowns with 0 hangs, so the Lodestar-side changes are not required.

@GrapeBaBa GrapeBaBa closed this Aug 12, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Lodestar Team Coordination Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant