Skip to content

fix: correct pubsub provider lifecycle - #3038

Closed
endigma wants to merge 3 commits into
mainfrom
jesse/router-11-pubsub-goroutine-leak
Closed

fix: correct pubsub provider lifecycle#3038
endigma wants to merge 3 commits into
mainfrom
jesse/router-11-pubsub-goroutine-leak

Conversation

@endigma

@endigma endigma commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • Bug Fixes
    • Improved graph server hot-reload shutdown so reused graph muxes remain intact and their pubsub providers are not stopped prematurely.
    • Refined pubsub provider lifecycle so startup/shutdown is controlled by the graph mux, improving reliability during server swaps.
  • Tests
    • Added a hot-reload/shutdown test to confirm reused muxes persist into the next server without shutting down their pubsub providers.
    • Strengthened NATS event test assertions to verify per-provider connection/disconnection/close logging counts.

Checklist

Open Source AI Manifesto

This project follows the principles of the Open Source AI Manifesto. Please ensure your contribution aligns with its principles.

@github-actions github-actions Bot added the router label Jul 1, 2026
@endigma
endigma changed the base branch from main to jesse/router-1-reload-memory-leak July 1, 2026 13:32
@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 49dac988-d2cd-44c9-94aa-3d887e0e1493

📥 Commits

Reviewing files that changed from the base of the PR and between 3d5f1edafc4cbbf4d0fa27a70638c24a54c33fa9 and 8deefb9.

📒 Files selected for processing (3)
  • router-tests/events/nats_events_test.go
  • router/core/graph_server.go
  • router/core/graph_server_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • router/core/graph_server_test.go
  • router/core/graph_server.go

Walkthrough

Pubsub provider ownership moves from graphServer to graphMux. Providers now start with mux construction, shut down with mux shutdown, and reuse coverage was expanded to verify retained providers and updated NATS lifecycle logging.

Changes

Router PubSub Lifecycle Refactor

Layer / File(s) Summary
PubSub provider ownership move to graphMux
router/core/graph_server.go
pubSubProviders moves from graphServer to graphMux; providers start in buildGraphMux, are stored on the mux, and shut down in graphMux.Shutdown with a timeout. The server-level shutdown step is removed and the helper path is refactored.
Reuse shutdown coverage
router/core/graph_server_test.go, router-tests/events/nats_events_test.go
Adds a shutdown-tracking provider and server shutdown test for reused mux retention, and updates NATS hot-reload assertions to expect the revised connection and disconnect log counts per provider.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • wundergraph/cosmo#2838: Both PRs modify router/core/graph_server.go around graphMux lifecycle and reuse handling so mux-scoped resources survive server swaps.
  • wundergraph/cosmo#3035: Both PRs refactor pubsub provider startup/shutdown ownership and add shutdown-related coverage for reused mux behavior.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: fixing the pubsub provider lifecycle management.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch jesse/router-11-pubsub-goroutine-leak

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands.

@endigma
endigma force-pushed the jesse/router-11-pubsub-goroutine-leak branch from c972bab to 2d466e1 Compare July 1, 2026 13:35
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

Router-nonroot image scan passed

✅ No security vulnerabilities found in image:

ghcr.io/wundergraph/cosmo/router:sha-4768405956d2e8c93a95fd7fc64a1ea8ec0e97da-nonroot

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.81818% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.39%. Comparing base (3ed663b) to head (084c4b3).

Files with missing lines Patch % Lines
router/core/graph_server.go 81.81% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3038      +/-   ##
==========================================
- Coverage   61.46%   60.39%   -1.07%     
==========================================
  Files         261      261              
  Lines       30596    30594       -2     
==========================================
- Hits        18805    18478     -327     
- Misses      10277    10581     +304     
- Partials     1514     1535      +21     
Files with missing lines Coverage Δ
router/core/graph_server.go 84.40% <81.81%> (-0.99%) ⬇️

... and 25 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Base automatically changed from jesse/router-1-reload-memory-leak to main July 1, 2026 14:11
@endigma
endigma force-pushed the jesse/router-11-pubsub-goroutine-leak branch from 2d466e1 to 440fb1f Compare July 1, 2026 15:29

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
router/core/graph_server_test.go (1)

812-846: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Cover provider context lifetime, not only Shutdown calls.

This test would still pass if reused providers were started with the previous server context and then stopped when prev.Shutdown cancels that context. Consider extending the fake provider to capture its startup context and assert it remains live for reused muxes.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@router/core/graph_server_test.go` around lines 812 - 846, The current
TestGraphServerShutdown only asserts that reuseTrackingProvider.shutdown stays
false after prev.Shutdown, but it does not verify the provider was started with
a context that outlives the previous server. Update the fake provider used in
graphServer/commitReusedMuxes to capture its startup context, then extend the
TestGraphServerShutdown reused-mux case to assert that the provider’s context is
still active after prev.Shutdown, using the existing reuseTrackingProvider,
graphServer, and commitReusedMuxes symbols to locate the right test and helper.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@router/core/graph_server.go`:
- Around line 1543-1547: The pub/sub providers are being started against
s.graphServerCtx and only attached to gm.pubSubProviders after startup, which
breaks reused muxes and leaves partially started providers untracked on failure.
Update buildGraphMux/startupPubSubProviders so providers are registered on the
graph mux before startup and are started with the mux’s own context/lifecycle,
and ensure any startup error path cleans up already-started providers through
gm.pubSubProviders/graphMux.Shutdown handling.

---

Nitpick comments:
In `@router/core/graph_server_test.go`:
- Around line 812-846: The current TestGraphServerShutdown only asserts that
reuseTrackingProvider.shutdown stays false after prev.Shutdown, but it does not
verify the provider was started with a context that outlives the previous
server. Update the fake provider used in graphServer/commitReusedMuxes to
capture its startup context, then extend the TestGraphServerShutdown reused-mux
case to assert that the provider’s context is still active after prev.Shutdown,
using the existing reuseTrackingProvider, graphServer, and commitReusedMuxes
symbols to locate the right test and helper.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2e5cf0f5-1aeb-410b-8366-585c3cecb346

📥 Commits

Reviewing files that changed from the base of the PR and between 1d3b30e and 440fb1f12d580c417538ce0f2ebb7cf5541dfd4d.

📒 Files selected for processing (2)
  • router/core/graph_server.go
  • router/core/graph_server_test.go

Comment thread router/core/graph_server.go
@endigma
endigma force-pushed the jesse/router-11-pubsub-goroutine-leak branch 2 times, most recently from 9855988 to 3d5f1ed Compare July 1, 2026 18:09
@endigma
endigma force-pushed the jesse/router-11-pubsub-goroutine-leak branch from 3d5f1ed to 356c72d Compare July 2, 2026 13:03

@dkorittki dkorittki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Commented some optional things and fixed the failing test

Comment on lines +714 to +717
// pubSubProviders are the EDFS providers built for this mux. They are owned by
// the mux (not the server) so that a mux reused by the next server keeps its
// providers alive: Shutdown skips reused muxes, so their providers are not torn
// down until the mux itself is finally discarded.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo we don't need this comment

Comment on lines -2283 to -2290
func (s *graphServer) shutdownPubSubProviders(ctx context.Context) error {
// Default timeout for pubsub provider shutdown
const defaultShutdownTimeout = 5 * time.Second

return s.providersActionWithTimeout(ctx, func(ctx context.Context, provider datasource.Provider) error {
return provider.Shutdown(ctx)
}, defaultShutdownTimeout, "pubsub provider shutdown timed out")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also remove the startupPubSubProviders method from the graph server. The graph server is not responsible for anything regarding pubsub providers anymore, so imo it makes sense there are no methods related to this on a graph server.

Comment on lines +1032 to +1037
const defaultShutdownTimeout = 5 * time.Second
if pErr := providersActionWithTimeout(ctx, s.pubSubProviders, func(ctx context.Context, provider datasource.Provider) error {
return provider.Shutdown(ctx)
}, defaultShutdownTimeout, "pubsub provider shutdown timed out"); pErr != nil {
err = errors.Join(err, pErr)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea: Have start/stop methods for pubsub providers on graph muxes, which are called here instead of inlined code. Matches what startupPubSubProviders / shutdownPubSubProviders has been but for graph muxes instead of graph servers.

@dkorittki

Copy link
Copy Markdown
Contributor

Further handled via #3047

@dkorittki dkorittki closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants