Skip to content

feature(tunnel): add Tunnel.keep_serving() to keep a tunnel serving - #1053

Merged
kcoopermiller merged 11 commits into
mainfrom
improvement/tunnel-supervisor
Nov 2, 2026
Merged

kcoopermiller merged 11 commits into
mainfrom
improvement/tunnel-supervisor

Conversation

@kcoopermiller

@kcoopermiller kcoopermiller commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds Tunnel.keep_serving(health_path), which keeps a started tunnel serving at its URL. A running frpc and a connected registration don't prove that requests get through: frpc can stay connected while requests through the URL hang or return the "Tunnel not found" page. This checks the URL itself and restarts frpc when it stops working.

It's a coroutine the caller runs as a task, and nothing changes for callers that don't use it.

watch = asyncio.create_task(tunnel.keep_serving("/health"))

try:
    await watch
except TunnelGoneError:
    await tunnel.stop()
    # start a new Tunnel; it gets a new URL

Cancel the task to stop watching. It also ends, without an error, when the tunnel is stopped, and stays ended if the same Tunnel is started again, because that's a new registration.

New public API

One method: Tunnel.keep_serving(health_path: str) -> None. health_path must answer 2xx on the local service, quickly and without side effects. It raises TunnelGoneError when the registration can no longer be used.

How it decides

  1. GET health_path through the URL, every 15 s. A 2xx means it's serving.
  2. The same GET straight to the local service, only when step 1 fails. If that fails too, the tunnel can't be blamed and nothing restarts.
  3. A second GET through the URL 5 s later, which gives frpc time to reconnect by itself. If it fails too, the local service is checked again, so a service that went down in the meantime causes no restart.
  4. restart(), at once the first time, then with doubling delays plus jitter (up to 60 s) while the tunnel stays down. The delay resets when a probe through the URL passes.

An exited frpc skips the probes and goes straight to step 4, once the current 15 s wait ends. A rejected registration surfaces through restart(), which raises TunnelGoneError and ends the rejected frpc.

A deleted registration whose frpc is still connected keeps serving, so it isn't treated as gone until a restart is needed.

Testing

  • Unit: 19 new tests in tests/test_keep_serving.py, against real local HTTP servers and a real child process standing in for frpc. 100 pass in the package on Python 3.11, 3.12 and 3.13. ruff and ty pass.
  • Live, production, one temporary tunnel per run:
Fault Result
frpc killed serving again in 13.2 s
frpc frozen (SIGSTOP) serving again in 38.7 s, new frpc
Local service down for 40 s no restart, same frpc
Registration deleted, frpc still connected keeps serving for 40 s, no restart
frpc killed after the delete TunnelGoneError in 2.7 s, frpc not running
stop() task ends without an error

Not tested live: frpc connected and not frozen but not serving, an expired registration, a network blackhole, and a restart of the tunnel server. The closest live case to the first is the frozen frpc, and the unit tests cover it with a URL that hangs while the local service answers.

Notes for review

  • Noticing an exited frpc can take up to 15 s, because the check only runs between probes. That's the cost of having no separate process watcher.
  • Calling restart() yourself while this runs can cause one extra restart. restart() holds a lock, so the two don't overlap.
  • Each watched tunnel adds one GET to health_path every 15 s.
  • Its log messages go to the prime_tunnel.tunnel logger.

This PR's first version added a supervisor mode to Tunnel, with a state enum, a callback and wait_dead(). It was replaced because callers can only act on one outcome, which is that the registration is gone. See the commit message for details.

🤖 Generated with Claude Code


Note

Medium Risk
Introduces an automated restart loop around frpc and tunnel registration state; misconfigured health paths or races with manual restart/stop could cause extra restarts, though behavior is opt-in and local-down cases are explicitly excluded.

Overview
Adds Tunnel.keep_serving(health_path), an optional long-running coroutine meant to run as an asyncio task. It periodically GETs a health path through the tunnel URL (with basic auth when configured) and only restarts frpc via existing restart() when the public path fails while the local service still returns 2xx—after a grace re-probe so transient blips and local outages do not trigger restarts.

Supporting pieces include probe/backoff timing constants, _restart_delay() with exponential backoff and jitter (capped at 60s), a small _answers() HTTP helper, and logging on the prime_tunnel.tunnel logger. The watch loop ties to a specific tunnel_id, exits cleanly on stop or re-registration, and raises TunnelGoneError when restart() finds the registration unusable.

README documents usage with asyncio.create_task. tests/test_keep_serving.py adds broad coverage (mocked and fake-frpc integration) for happy path, restart conditions, backoff reset, lifecycle, auth/host headers, and TunnelGoneError.

Reviewed by Cursor Bugbot for commit 1f1413e. Bugbot is set up for automated code reviews on this repo. Configure here.

Tunnel(supervise=True) watches the tunnel after start(): the frpc process,
frpc's own events, and a GET through the public URL. A failed GET is blamed
on the tunnel only if the same GET sent straight to the local service
succeeds. A tunnel that stays down is restarted under the same URL, with
backoff and jitter; one whose registration is gone is marked dead.

Adds Tunnel.state (TunnelState), Tunnel.wait_dead() and an on_state_change
callback, which also work without the supervisor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-31T00:38:29.495739Z 1f1413e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 511e7a74dd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

- Cap the backoff exponent, so a very long outage cannot overflow it and
  end the supervisor.
- Record on the tunnel that frpc itself reported the registration gone, so
  a rejection that arrives before the supervisor exists still ends frpc.
- After a backoff, skip the restart if someone else replaced frpc meanwhile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0a6a6ddd4a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

kcoopermiller and others added 2 commits October 12, 2026 10:44
- Deprecate Tunnel.is_gone (use Tunnel.state) and Tunnel.check_registered()
  (use Tunnel.status()). Both keep working and warn.
- Rename TunnelState.CONNECTED to UP, so it is not mistaken for the tunnel
  service's "connected" status.
- Move the probe settings into a Supervision passed as supervise=, so an
  unsupervised Tunnel takes no arguments it ignores.
- Say in is_running's docstring that it only reports the frpc process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A manual restart() that completes while the supervisor's probe is in
flight left it acting on a verdict about the old frpc, and restarting the
new one. The supervisor now notes which frpc a round is about and starts
over if it was replaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 986d98077e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

A failed probe of a frpc that someone then replaced still counted against
the replacement, so one transient failure was enough to restart it. The
supervisor now starts counting again whenever frpc changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96cea27f9f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

kcoopermiller and others added 3 commits October 26, 2026 17:12
…or.stop()

Awaiting the supervisor task directly made a cancelled caller look like a
cancelled task, so the caller's CancelledError was swallowed and stop()
went on with cleanup. Wait with asyncio.wait() instead, which raises only
when the caller itself is cancelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The supervisor added a mode to Tunnel with a state enum, a state callback,
wait_dead() and a settings class, for what is one decision: requests through
the URL fail while the local service works, so restart frpc. Callers can only
act on one outcome, that the registration is gone.

keep_serving(health_path) is a coroutine the caller runs as a task: it raises
TunnelGoneError when the registration is gone and stops when cancelled, so
none of that API is needed. It only uses Tunnel's existing restart(). Requiring
a health path means healthy is a 2xx, which removes the header and
status-comparison rules and a false pass: without one, a 404 from the tunnel
service matched a 404 from a service with no route at "/".

is_gone and check_registered() are no longer deprecated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller kcoopermiller changed the title feature(tunnel): add an opt-in supervisor that keeps a tunnel serving feature(tunnel): add Tunnel.keep_serving() to keep a tunnel serving Oct 30, 2026

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 640635b9ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

keep_serving kept going after stop() and start() on the same Tunnel, watching
the new registration with the old registration's password. Probes then failed
and it restarted frpc over and over. It now ends when the registration it was
started on changes, and checks that again right before a restart.

A local service that went down during the grace period still caused a restart.
The second failed probe now checks the service again first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: e4d1db9d57

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Drop the browser-like User-Agent. Cloudflare never blocked httpx's default
one, and all three user agents tested get through the tunnel edge today.

Shorten the docstring, comments and README section. Merge the two tests for a
failing URL, and drop the ones that repeat another test or only exercise
asyncio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1f22834. Configure here.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1f22834592

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

… step

A task created just before stop() only starts once stop() has set _stopping,
so the not-started guard raised TunnelError during ordinary cleanup. A stopped
tunnel now ends the task normally. Only a tunnel that was never started raises.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kcoopermiller

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1f1413efb6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kcoopermiller
kcoopermiller merged commit 46608e3 into main Nov 2, 2026
27 checks passed
@kcoopermiller
kcoopermiller deleted the improvement/tunnel-supervisor branch November 2, 2026 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants