fix(gateway): prevent duplicate Photon sidecar storms - #61868
Conversation
teknium1
left a comment
There was a problem hiding this comment.
Thanks for addressing a real Photon multiplexing gap: current gateway/run.py:8621-8643 does not fingerprint _project_secret, while PhotonAdapter stores the resolved secret at plugins/platforms/photon/adapter.py:290-294.
Problems
- The new fingerprint only detects equal secrets. Two profiles with different Photon credentials still bypass the claim check, but both default to
127.0.0.1:8789(plugins/platforms/photon/adapter.py:298-301) and eachconnect()starts a sidecar (plugins/platforms/photon/adapter.py:414-436). That leaves a multiplexed EADDRINUSE path. - The added test hashes a fake secret but does not cover Photon multiplex startup or distinct credentials sharing the default sidecar port.
Suggested changes
- Guard Photon sidecar bind/port collisions independently from credential fingerprints, or require an explicit distinct sidecar port for a secondary Photon profile.
- Add focused coverage for same credentials, distinct credentials on the default port, and distinct ports if supported.
Automated hermes-sweeper review.
| # Photon/Spectrum authenticates with project credentials instead | ||
| # of a bot token. Including its secret keeps multiplexed profiles | ||
| # from spawning competing sidecars for the same account and port. | ||
| "_project_secret", |
There was a problem hiding this comment.
This only catches identical secrets. Photon adapters with different project credentials still bypass this guard, but both default to the same sidecar listener (plugins/platforms/photon/adapter.py:298-301) and start a sidecar in connect(); please also guard the bind/port collision or require a distinct secondary sidecar port.
|
Addressed the automated sweeper feedback with additive follow-up commits on the existing PR branch; the original commit and its same-credential/disconnect fix remain intact. The follow-up now guards Photon sidecar bind/port collisions independently from credentials, allows distinct configured ports, releases claims after failed secondary starts, and reserves retrying-primary ownership so reconnects cannot collide with a secondary. Focused coverage now includes all requested credential/port combinations plus failure/retry lifecycle cases. Final fast-forward-branch validation: 16 passed in |
4cd8e4c to
8c69d1b
Compare
|
Rebased the four Photon commits onto current upstream main and resolved the intervening multiplex configuration/reconnect refactors. The PR is now ready for review and GitHub reports it mergeable with no conflicts. Post-rebase validation: focused multiplex suite 37 passed; adjacent gateway startup/reconnect suites 56 passed; full Ruff passed; Windows footgun scan passed (777 files); py_compile and git diff --check passed. Upstream CI was triggered but is currently action_required and needs maintainer workflow approval. |
|
Merged via #73563 — cherry-picked (incl. your rebase follow-ups) with authorship preserved. First in the train since it was the only gateway/run.py piece. |
What does this PR do?
Prevents multiplexed gateways from launching duplicate Photon sidecars and from marking a healthy shared platform disconnected during duplicate rejection.
When
gateway.multiplex_profilesis enabled, credential-backed platforms can be discovered for every served profile. The duplicate guard recognized bot-token fields but not Photon/Spectrum's_project_secret, so each profile attempted to launch a Node sidecar on the same127.0.0.1:8789listener. The rejection path also calleddisconnect()on freshly constructed adapters that had never connected; for shared-credential adapters that could mutate aggregate platform state and shut down the primary Photon sidecar.This change makes Photon credentials participate in the existing log-safe fingerprint and treats a rejected, never-connected adapter as owning no resources to disconnect.
Related Issue
No tracking issue filed.
Companion: #54565 handles the separate Windows
CREATE_NO_WINDOWhardening for Photon subprocesses. This PR intentionally does not duplicate that contributor's work.Type of Change
Changes Made
gateway/run.pydisconnect()for duplicates rejected beforeconnect().tests/gateway/test_multiplex_adapter_registry.pySweeper Feedback Follow-up
The automated review correctly identified that credential fingerprints alone do not prevent two different Photon projects from binding the same sidecar endpoint. Follow-up commits preserve the original fix and additionally:
connect()ordisconnect().How to Test
gateway.multiplex_profiles: trueand shared Photon credentials.connected.scripts/run_tests.sh tests/gateway/test_multiplex_adapter_registry.py -qpython scripts/check-windows-footguns.py --allruff check .Validation completed:
git diff --check: passed.EADDRINUSE/sidecar-exit events, Telegram + Photon + API server connected, and repeated five-minute watchdog checks returned 0 without a restart storm.Native-Windows wrapper note: the unmodified wrapper currently fails before collection under Git Bash because
env -idrops Windows home variables and the progress renderer uses CP1252. The same wrapper passed unchanged in the Linux container, and the focused tests passed directly in the native Windows environment. The advisory full-treetyscan is also not claimed green:ty 0.0.21panicked on an unrelated existing file and reported the repository's baseline diagnostics.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests pass — focused wrapper/native suites passed; full matrix is delegated to CIDocumentation & Housekeeping
docs/, docstrings) — N/A; behavior is internal and documented inlinecli-config.yaml.exampleif I added/changed config keys — N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/AScreenshots / Logs
Before, one restart produced ten secondary Photon sidecar exits on the shared port and duplicate cleanup left aggregate platform state disconnected. After the fix: