Skip to content

test(gateway): fix order-dependent telegram-mock flake cluster - #68873

Closed
mehmetkr-31 wants to merge 1 commit into
NousResearch:mainfrom
mehmetkr-31:tests-gateway-telegram-mock-isolation
Closed

test(gateway): fix order-dependent telegram-mock flake cluster#68873
mehmetkr-31 wants to merge 1 commit into
NousResearch:mainfrom
mehmetkr-31:tests-gateway-telegram-mock-isolation

Conversation

@mehmetkr-31

Copy link
Copy Markdown
Contributor

Summary

Eight telegram gateway tests pass in isolation and in their own files but fail in a full tests/gateway run — a long-standing order-dependent flake cluster:

  • test_telegram_slash_confirm.py::TestSendSlashConfirm::test_uses_markdown_v2_and_escapes_special_chars
  • test_telegram_approval_buttons.py (2 tests)
  • test_telegram_model_picker.py (3 tests)
  • test_telegram_network_reconnect.py::test_network_error_classifier_matches_ptb_semantics (2 params)

Root cause

tests/gateway/test_dm_topics.py installed its telegram sys.modules mock unconditionally — no hasattr(sys.modules["telegram"], "__file__") real-library guard like every other telegram test — and force-reimported the adapter against it (sys.modules.pop("plugins.platforms.telegram.adapter")). Its stub also diverged from the shared one in two ways:

  1. It registered a separate string-valued telegram.constants module, so ParseMode.MARKDOWN_V2 was the plain string "MarkdownV2" — its repr() is 'MarkdownV2', which fails assertions like "MARKDOWN_V2" in repr(parse_mode) that expect PTB's StrEnum repr (<ParseMode.MARKDOWN_V2>).
  2. Its telegram.error was a bare MagicMock, so isinstance(exc, telegram.error.TimedOut) checks in test_telegram_network_reconnect.py broke.

Because the stub installed unconditionally and force-reimported the adapter, whichever collection order reached test_dm_topics first leaked that divergent stub into sys.modules for the rest of the session — so later telegram tests asserting real-PTB semantics failed order-dependently while passing alone.

Fix (test-only)

  • test_dm_topics.py now uses the shared tests/gateway/conftest.py::_ensure_telegram_mock (guarded on the real library, comprehensive telegram.error hierarchy) instead of a divergent local stub. It was the only file that unconditionally overwrote the telegram mock.
  • The shared mock now models ParseMode/ChatType as PTB-faithful StrEnum members via a small _FakeEnumMember(str): str(x) and equality return the value ("supergroup") while repr(x) shows the qualified name (<ChatType.SUPERGROUP: 'supergroup'>). That satisfies both the adapter's str(chat.type) normalization in _build_message_event and the tests' repr-based assertions, regardless of which file's mock installed first.

Verification

pytest tests/gateway -k telegram   # 1231 passed, 2 skipped (previously 8 failed in full runs)
pytest tests/gateway/test_dm_topics.py   # 40 passed

A full tests/gateway run drops from 20 order-dependent failures to only the pre-existing non-telegram flakes (test_wecom_callback, test_systemd_notify, test_channel_directory, …) that also fail on pristine main and are unrelated to this change (separate root causes, out of scope here).

🤖 Generated with Claude Code

Eight telegram gateway tests (test_telegram_slash_confirm, _approval_buttons,
_model_picker, _network_reconnect) pass in isolation but fail in a full
tests/gateway run. Root cause: test_dm_topics.py installed its telegram
sys.modules mock UNCONDITIONALLY (no real-library guard) and force-reimported
the adapter against it. Its stub also diverged from every other telegram
test's mock — a separate string-valued telegram.constants module (so
ParseMode members were plain strings, not PTB StrEnum-faithful) and a bare
MagicMock telegram.error (no real exception hierarchy). Whichever order
collected test_dm_topics first, that divergent stub leaked into sys.modules
for the rest of the session, so later tests asserting real-PTB semantics
(ParseMode repr, isinstance against telegram.error, str(ChatType)
normalization) failed.

Fixes:
- test_dm_topics.py now uses the shared tests/gateway/conftest.py
  _ensure_telegram_mock (guarded on the real library, comprehensive error
  hierarchy) instead of a divergent local stub.
- conftest's mock now models ParseMode/ChatType as PTB-faithful StrEnum
  members via a small _FakeEnumMember: str(x)/equality give the value
  ('supergroup') while repr(x) shows the qualified name (<ChatType.SUPERGROUP>),
  so both the adapter's str() normalization and tests' repr assertions agree
  regardless of which file's mock installed first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/test Test coverage or test infrastructure P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation duplicate This issue or pull request already exists labels Jul 21, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #33875: both repair the Telegram mock enum semantics causing order-dependent gateway-test failures; #33875 is the broader open fixture-cluster fix.

teknium1 pushed a commit that referenced this pull request Jul 30, 2026
The file-local telegram mock in test_dm_topics.py installed unconditionally
(no __file__ guard), registered a separate string-valued telegram.constants
module, and force-popped the adapter — poisoning the session for any later
telegram test in the same process (assert 'MARKDOWN_V2' in "'MarkdownV2'").

Fix at the source:
- conftest: _FakeEnumMember(str) with PTB-faithful str()==value and
  repr()==<ChatType.X: 'x'>, satisfying both repr assertions and the
  adapter's str(chat.type) normalization; the same object is bound to
  mod.ParseMode and mod.constants.ParseMode.
- test_dm_topics.py: delete the divergent local mock installer; import the
  shared conftest one.
- release.py: mailmap entry for the author.

Verified: the 5-failure cluster repro (dm_topics + slash_confirm +
approval_buttons + model_picker + network_reconnect + telegram_format in one
process) goes 83/83 green (3x); full tests/gateway single-process run drops
10 -> 5 failed, the remainder being pre-existing discord order-dep failures
out of scope here.

Salvaged from #68873. Credit to @liuhao1024 for the earliest root-cause
diagnosis of this str-enum mock class in PR #33875, two months earlier.

Fixes the telegram-mock order-dependent flake cluster.
teknium1 added a commit that referenced this pull request Jul 30, 2026
…ontributors/emails/

The #68873 salvage re-added a line to the frozen dict; the canonical
mapping (contributors/emails/mehmet.kar@std.yildiz.edu.tr) already
exists from the #68872 salvage.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via the round-3 stability PR #74576 (commit 4673d27, your authorship preserved; the release.py hunk was rerouted to contributors/emails/ since the legacy map is frozen). Your fix landed at the poison source — PTB-faithful _FakeEnumMember in the shared conftest + removal of dm_topics' divergent local mock; the telegram cluster failures are gone (5→0). @liuhao1024 is credited in the commit as the earliest diagnoser of the class (#33875). Thanks @mehmetkr-31!

@teknium1 teknium1 closed this Jul 30, 2026
@teknium1 teknium1 added the sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users label Jul 30, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
The file-local telegram mock in test_dm_topics.py installed unconditionally
(no __file__ guard), registered a separate string-valued telegram.constants
module, and force-popped the adapter — poisoning the session for any later
telegram test in the same process (assert 'MARKDOWN_V2' in "'MarkdownV2'").

Fix at the source:
- conftest: _FakeEnumMember(str) with PTB-faithful str()==value and
  repr()==<ChatType.X: 'x'>, satisfying both repr assertions and the
  adapter's str(chat.type) normalization; the same object is bound to
  mod.ParseMode and mod.constants.ParseMode.
- test_dm_topics.py: delete the divergent local mock installer; import the
  shared conftest one.
- release.py: mailmap entry for the author.

Verified: the 5-failure cluster repro (dm_topics + slash_confirm +
approval_buttons + model_picker + network_reconnect + telegram_format in one
process) goes 83/83 green (3x); full tests/gateway single-process run drops
10 -> 5 failed, the remainder being pre-existing discord order-dep failures
out of scope here.

Salvaged from NousResearch#68873. Credit to @liuhao1024 for the earliest root-cause
diagnosis of this str-enum mock class in PR NousResearch#33875, two months earlier.

Fixes the telegram-mock order-dependent flake cluster.
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
…ontributors/emails/

The NousResearch#68873 salvage re-added a line to the frozen dict; the canonical
mapping (contributors/emails/mehmet.kar@std.yildiz.edu.tr) already
exists from the NousResearch#68872 salvage.
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
The file-local telegram mock in test_dm_topics.py installed unconditionally
(no __file__ guard), registered a separate string-valued telegram.constants
module, and force-popped the adapter — poisoning the session for any later
telegram test in the same process (assert 'MARKDOWN_V2' in "'MarkdownV2'").

Fix at the source:
- conftest: _FakeEnumMember(str) with PTB-faithful str()==value and
  repr()==<ChatType.X: 'x'>, satisfying both repr assertions and the
  adapter's str(chat.type) normalization; the same object is bound to
  mod.ParseMode and mod.constants.ParseMode.
- test_dm_topics.py: delete the divergent local mock installer; import the
  shared conftest one.
- release.py: mailmap entry for the author.

Verified: the 5-failure cluster repro (dm_topics + slash_confirm +
approval_buttons + model_picker + network_reconnect + telegram_format in one
process) goes 83/83 green (3x); full tests/gateway single-process run drops
10 -> 5 failed, the remainder being pre-existing discord order-dep failures
out of scope here.

Salvaged from NousResearch#68873. Credit to @liuhao1024 for the earliest root-cause
diagnosis of this str-enum mock class in PR NousResearch#33875, two months earlier.

Fixes the telegram-mock order-dependent flake cluster.
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
…ontributors/emails/

The NousResearch#68873 salvage re-added a line to the frozen dict; the canonical
mapping (contributors/emails/mehmet.kar@std.yildiz.edu.tr) already
exists from the NousResearch#68872 salvage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have platform/telegram Telegram bot adapter sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants