Skip to content

fix(gateway): stop stale Telegram typing refreshes - #29919

Closed
zhonghui5207 wants to merge 1 commit into
NousResearch:mainfrom
zhonghui5207:fix/telegram-typing-lifecycle
Closed

fix(gateway): stop stale Telegram typing refreshes#29919
zhonghui5207 wants to merge 1 commit into
NousResearch:mainfrom
zhonghui5207:fix/telegram-typing-lifecycle

Conversation

@zhonghui5207

Copy link
Copy Markdown

Summary

  • add per-chat typing generation leases so stale _keep_typing loops exit before refreshing platform typing indicators
  • use a dedicated typing stop event instead of sharing the session interrupt event
  • avoid re-triggering Telegram typing after final notified responses while preserving progress-message typing refreshes

Test Plan

  • ./venv/bin/python -m pytest tests/gateway/test_keep_typing_timeout.py tests/gateway/test_telegram_thread_fallback.py tests/gateway/test_telegram_network.py tests/gateway/test_telegram_noise_filter.py -q

Notes

This addresses cases where Telegram can keep showing typing after the final response because a stale keep-typing task survived cleanup or because send() refreshed typing after the final notified response.

Give each typing loop a per-chat generation lease so stale tasks cannot keep refreshing chat actions after cleanup or newer turns take ownership. Avoid re-triggering Telegram typing after final notified responses while preserving progress-message typing refreshes.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter P2 Medium — degraded but workaround exists labels May 21, 2026
@peter20201011-cmyk

Copy link
Copy Markdown

I ran into the same failure mode on a local macOS gateway and did a small health check against current origin/main (5e6749fbf).

Observed symptom:

  • Telegram kept showing the bot as typing after the final response was already delivered.
  • Gateway state had active_agents: 0.
  • The process registry was empty.
  • The latest gateway log had already reached response ready / Sending response, with no new inbound message.

That made it look like an orphaned typing refresh loop rather than an active agent. Current origin/main still starts _keep_typing() with only the shared stop_event, so a stale refresher has no independent ownership/lease check once turn ownership is released or transferred.

This PR’s generation lease approach looks like the right direction for that class of bug. One extra regression that may be worth adding is the dynamic case where the typing loop starts as the current owner, then loses ownership while it is sleeping:

@pytest.mark.asyncio
async def test_invalidated_typing_generation_stops_running_loop(monkeypatch):
    adapter = _StubAdapter()
    calls = []

    async def recording_send_typing(chat_id, metadata=None):
        calls.append(chat_id)

    monkeypatch.setattr(adapter, "send_typing", recording_send_typing)
    adapter.stop_typing = MagicMock(return_value=asyncio.sleep(0))
    adapter._typing_generations["chat"] = 1

    task = asyncio.create_task(
        adapter._keep_typing(
            chat_id="chat",
            interval=0.2,
            typing_generation=1,
        )
    )
    await asyncio.sleep(0.45)
    assert calls

    adapter._typing_generations["chat"] = 2
    await asyncio.wait_for(task, timeout=1.0)

    calls_after_exit = len(calls)
    await asyncio.sleep(0.4)
    assert len(calls) == calls_after_exit

As a cross-check, I tested an alternative minimal session-guard ownership patch on top of origin/main and ran:

python -m pytest \
  tests/gateway/test_keep_typing_timeout.py \
  tests/gateway/test_pending_drain_no_recursion.py \
  tests/gateway/test_pending_drain_race.py \
  tests/gateway/test_session_split_brain_11016.py \
  -q

Result:

26 passed

So the issue is reproducible as an upstream typing lifecycle gap, and this PR appears to target the right area. I would mainly suggest making sure the test coverage includes a typing loop that becomes stale after it has already started, not only one that is stale before the first refresh.

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Thanks for this. A fix for the post-delivery typing-stop boundary in base.py has landed on main via #37556 (merged as 6a30cfca8): it moves _stop_typing_task() ahead of the post-delivery callback and bounds the callback with a timeout, which also closes the root-cause issue #24971.

This PR overlaps that area but also touches paths #37556 didn't (e.g. the stream-consumer / final-delivery path), so I'm not closing it — could you rebase on current main and let me know if there's still a residual gap your change covers after #37556? Happy to take the remaining delta as a focused follow-up if so.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused typing-lifecycle work. Automated hermes-sweeper review found that the requested behavior is now implemented on main.

  • 565b7c8d9d879c6423c55e9be84596936bc489ba gates Telegram's post-send typing refresh on final metadata["notify"] replies in both rich and normal send paths; see plugins/platforms/telegram/adapter.py:3596 and plugins/platforms/telegram/adapter.py:3839.
  • 331cb38e21affee3527dbe5a646ac6d5e25f2996 introduced bounded base cleanup which pauses a chat before cancelling a typing task and performs a final bounded stop; see gateway/platforms/base.py:3887 and gateway/platforms/base.py:5293.
  • Current regression coverage includes late-cancel refresh prevention in tests/gateway/test_keep_typing_timeout.py:203 and typing shutdown before hung post-delivery callbacks in tests/gateway/test_run_progress_topics.py:1153.

The later mainline implementation covers the PR's final-response and stale-refresh guarantees, so this is redundant.

@teknium1 teknium1 closed this Jul 13, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants