Skip to content

fix(gateway): deduplicate Weixin messages by content fingerprint - #16190

Closed
PratikRai0101 wants to merge 1 commit into
NousResearch:mainfrom
PratikRai0101:fix/weixin-message-dedup
Closed

fix(gateway): deduplicate Weixin messages by content fingerprint#16190
PratikRai0101 wants to merge 1 commit into
NousResearch:mainfrom
PratikRai0101:fix/weixin-message-dedup

Conversation

@PratikRai0101

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR resolves an issue where the Weixin (ilinkai) adapter would occasionally deliver the same logical message twice within a short window (~3 seconds). Because the upstream API assigns unique message_id values to these duplicate deliveries, they bypassed the existing ID-based deduplicator, causing the agent to respond twice to the same user query.

The fix implements a secondary deduplication layer in the Weixin adapter that fingerprints messages based on a hash of the sender_id and the text content. This ensures that identical text from the same sender within the deduplicator's 5-minute TTL window is only processed once.

Related Issue

Fixes #16182

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • gateway/platforms/weixin.py: Updated _process_message to calculate an MD5 content fingerprint (content:{sender_id}:{text_hash}) for messages containing text. If this fingerprint is already present in the deduplicator, the message is dropped early.
  • tests/gateway/test_weixin.py: Added a new TestWeixinContentDedup class with regression tests to verify:
    1. Duplicate content with different message IDs is correctly caught and dropped.
    2. Messages without text (media-only) bypass the content-based logic to prevent false positives on consecutive media uploads.

How to Test

  1. Automated Tests: Run the targeted test suite using:
    pytest tests/gateway/test_weixin.py -v
    
  2. Manual Verification: Monitor gateway logs during a Weixin session. Verify that the debug log [Weixin] Content-dedup: skipping duplicate message from ... appears if the upstream API sends a redundant poll response.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix
  • I've run pytest tests/ -q and all relevant tests pass
  • I've added tests for my changes
  • I've tested on my platform: Arch Linux

Documentation & Housekeeping

  • I've updated relevant documentation — N/A
  • I've updated cli-config.yaml.example — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md — N/A
  • I've considered cross-platform impact (Windows, macOS)
  • I've updated tool descriptions/schemas — N/A

Screenshots / Logs

Test Execution Output:

tests/gateway/test_weixin.py::TestWeixinContentDedup::test_duplicate_content_with_different_message_ids_is_dropped PASSED
tests/gateway/test_weixin.py::TestWeixinContentDedup::test_content_dedup_not_called_for_messages_without_text PASSED

======================== 44 passed, 2 warnings in 4.68s =======================

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists platform/wecom WeCom / WeChat Work adapter comp/gateway Gateway runner, session dispatch, delivery labels Apr 26, 2026
@teknium1

teknium1 commented May 4, 2026

Copy link
Copy Markdown
Contributor

Salvaged via #19742 onto current main. Test-file conflict resolved by preserving both the TestIsStaleSessionRet class (from a separate landed fix) and your new TestWeixinContentDedup class. Thanks @PratikRai0101!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/wecom WeCom / WeChat Work adapter type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Weixin adapter: duplicate inbound messages — same content delivered twice with different message_id

3 participants