Fix UTF-8 encoding for Chinese search queries in advanced mode #118
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add UTF-8 validation and normalization for API response content - Implement proper encoding handling for advanced search mode - Fix display of Chinese characters in search results content field - Prevent escape sequence artifacts in non-ASCII text responses Resolves GitHub Issue tavily-ai#93 where Chinese queries with search_depth="advanced" returned improperly encoded content displaying as escape sequences.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Automated Changes by SimulateDev
Reviewed by a human @saharmor
Setup
Task
Fix UTF-8 encoding issues causing Chinese search queries to display as escape sequences in advanced search mode. Implement proper encoding validation and normalization for API responses.
Coding agents used
Execution time
⏱️ 15m 49.6s
Summary
This PR resolves GitHub Issue #93 where Chinese search queries using
search_depth="advanced"in the Tavily Python library returned improperly encoded content. The fix implements UTF-8 validation and normalization in the response processing pipeline to ensure Chinese characters display correctly instead of as escape sequences. The solution specifically targets the advanced search mode where this encoding issue was occurring, adding proper UTF-8 handling without affecting other search functionality.What changed?
Review Instructions
Please carefully review all changes before merging. While AI agents are powerful, human oversight is always recommended.
Generated by SimulateDev, the AI coding agents collaboration platform.