fix(gateway): sync MEDIA: directive extension whitelist with extract_local_files - #33522
Closed
liuhao1024 wants to merge 1 commit into
Closed
fix(gateway): sync MEDIA: directive extension whitelist with extract_local_files#33522liuhao1024 wants to merge 1 commit into
liuhao1024 wants to merge 1 commit into
Conversation
…local_files The extract_media() regex in base.py only recognized a subset of file extensions (images, video, audio, common docs), while extract_local_files() accepted a much broader set including .html, .json, .svg, .md, .tar, .gz, and many others. When the model emitted MEDIA:/path/file.html, the regex didn't match, the file was silently dropped, and send_message reported success — causing silent data loss with no log output. This fix syncs the two whitelists by adding the missing extensions to the extract_media regex: bmp, tiff, svg, odt, rtf, md, xls, ods, tsv, json, xml, yaml, yml, ppt, odp, key, tar, gz, tgz, bz2, xz, html, htm. Fixes NousResearch#31137
alt-glitch
added
type/bug
Collaborator
Contributor
Author
|
Closing as duplicate per triage. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
gateway/platforms/base.pyhas two independent file-extension whitelists used for media routing:extract_media()— parses explicitMEDIA:<path>directives from the modelextract_local_files()— finds bare local paths in response textThese whitelists had drifted out of sync.
extract_local_filesaccepts.html,.json,.svg,.md,.tar,.gz,.bmp,.tiff, and many more — but the regex inextract_mediadid not. When the model emittedMEDIA:/path/file.html, the regex didn't match, the file was silently dropped, andsend_messagereportedsuccess: true. Silent data loss with no log output.Fix
Synced the
extract_mediaregex extension group with_LOCAL_MEDIA_EXTSby adding all missing extensions:bmp,tiff,svgodt,rtf,mdxls,ods,tsv,json,xml,yaml,ymlppt,odp,keytar,gz,tgz,bz2,xzhtml,htmKept existing extras not in
_LOCAL_MEDIA_EXTS(epub,opus,apk,ipa) since they're valid media types.Testing
TestExtractMediatests passtest_extract_local_filestests passCode Intelligence
gateway/platforms/base.pyextract_media()(line ~2416) and_LOCAL_MEDIA_EXTS(line ~2458)extract_local_files()uses the same conceptual whitelist; the two must stay in syncFixes #31137