Skip to content

feat(vision): cache vision-proxy image descriptions - #120

Merged
ltmoerdani merged 2 commits into
ltmoerdani:mainfrom
ChauThan:feat/vision-proxy-description-cache
Aug 10, 2026
Merged

feat(vision): cache vision-proxy image descriptions#120
ltmoerdani merged 2 commits into
ltmoerdani:mainfrom
ChauThan:feat/vision-proxy-description-cache

Conversation

@ChauThan

@ChauThan ChauThan commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Caches the text description produced by the vision proxy per image (SHA-256 of its base64 bytes), so images that were already described in earlier turns are reused without calling the vision model again. This saves Copilot quota and latency in multi-turn conversations where text-only models receive the same image attachments on every turn.

What changed

  • New src/visionProxyCache.ts - an in-memory cache keyed by image content hash, capped at 200 entries with FIFO eviction (mirroring the reasoning-content cache).
  • src/extension.ts - proxyVision() now:
    • describes only the message that contains a new image (instead of re-sending the whole conversation), and
    • reuses cached descriptions when all images in a message were already described - no vision-model request at all in that case, so selectChatModels()/sendRequest() are skipped lazily.
    • messages are flattened before the vision pass so the correct description is applied to the right apiMessage when one input message expands into several (e.g. tool results).
  • src/test/visionProxy.test.ts - unit tests for the cache (stable keys, lookup/store, partial hits, reuse, FIFO eviction, clear).
  • package-lock.json - version synced to 0.5.1.

Why

Without a cache, proxyVision() calls the vision model via model.sendRequest() on every turn to describe the same bytes again - wasting Copilot quota, adding latency, and returning a different description each time.

Validation

  • npm run compile passes
  • npm test passes - 168 tests (7 new for the cache)
  • npm run lint:js passes

Issue

Closes #119

Cache the text description produced by the vision proxy per image
(sha-256 of its base64 bytes) so already-described images are reused
on later turns without calling the vision model again - saving Copilot
quota and latency. Only the message containing a new image is sent to
the vision model. The cache is capped at 200 entries with FIFO
eviction. Also sync package-lock.json version to 0.5.1.
@ChauThan
ChauThan marked this pull request as ready for review August 8, 2026 15:05
@ltmoerdani

Copy link
Copy Markdown
Owner

Nice one @ChauThan this is a clean fix. the cache mirrors the reasoning-content cache nicely, and the 7 tests cover the important cases (the partial-hit one especially).

a couple of things i want to double check before merging:

  1. multi-image messages. you store the combined description under every image hash, so if one of those images shows up alone in a later message it gets a description that also mentions the other image. not a correctness issue since a partial hit still forces a re-describe, just confirming that's intended.

  2. the proxy now describes only the message that has a new image instead of the whole conversation. that changes the behavior a bit, descriptions lose some conversation context. makes sense for the token savings, just wanted to flag it.

also, the lockfile sync to 0.5.1 fixes a mismatch that was sitting on main (package.json was already 0.5.1 but the lockfile was still 0.5.0), so that's a welcome side effect. the extra peer: true flags on a few dev deps look like npm version noise, harmless either way.

i'll run compile + the full test suite locally before merging. thanks for the PR.

@ChauThan

ChauThan commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

@ltmoerdani
Thanks for the review!

  1. Yes, that's intentional. I store the same description under every image hash. A partial hit returns nothing, so the message gets described again - no risk of wrong descriptions.

  2. That was not intentional - I didn't think it through carefully. I'm thinking about adding a setting flag so users can choose whether to accept this trade-off or not. Thanks for pointing it out.

I'll do it if you agree.

@ltmoerdani

Copy link
Copy Markdown
Owner

@ChauThan thanks for the follow-up.

  1. good, that matches what i read in the code. partial hit forces a re-describe, so no stale description gets reused.

  2. sure, go ahead with the setting flag. a couple of guardrails so it stays small: keep the cache behavior as the default, gate the whole-conversation mode behind the flag, and add a short docs entry for it. ping me when it's ready and i'll review + test before merging.

@ChauThan

Copy link
Copy Markdown
Contributor Author

@ltmoerdani done - added the setting flag as discussed.

  • opencodego.visionProxyWholeConversation (default false) - off keeps the cache behavior (describe only the message with a new image and reuse cached descriptions), on describes the whole conversation so descriptions keep full context (still cached, more tokens).
  • proxyVision() now takes describeWholeConversation; the whole-conversation branch sends one request over all messages and stores the combined description under every image hash (same no-partial-reuse rule as before).
  • Docs: README settings table + a short section in the vision-proxy feature doc.
  • compile / test (168 pass) / lint:js all green.

Ready for review + test whenever. Thanks!

@ltmoerdani

Copy link
Copy Markdown
Owner

@ChauThan nice one, this rounds it out well. i pulled the branch and ran the checks locally: compile passes, 168 tests pass (0 fail), lint:js and markdown lint on the feature doc are clean. the setting registration, README table, and the feature doc section all line up. merging now, thanks!

@ltmoerdani
ltmoerdani merged commit 8f6cb9f into ltmoerdani:main Aug 10, 2026
2 checks passed
ltmoerdani added a commit that referenced this pull request Aug 11, 2026
Bump version 0.5.1 -> 0.5.2 and finalize CHANGELOG [Unreleased] -> [0.5.2].

Shipped in 0.5.2:
- PR #123 (merged fec411b): DeepSeek V4 multi-turn reasoning_content echo fix
- PR #124 (merged): BYOK group flow fix (#121)
- PR #125 (merged 3001d68): Agents window BYOK bridge + provider remove (#122)
- PR #120 (merged): Vision proxy description cache + whole-conversation mode
- PR #126 (merged 7be0c06): typeof guard + reasoning-history unit tests (follow-up on #123)

Docs sync:
- New issue docs: 55 (PR #123), 56 (PR #120), 57 (PR #124), 58 (PR #125), 59 (PR #126), 60 (release 0.5.2 plan)
- Architecture timeline: +3 entries (#123, #126, reasoning module note)
- Devlog: PR #126 merge entry, release phase update
- Features: agents-window #122 update, vision-proxy #120 enhancement note
- Corrected #55 false claim that #126 was already merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Vision proxy re-describes images on every turn - no cache

2 participants