chore(local-inference): bump llama.cpp gitlink to ABI v13 (token-by-token vision) - #9507
Conversation
…oken vision describe) Points the fused-lib submodule at the fork commit that adds eliza_inference_describe_image_stream_open (ABI 12->13), on top of develop's current pinned base (keeps the iOS fused-slice + diarizer fixes). Activates the JS vision streaming landed in #9289 (it gracefully fell back to one-shot describe until now). Validated on Windows CPU: real eliza-1-0_8b bundle + published mmproj streams token-by-token with OCR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
1 participant
Bumps the fused-lib submodule (
plugins/plugin-local-inference/native/llama.cpp) to the fork commit that adds the ABI-v13eliza_inference_describe_image_stream_open(token-by-token vision describe), applied on top of develop's current pinned base (c9537481) so the iOS fused-slice + diarizer fixes are preserved.This activates the JS vision streaming landed in #9289 — until now develop's lib was v12, so the JS cascade gracefully fell back to the buffered one-shot describe. With the v13 lib,
IMAGE_DESCRIPTIONstreams token-by-token through the same SSE pipe as chat.Validated on Windows CPU with the real eliza-1-0_8b bundle + the published
vision/mmproj-0_8b.gguf: streams token-by-token with accurate OCR ("a red circle with the text ELIZA OCR … a total of $42.00"). Native PR: elizaOS/llama.cpp#33 (merged to fork main).🤖 Generated with Claude Code