Add vision/image support for local inference models#8442

Merged

jh-block merged 20 commits into

mainfrom

jhugo/local-inference-multimodal

Apr 13, 2026

Commits on Apr 9, 2026

Fix FFI crash when images are sent to local inference models
jh-block
committed
Enable mtmd feature and manage mmproj files for local vision models
jh-block
committed
Add multimodal image extraction for local inference (Phase 2)
jh-block
committed
Add MtmdContext lifecycle management for local inference vision support
jh-block
committed
feat(local-inference): multimodal prefill for vision-capable models
jh-block
committed
local inference vision: add image token budget estimation and context overflow handling
jh-block
committed
Add Phase 6 tests for local inference vision support
jh-block
committed
Add vision capability indicators to local inference settings UI
jh-block
committed
Backfill vision encoder data for pre-existing local models
jh-block
committed
Fix vision backfill for non-featured quantizations and update placeholder text
jh-block
committed
Poll model list during vision encoder download to reflect progress
jh-block
committed
Lazily initialize vision encoder when mmproj becomes available
jh-block
committed
Fix crash in multimodal prefill: request logits for last token
jh-block
committed
Fix shared mmproj deletion and populate mmproj_size_bytes
jh-block
committed
Namespace mmproj files by repo to avoid cross-model collisions
jh-block
committed
Remove mmproj flat-path migration logic
jh-block
committed
Fix vision_capable not set when mmproj path already matches, deduplicate mmproj downloads
jh-block
committed
Retry failed mmproj downloads and fix stale Completed status
jh-block
committed
Move mmproj enrichment into registry and fix memory cap with large mmproj
jh-block
committed
Cancel mmproj download when model download is cancelled
jh-block
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add vision/image support for local inference models#8442

Add vision/image support for local inference models#8442
jh-block merged 20 commits into
mainfrom
jhugo/local-inference-multimodal

Commits on Apr 9, 2026

Fix FFI crash when images are sent to local inference models

Enable mtmd feature and manage mmproj files for local vision models

Add multimodal image extraction for local inference (Phase 2)

Add MtmdContext lifecycle management for local inference vision support

feat(local-inference): multimodal prefill for vision-capable models

local inference vision: add image token budget estimation and context overflow handling

Add Phase 6 tests for local inference vision support

Add vision capability indicators to local inference settings UI

Backfill vision encoder data for pre-existing local models

Fix vision backfill for non-featured quantizations and update placeholder text

Poll model list during vision encoder download to reflect progress

Lazily initialize vision encoder when mmproj becomes available

Fix crash in multimodal prefill: request logits for last token

Fix shared mmproj deletion and populate mmproj_size_bytes

Namespace mmproj files by repo to avoid cross-model collisions

Remove mmproj flat-path migration logic

Fix vision_capable not set when mmproj path already matches, deduplicate mmproj downloads

Retry failed mmproj downloads and fix stale Completed status

Move mmproj enrichment into registry and fix memory cap with large mmproj

Cancel mmproj download when model download is cancelled