Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 13, 2026

Implementation Plan: Llama 3.2 Vision Docker with Flask Webhook

  • Create new directory llama-vision for the project
  • Create Dockerfile for llama.cpp with CPU-only support
  • Create Flask webhook application with inference and health endpoints
  • Implement response schema for structured output
    • Created response_schema.json with formal JSON schema
    • Python models.py for internal validation with Literal types
  • Create requirements.txt with secure dependency versions
  • Add environment variable support for flexible configuration
  • Create comprehensive README with usage instructions
  • Add docker-compose.yml for easy deployment
  • Create Makefile for convenient build/run commands
  • Add test script for API validation
  • Add .gitignore for models and artifacts
  • Update main README to include llama-vision
  • Add proper CLIP model support for vision features
  • Fix security and resource handling issues
  • Address code review feedback
  • Refactor schema to proper JSON format
  • Add enum validation for consistency

Latest Changes:

  • Added Literal types to ErrorResponse.error_type and HealthResponse.status
  • Ensures Python runtime validation matches JSON schema enum constraints
  • Updated docstring to clarify models.py is for validation, not schema definition

Complete Implementation:
All requirements met with proper schema definition (JSON), runtime validation (Pydantic with type constraints), and comprehensive documentation.

Original prompt

Setup docker image that will run llama 3.2 vision (use env var for model) on cpu (no gpu, nvidia acceleration), use recommended quantization based on comments in ggml-org/llama.cpp#5780

Expose the prompt through a flask webhook interface accept images and text in prompts and provide a response via json

Prepare a schema for the response format


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 4 commits January 13, 2026 21:31
Co-authored-by: wpowiertowski <671688+wpowiertowski@users.noreply.github.com>
Co-authored-by: wpowiertowski <671688+wpowiertowski@users.noreply.github.com>
Co-authored-by: wpowiertowski <671688+wpowiertowski@users.noreply.github.com>
Co-authored-by: wpowiertowski <671688+wpowiertowski@users.noreply.github.com>
Copilot AI changed the title [WIP] Setup Docker image for Llama 3.2 vision with Flask webhook Add Llama 3.2 Vision inference service with CPU-only Q4_K_M quantization Jan 13, 2026
Copilot AI requested a review from wpowiertowski January 13, 2026 21:42
Copy link
Owner

@wpowiertowski wpowiertowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema file should be a json representing the expected response format but looks like you implemented some model logic there

- Add response_schema.json with formal JSON schema definitions
- Rename schema.py to models.py to clarify it contains internal validation logic
- Update README to reference the JSON schema file
- Update documentation with JSON examples instead of Python type hints

Co-authored-by: wpowiertowski <671688+wpowiertowski@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 13, 2026

Schema file should be a json representing the expected response format but looks like you implemented some model logic there

Fixed in b49c532. Created response_schema.json with formal JSON Schema definitions and renamed the Python file to models.py to clarify it's for internal validation logic, not the schema definition.

- Use Literal types for error_type and status fields
- Ensures runtime validation matches JSON schema constraints
- Update docstring to reference response_schema.json

Co-authored-by: wpowiertowski <671688+wpowiertowski@users.noreply.github.com>
@wpowiertowski wpowiertowski marked this pull request as ready for review January 13, 2026 22:21
@wpowiertowski wpowiertowski merged commit 8105e3f into main Jan 13, 2026
@wpowiertowski wpowiertowski deleted the copilot/setup-docker-image-llama-vision branch January 17, 2026 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants