Skip to content

feat: Model tool use capability should be auto-detected #5917

@louis-menlo

Description

@louis-menlo

Users now have to edit model to enable tools which is really bad UX. Tool capability should be auto-detected via GGUF chat template.

Current issue:

  • Users must manually edit models to enable tool use
  • Poor user experience requiring manual configuration
  • Tool capability is not automatically recognized

Proposed solution:

  • Auto-detect tool use capability via GGUF chat template
  • Eliminate manual tool configuration steps
  • Improve overall user experience

Edge cases to cover:

  • Users with existing cortex models in their app data folder
  • Import model functionality should also have tool capability auto-detected
  • The initial idea using model catalog doesn't cover these edge cases

Implementation considerations:

  • Parse GGUF chat templates to determine tool support
  • Handle existing models in user directories
  • Ensure imported models are properly detected
  • Provide fallback mechanisms for edge cases

Metadata

Metadata

Labels

No labels
No labels

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions