Skip to content

docs : add TurboQuant project overview to AGENTS.md and skills - #272

Merged
TheTom merged 1 commit into
TheTom:feature/turboquant-kv-cachefrom
giveen:disclosure
Aug 7, 2026
Merged

docs : add TurboQuant project overview to AGENTS.md and skills#272
TheTom merged 1 commit into
TheTom:feature/turboquant-kv-cachefrom
giveen:disclosure

Conversation

@giveen

@giveen giveen commented Aug 6, 2026

Copy link
Copy Markdown

Overview

  • AGENTS.md: fork overview (turbo/TQ types, key files, env knobs, test gates, git workflow, known pitfalls)
  • skills/code-review: fork-specific TurboQuant checklist
  • skills/add-new-model: fork-specific validation and dedup steps
  • remove REBASE_PLAN.md (planning doc, superseded)

Requirements

- AGENTS.md: fork overview (turbo/TQ types, key files, env knobs,
  test gates, git workflow, known pitfalls)
- skills/code-review: fork-specific TurboQuant checklist
- skills/add-new-model: fork-specific validation and dedup steps
- remove REBASE_PLAN.md (planning doc, superseded)

Assisted-by: DeepSeek V4 Flash
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 6, 2026
@giveen
giveen marked this pull request as ready for review August 6, 2026 02:04
@TheTom

TheTom commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Merging. I fact-checked the claims rather than skimming, because a wrong line in an orientation doc propagates further than a wrong line in a function: it gets believed by everyone who arrives afterwards, including agents.

Everything checkable, checked against the tree:

claim verdict
TURBO2_0 = 43, TURBO3_0 = 44, TQ3_1S = 45, TQ4_1S = 46, TURBO4_0 = 47 correct, including TURBO4_0 at 47 rather than the intuitive 45
turbo types are runtime-only, never in GGUF correct
TQ3_1S/TQ4_1S have CPU, CUDA/HIP, Metal, Vulkan and SYCL kernels correct, SYCL included (25 matching sources)
docs/KV-cache-quantization.md is the authoritative usage doc exists
ggml/src/ggml-cuda/mmvq-tq.cu native TQ kernels exists
MLA models skip V rotation, K and V types must match consistent with the is_mla handling in llama-kv-cache.cpp

The type table getting TURBO4_0 = 47 right is the detail I would expect a hand-written doc to get wrong, since the sequence looks like it should be contiguous. That it survives upstream's Q2_0 claiming 42 is exactly the kind of thing a newcomer needs told.

Removing REBASE_PLAN.md is right; it described a migration that has since happened, and stale plans are worse than absent ones.

One suggestion for a follow-up rather than a change request: the doc would be more useful with a short section on what the test suites do and do not cover, because that has been this fork's most expensive recurring surprise. Three separate times in the past week a green run turned out to be measuring nothing: MUL_MAT_ID passing at n=16 while the decode path was never exercised, SET_ROWS TQ4_1S cases passing vacuously on zero rows, and test-backend-ops reporting OK for an op whose every case was skipped as unsupported (#242). A paragraph saying "a passing suite means the cases that ran passed, check that yours ran" would save the next person the same three discoveries.

Thanks for writing this down. Orientation docs are unglamorous and this one is accurate.

@TheTom
TheTom merged commit 3608b9d into TheTom:feature/turboquant-kv-cache Aug 7, 2026
1 check passed
@giveen
giveen deleted the disclosure branch August 19, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants