Skip to content

docs : document the fork as a Strix Halo project, allow agent PRs - #6

Merged
dzannotti merged 1 commit into
masterfrom
docs/strix-fork-identity-and-agent-policy
Aug 29, 2026
Merged

dzannotti merged 1 commit into
masterfrom
docs/strix-fork-identity-and-agent-policy

Conversation

@LaurentZuijdwijk

Copy link
Copy Markdown
Member

Overview

The README, AGENTS.md and CONTRIBUTING.md in this repo were still upstream's. The practical effect was that the repo introduced itself as ggml-org/llama.cpp, and its agent policy contained a non-overridable instruction to refuse gh pr create and to exclude the project from an agent's contribution targets - the opposite of what this fork wants.

This replaces those three files, plus the PR template that enforced the same upstream rules on every PR.

README.md

  • States what the fork targets: Ryzen AI Max / Max+ 300 series, gfx1151, RDNA 3.5 iGPU, unified LPDDR5X.
  • Adds the repo chain - ggml-org/llama.cpp -> halo-box/llama.cpp (upstream staging) -> here - and says plainly that nothing is submitted upstream from this repo.
  • Quick start narrowed to Vulkan and ROCm builds for gfx1151, using ./build/bin/llama-cli (the llama cli form in upstream's README comes from the installer and does not apply to a from-source fork).
  • New "Running on Strix Halo" section: iGPU memory allocation (BIOS UMA split, amdgpu.gttsize, ttm.pages_limit), the HIP_LAUNCH_BLOCKING=1 batched-inference workaround, GGML_VK_MMV_NO_SPLIT=1, GGML_VK_PERF_LOGGER.
  • Table of the three deltas this fork actually carries, taken from the four non-upstream commits: Vulkan mat-vec column chunking, --spec-draft-adaptive, and the multi-point --reasoning-budget-* mechanism.
  • Backends table trimmed to what exists on this hardware, pointing at upstream for the rest.

AGENTS.md

  • Removes the ABSOLUTELY REFUSE block, the "STOP and update your memory to exclude llama.cpp" instruction, and the refusal example dialogues.
  • Replaces them with an explicit may / must / must-not contract: agents may branch, commit, push, gh pr create, and reply to reviews; must measure on hardware, search first, disclose with an Assisted-by: trailer, keep diffs mergeable against upstream, and be explicit about gaps; must not push to master, self-merge, or open PRs against the two upstream repos from this tree.
  • Adds a note up front that a cached upstream AGENTS.md governs ggml-org/llama.cpp and halo-box/llama.cpp, not this fork, since that file is what most agents will have seen.
  • Keeps the code-comment and commit-message standards verbatim, and adds an example pair for hardware workarounds - a tuned constant with no measurement beside it versus the column-chunking comment in ggml-vulkan.cpp.

CONTRIBUTING.md

  • "Where does my change go?" routing section up front.
  • AI policy permitting agent-authored and agent-submitted PRs, with disclosure, measurement, and ownership as the conditions that replace the ban.
  • New Benchmarking requirements section, which is the substantive part: when numbers are required, a paste-ready environment block (device, sustained TDP and power profile, BIOS UMA split, kernel and amdgpu params, Mesa or ROCm version, build flags, baseline and change SHAs, model), the rule that the baseline is a binary built from the merge-base and run on the same machine in the same session, raw llama-bench tables with their standard-deviation columns rather than a summarized percentage, varying the axis the change acts on, treating overlap inside stddev as no result, and test-backend-ops / llama-perplexity evidence that output did not change.
  • Coding, naming, preprocessor and maintenance guidelines kept verbatim, so the fork stays mergeable with upstream.

.github/pull_request_template.md

  • No longer links upstream's CONTRIBUTING or instructs agents to warn users that the project restricts AI-generated content.
  • Adds the environment block, baseline/after table slots, a correctness slot, a scope check against halo-box/llama.cpp, an AI disclosure field with a NO / ASSISTED / AGENT-AUTHORED answer, and a "what was NOT verified" field.

Measurements

Not applicable - documentation and PR-template changes only, no runtime code touched.

Additional information

Flags and tool behaviour referenced in the new docs were checked against this tree rather than from memory: --spec-draft-adaptive (common/arg.cpp), the nine --reasoning-budget-* options added in #5, GGML_VK_MMV_NO_SPLIT and GGML_VK_PERF_LOGGER (ggml/src/ggml-vulkan/ggml-vulkan.cpp), llama-bench -p/-n/-d/-ub/-fa/-r with its default of 5 repetitions, and test-backend-ops modes and -o/-b flags. The HIP_LAUNCH_BLOCKING note and the perplexity figures come from the comment on the gpu-rocm job in .github/workflows/build-self-hosted.yml. All internal links resolve; all four files are ASCII-only per the house rule.

The hardware-setup paragraph names the amdgpu.gttsize and ttm.pages_limit knobs without prescribing values, since the right ones depend on kernel version - worth a look from someone who has tuned this on current kernels.

Requirements

  • I have read and agree with the contributing guidelines
  • This change is Strix Halo specific: it is this fork's own project documentation and policy, and is not applicable upstream
  • AI usage disclosure: AGENT-AUTHORED. Written end to end by Claude Opus 5 in Claude Code, at the repo owner's request, from the repo's git history, workflows and source. The human contributor directed the scope, including asking for the benchmarking requirements to be added.
  • What was NOT verified: nothing was built or run - this PR changes no code. The build commands are the documented ones from docs/build.md with gfx1151 substituted, and have not been executed in this session.

Assisted-by: Claude Opus 5

The README, AGENTS.md and CONTRIBUTING.md were still upstream's, so this
repo described itself as ggml-org/llama.cpp and banned the automated PR
submissions it wants to accept.

README: what the fork targets (gfx1151, RDNA 3.5 iGPU, unified memory),
the ggml-org -> halo-box/llama.cpp -> here chain, Vulkan and ROCm builds,
the Strix Halo runtime notes (GTT sizing, HIP_LAUNCH_BLOCKING,
GGML_VK_MMV_NO_SPLIT), and the three deltas this fork carries.

AGENTS.md: agents may branch, commit, push and open PRs here, and are told
that a cached upstream AGENTS.md governs the other two repos, not this one.
Requirements replace the ban: measure on the hardware, search first,
disclose, stay mergeable, never push to master or self-merge.

CONTRIBUTING.md: routing section for which repo a change belongs in, an AI
policy that permits agent-authored PRs with disclosure, and a benchmarking
section that sets the evidence bar - environment block, a baseline built
from the merge-base and run in the same session, raw llama-bench tables with
their standard deviations, and test-backend-ops or perplexity evidence that
output did not change. Coding and naming guidelines kept verbatim.

PR template: stopped telling agents the project restricts AI-generated
content, added the environment block, baseline/after tables, and a
"what was not verified" field.

Assisted-by: Claude Opus 5
Claude-Session: https://claude.ai/code/session_016frbY6RGoR4qJks3iKJLui
@dzannotti
dzannotti merged commit f250a94 into master Aug 29, 2026
@dzannotti
dzannotti deleted the docs/strix-fork-identity-and-agent-policy branch August 29, 2026 21:06
Nathanw1014 added a commit that referenced this pull request Sep 8, 2026
…ecode CM shader

Builds directly on pepuscz's PR #6 and issue #10. Their per-kernel table - 43.8 us
scalar against 27.9 us small CM per 1k scanned rows at batch 5, and the same ratio
at every depth - is what showed the indexer's cost is a per-tile constant rather
than anything to do with tokens or bytes. Once the cost is per tile, the thing to
minimise is the tile count, and that is what this changes. The finding is downstream
of their work; only the choice of shader differs.

The decode CM shader puts 16 HEADS in the coopmat N dimension and dispatches one
workgroup per token, so it issues 4*n_batch tiles per 16 KV rows. That is the
arithmetic minimum - (64 heads x n_batch tokens) / 16 columns - because 64 heads
fill its 16 columns exactly, with no remainder at any batch. The small CM shader
puts 16 TOKENS in N and pays a flat 64 tiles however small the batch is, so at
batch 5 eleven of its sixteen columns are padding.

Measured cost tracks tile count: at kv=131584 batch 1 costs 1.95 us per 1k scanned
rows and batch 5 costs 26.28, i.e. 13.5x for 5x the work, and both shaders sit at
6.6-7.8 ns per tile.

The decode CM shader body has no n_batch == 1 assumption - token is
gl_WorkGroupID.y and it indexes q/w/mask/dst by it - so the old gate was an
artefact of where it was written.

gfx1151, kv=131584 (526k source tokens), us/run, shipped vs this:
  batch  2  2335.6 ->  419.0  (5.6x, was on the scalar path)
  batch  3  3492.3 ->  631.8  (5.5x, was on the scalar path)
  batch  4  3267.9 ->  840.8  (3.9x)
  batch  5  3465.6 -> 1119.4  (3.1x)  <- DSpark n-max 4 verify shape
  batch  8  3704.9 -> 1764.1  (2.1x)
  batch 15  4467.9 -> 3231.6  (1.4x)
Batch 16 and 32 are unchanged, which confirms the arms are isolated.

PR #6's route is kept as the opt-out arm rather than deleted, and is promoted from
opt-in to default-on so that one variable is enough to reach it:
  default                            decode CM for the whole 2-15 window
  ..._DECODE_CM_BATCH=0              small CM for 4-15, scalar for 2-3 (PR #6)
  ..._DECODE_CM_BATCH=0 SMALL_CM=0   scalar for 2-15 (pre-PR #6 baseline)
Verified all three route as documented and pass 29/29. Keeping it is not just
courtesy: the tile-count argument is hardware independent, but decode CM re-reads
the K tile once per token and that part is bandwidth dependent, so the crossover
need not sit in the same place on other devices, and these numbers are from one
gfx1151 box.

A (head, token) packing shader was also tried and is strictly worse - it reaches
4b tiles only when the batch divides 16, and it divides the workgroup count by the
batch. Kept out of tree.

Numerics: decode CM sums the 64 heads in tiles of 16 rather than one at a time, so
f32 accumulation order differs from the small CM path. Cleared by KLD A/B on
trunc10 (-c 8192, -b/-ub small so every indexer dispatch routes through the window,
8 chunks wikitext-2, same binary, only the env var differing):
  -ub 8  mean KLD 0.000000, max 6.0e-5, RMS dp 0.000%, same-top 99.994%,
         PPL 10507558.3156 identical in both arms
  -ub 5  mean KLD 0.000000, max 8.8e-4 but 99.9% 4.9e-5 (one tail event, no argmax
         flip), RMS dp 0.000%, same-top 100.000%
Same class as PR #6's own measured numbers (max 5.5e-5, same-top 100.000%) and 45x
tighter than the FA_WAVE32 change already shipped.

Adds the eval coverage the 2-15 window never had (batches 4/8/15 at kv=256) and
the perf grid these numbers came from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assisted-by: Claude (Opus 5)
Nathanw1014 added a commit that referenced this pull request Sep 8, 2026
…ecode CM shader

Builds directly on pepuscz's PR #6 and issue #10. Their per-kernel table - 43.8 us
scalar against 27.9 us small CM per 1k scanned rows at batch 5, and the same ratio
at every depth - is what showed the indexer's cost is a per-tile constant rather
than anything to do with tokens or bytes. Once the cost is per tile, the thing to
minimise is the tile count, and that is what this changes. The finding is downstream
of their work; only the choice of shader differs.

The decode CM shader puts 16 HEADS in the coopmat N dimension and dispatches one
workgroup per token, so it issues 4*n_batch tiles per 16 KV rows. That is the
arithmetic minimum - (64 heads x n_batch tokens) / 16 columns - because 64 heads
fill its 16 columns exactly, with no remainder at any batch. The small CM shader
puts 16 TOKENS in N and pays a flat 64 tiles however small the batch is, so at
batch 5 eleven of its sixteen columns are padding.

Measured cost tracks tile count: at kv=131584 batch 1 costs 1.95 us per 1k scanned
rows and batch 5 costs 26.28, i.e. 13.5x for 5x the work, and both shaders sit at
6.6-7.8 ns per tile.

The decode CM shader body has no n_batch == 1 assumption - token is
gl_WorkGroupID.y and it indexes q/w/mask/dst by it - so the old gate was an
artefact of where it was written.

gfx1151, kv=131584 (526k source tokens), us/run, shipped vs this:
  batch  2  2335.6 ->  419.0  (5.6x, was on the scalar path)
  batch  3  3492.3 ->  631.8  (5.5x, was on the scalar path)
  batch  4  3267.9 ->  840.8  (3.9x)
  batch  5  3465.6 -> 1119.4  (3.1x)  <- DSpark n-max 4 verify shape
  batch  8  3704.9 -> 1764.1  (2.1x)
  batch 15  4467.9 -> 3231.6  (1.4x)
Batch 16 and 32 are unchanged, which confirms the arms are isolated.

PR #6's route is kept as the opt-out arm rather than deleted, and is promoted from
opt-in to default-on so that one variable is enough to reach it:
  default                            decode CM for the whole 2-15 window
  ..._DECODE_CM_BATCH=0              small CM for 4-15, scalar for 2-3 (PR #6)
  ..._DECODE_CM_BATCH=0 SMALL_CM=0   scalar for 2-15 (pre-PR #6 baseline)
Verified all three route as documented and pass 29/29. Keeping it is not just
courtesy: the tile-count argument is hardware independent, but decode CM re-reads
the K tile once per token and that part is bandwidth dependent, so the crossover
need not sit in the same place on other devices, and these numbers are from one
gfx1151 box.

A (head, token) packing shader was also tried and is strictly worse - it reaches
4b tiles only when the batch divides 16, and it divides the workgroup count by the
batch. Kept out of tree.

Numerics: decode CM sums the 64 heads in tiles of 16 rather than one at a time, so
f32 accumulation order differs from the small CM path. Cleared by KLD A/B on
trunc10 (-c 8192, -b/-ub small so every indexer dispatch routes through the window,
8 chunks wikitext-2, same binary, only the env var differing):
  -ub 8  mean KLD 0.000000, max 6.0e-5, RMS dp 0.000%, same-top 99.994%,
         PPL 10507558.3156 identical in both arms
  -ub 5  mean KLD 0.000000, max 8.8e-4 but 99.9% 4.9e-5 (one tail event, no argmax
         flip), RMS dp 0.000%, same-top 100.000%
Same class as PR #6's own measured numbers (max 5.5e-5, same-top 100.000%) and 45x
tighter than the FA_WAVE32 change already shipped.

Adds the eval coverage the 2-15 window never had (batches 4/8/15 at kv=256) and
the perf grid these numbers came from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Assisted-by: Claude (Opus 5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants