Catalog Laguna XS 2.1 and S 2.1 - #27
Merged
Merged
Conversation
Two rows for Poolside's Laguna 2.1 family, the first models in the catalog published after 2025-07-31. Each of them exercises part of the schema that no existing row did. Attribution. Unsloth mirrors S 2.1 as GGUF and is where most people meet the model, so `unsloth/...` reads as the obvious repo. It is not: canonical_repo names the original publisher, and Unsloth does not mirror XS 2.1 at all, so the mirror is not even a consistent answer. Both rows name Poolside, and a test rejects a fact_source pointing at the mirror. Active parameters. The names say A3B and A8B; config.json says 2.7B and 7.8B. The speed score is built on the active count, so a row that copies the name claims the model is slower than it is. The formula is written above catalog_rows() and both figures reproduce the safetensors total to within 1.5%. UD quants. Unsloth's dynamic quants allocate bits per tensor and are not the plain quant they are named after -- UD-Q2_K_XL is 2.70 bpw where plain Q2_K is 3.35. Reusing the plain figure would misestimate Laguna S by roughly 75 GB, in the direction that says it fits. The four values this repo references are measured from the published files; an unlisted UD quant still returns status 1, so catalog_est_size_mb fails loudly rather than sizing a 70 GB download off an invented number. Both estimates now agree with reality: 20,165 MB estimated against 20,274 measured for XS at Q4_K_M, 73,353 against 73,395 for S at UD-Q4_K_XL. A test holds them to 3%. XS 2.1 has no quant fallback because Poolside publishes exactly two GGUFs of it. An invented IQ4_XS alternative would make select_quant_file match some other file instead of failing. Ratings stay unknown, and that is the interesting case rather than the default one: Poolside ships .eval_results/swe-bench_verified.yaml claiming 70.9% for XS 2.1. No other row here has a SWE-bench figure, so recording it would rank "published a number" against "did not" while looking like a quality judgement. CATALOG_MAX_ROWS 15 -> 17, with the date and reason in the comment. SELECT_MAX_LISTED moves with it, because selector_build drops everything past its cap silently -- a catalogued model that never reaches the menu is invisible in the one place a user would look. A test now enforces that the two agree. Two score tests were asserting a fixed list of three size classes, which was really an assertion that no catalogued model is unrunnable at 20 GB with no offload. That stopped being true, and it failed in the grouping tests rather than where the assumption lived. Both now assert the documented invariant -- each class once, contiguous, unsupported last -- so they hold whatever the catalog contains. Closes #26 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
rammsguns
commented
Aug 12, 2026
rammsguns
left a comment
Owner
Author
There was a problem hiding this comment.
PM review: product decisions accepted.
- Keep
CATALOG_MAX_ROWSandSELECT_MAX_LISTEDat 17. Adding the two current, relevant rows is preferable to silently retiring useful coverage; revisit retirement before the catalog exceeds roughly 20 rows. - Keep Laguna XS 2.1 and S 2.1 eligible for normal selection. Do not suppress or hand-penalize XS merely because metadata currently makes it the top medium pick. The unfinished local benchmark is the correct place to add comparable quality evidence.
- Keep both ratings
unknown; the isolated vendor SWE-bench disclosure is not comparable with the rest of this catalog. - No model download or runtime test is required for this issue. The PR clearly distinguishes sourced configuration/GGUF-byte facts from measurements not performed locally.
The acceptance criteria are covered and CI is green. One wording nit, not a blocker: the selector regression test enforces SELECT_MAX_LISTED >= CATALOG_MAX_ROWS, which is the safety invariant, rather than strict equality as the PR summary says.
Ready to merge from the PM side. Merge this before the TUNING.md queue because it does not participate in that conflict chain.
rammsguns
added a commit
that referenced
this pull request
Aug 13, 2026
One conflict, in README.md, and it was an insertion collision rather than a disagreement: this branch adds "Rating the models you serve" immediately before "## Configuration", and #25 adds "The llama-swap binary is pinned and verified" in the same place. Neither edits the other's text. Both sections are kept, ratings first. The ratings section continues the catalog thread the surrounding prose is already on -- it opens by answering "a quarter of the score is a neutral placeholder", which is the sentence two sections above it -- and the llama-swap section is about 40-serve.sh, so it reads as the last thing before Configuration rather than an interruption. Nothing was dropped from either side. Everything else merged clean, including the two places most likely to have collided: catalog_ratings() in lib/catalog.sh, where #27 added two Laguna rows carrying unknown ratings while this branch changes how ratings are produced, and tests/cases/catalog_test.sh.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #26.
Adds
laguna-xs-2.1andlaguna-s-2.1to the catalog withhf-apiprovenance dated 2026-08-12.What was verified, and how
Every figure is one
curlaway from thefact_sourcein the row:createdAtmax_position_embeddingsopenmdw-1.1openmdw-1.1toolsandreasoningcome from the chat template, which carriestool_call,tools,reasoning_contentandenable_thinking.The estimator, checked against real bytes
This is the part I'd most like reviewed, because it is the number every fit decision runs on:
laguna-xs-2.1Q4_K_Mlaguna-s-2.1UD-Q4_K_XLMeasured = summed GGUF file bytes from the published repos on 2026-08-12. A test holds both to 3%.
Behaviour change worth a decision
Laguna XS 2.1 becomes the top
mediumpick on this machine, ahead ofqwen3-coder-30b(81 vs 73), and Laguna S 2.1 becomes the toplargepick. Nothing was weighted to make that happen. With every coding rating stillunknownthe quality term does no discriminating work, so the ranking runs on freshness, hardware fit, speed and features — and a June 2026 MoE with a 262k context wins all four against a July 2025 one.That is the existing design behaving as designed. It is also an argument for finishing the local benchmark rather than for hand-weighting the table. Say so if you'd rather the new rows not be default picks and I'll change it.
Three judgement calls, flag any of them
CATALOG_MAX_ROWS15 → 17 rather than retiring two rows. The comment now carries the date and reason, and says that past ~20 the answer is to retire instead. Retiringphi-4andqwen3-1.7bwas the alternative; that is a product call I didn't want to make silently.unknowndespite Poolside publishing 70.9% SWE-bench Verified for XS. Reasoning in the table comment. This is the first time the rule has cost us a real number.SELECT_MAX_LISTEDraised to match. Left at 15, the two lowest-scoring rows would have vanished from the menu with no message. A test now enforces that the caps agree.Not done
Neither model has been downloaded or run here — that is 19 GB and 68 GB respectively, and the rows make no claim that depends on it. The architecture reached mainline llama.cpp in ggml-org/llama.cpp#25165 (merged 2026-07-22) and the binary on this machine is newer, so support is present; that is a version check, not a load test.
Tests
388 pass, 11 new (8 catalog, 1 score, 2 selector). Both runners green, including
tests/isolated.sh.Two existing score tests were changed rather than added to. They asserted a fixed list of three size classes, which was really an undocumented assertion that no catalogued model is unrunnable at 20 GB with zero offload. A 117.6B row makes that false, and it surfaced as a failure in the grouping tests rather than where the assumption lived. Both now assert the invariant
score_rankdocuments — each class once, contiguous,unsupportedlast.