Score and tune layout settings against the benchmark corpus - #380
Merged
Conversation
The bench harness could measure a configuration but not compare two, so choosing a default still came down to reading a table and forming an opinion. This adds the missing half: one number per configuration, a coordinate descent that moves one setting at a time against it, and the statistics needed to tell a real improvement from a lucky sample. LayoutScore is the judgement call written down. Eight normalised penalty terms, weighted so overlaps dominate and hidden links and unreadable starts come next, because those are the defects a reader cannot work around. It is deliberately arguable, and the remarks say where to argue. LayoutTuner sweeps one setting across explicit candidates with the rest held, keeps the best, and repeats until a round finds nothing. Candidates are scored in parallel into their own slots, so a run is several times faster and still deterministic. The measurement statistics turned out to matter more than the search. The corpus score is a random variable: scored eight times over independent families of starting arrangements, the same settings vary with a standard deviation of 0.317 at twelve starts, 0.096 at twenty-four and 0.032 at forty-eight. Two descents run without accounting for that reached values disagreeing on six settings out of fifteen while scoring within 0.01 of each other, which is what fitting to noise looks like. So: - BenchOptions.StartOffset selects an independent family of arrangements, which is what makes a holdout check possible at all. - LayoutTuner.DefaultMinimumGain requires a change to beat the incumbent by three times that deviation before it is kept. GraphCorpus.TwoClasses is a second real document, and the first in the corpus with two roots. Counter is a single class, so all of its edges are short; two classes with calls crossing between them produce the long edges that get drawn through whatever is parked in the middle, which is the case that makes a link disappear. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018b2bn5CKSzpjRV4nxv3c3y
|
This was referenced Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Follows #378. The bench harness could measure one configuration but not compare two, so picking a default still came down to reading a table and forming an opinion. This adds the missing half — and, more usefully than the search itself, the statistics that say which of its answers are real.
What's here
LayoutScore— one number per configuration. Eight normalised penalty terms, weighted so overlaps dominate, then hidden links and unreadable starts. It is a judgement call written down rather than a fact, and the remarks say so and say where to argue with it.LayoutTuner— coordinate descent. Sweeps one setting across explicit candidates with the rest held, keeps the best, repeats until a round finds nothing. Every step is one setting, one sweep and one reason, which is what makes a tuned default arguable instead of merely asserted. Candidates are scored in parallel into their own slots, so a run is several times faster and still deterministic.GraphCorpus.TwoClasses— a second real document, 34 nodes, and the first graph in the corpus with two roots.Counteris a single class, so all of its edges are short. Two classes withTransfercalling intoAccount.Withdraw/Depositproduce long edges crossing between the subtrees, which is the case that makes a link disappear behind a body parked in the middle.The part that changed the method
The corpus score is a random variable, and it is noisier than the gains a descent chases. Scored eight times over independent families of starting arrangements, the unchanged defaults vary by:
Two descents run before this was measured — at 8 and 12 starts, accepting every improvement — converged on values that disagreed on six of fifteen settings and scored within 0.01 of each other. Both were fitting the arrangements they happened to be handed.
So two additions exist purely to prevent that:
BenchOptions.StartOffsetselects an independent family of arrangements, which is what makes a holdout check possible at all: a value chosen on one family has to also win on one it was not chosen on.LayoutTuner.DefaultMinimumGainrequires a change to beat the incumbent by 0.1, roughly three times the deviation at 48 starts, before it is kept.Not in this PR
No default in
LayoutSettingsorPhysicsSettingsis changed here. The tuning run at 48 starts is what decides that, and it belongs in its own commit with its evidence next to it.Testing
LayoutTunerTestscovers the ways a tuner goes wrong quietly: a candidate list that never offers the value a setting already has (so a pass cannot decline to move), a writer that reads back a different setting than it names, a score that is not reproducible, and a descent that ignores its own gain floor. 11 tests, all passing, plus the existing suite.🤖 Generated with Claude Code
https://claude.ai/code/session_018b2bn5CKSzpjRV4nxv3c3y
Generated by Claude Code