Skip to content

Score and tune layout settings against the benchmark corpus - #380

Merged
matt-edmondson merged 1 commit into
mainfrom
claude/layout-tuning
Sep 9, 2026
Merged

Score and tune layout settings against the benchmark corpus#380
matt-edmondson merged 1 commit into
mainfrom
claude/layout-tuning

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Follows #378. The bench harness could measure one configuration but not compare two, so picking a default still came down to reading a table and forming an opinion. This adds the missing half — and, more usefully than the search itself, the statistics that say which of its answers are real.

What's here

LayoutScore — one number per configuration. Eight normalised penalty terms, weighted so overlaps dominate, then hidden links and unreadable starts. It is a judgement call written down rather than a fact, and the remarks say so and say where to argue with it.

LayoutTuner — coordinate descent. Sweeps one setting across explicit candidates with the rest held, keeps the best, repeats until a round finds nothing. Every step is one setting, one sweep and one reason, which is what makes a tuned default arguable instead of merely asserted. Candidates are scored in parallel into their own slots, so a run is several times faster and still deterministic.

GraphCorpus.TwoClasses — a second real document, 34 nodes, and the first graph in the corpus with two roots. Counter is a single class, so all of its edges are short. Two classes with Transfer calling into Account.Withdraw/Deposit produce long edges crossing between the subtrees, which is the case that makes a link disappear behind a body parked in the middle.

The part that changed the method

The corpus score is a random variable, and it is noisier than the gains a descent chases. Scored eight times over independent families of starting arrangements, the unchanged defaults vary by:

starts sd of corpus score
12 0.317
24 0.096
48 0.032

Two descents run before this was measured — at 8 and 12 starts, accepting every improvement — converged on values that disagreed on six of fifteen settings and scored within 0.01 of each other. Both were fitting the arrangements they happened to be handed.

So two additions exist purely to prevent that:

  • BenchOptions.StartOffset selects an independent family of arrangements, which is what makes a holdout check possible at all: a value chosen on one family has to also win on one it was not chosen on.
  • LayoutTuner.DefaultMinimumGain requires a change to beat the incumbent by 0.1, roughly three times the deviation at 48 starts, before it is kept.

Not in this PR

No default in LayoutSettings or PhysicsSettings is changed here. The tuning run at 48 starts is what decides that, and it belongs in its own commit with its evidence next to it.

Testing

LayoutTunerTests covers the ways a tuner goes wrong quietly: a candidate list that never offers the value a setting already has (so a pass cannot decline to move), a writer that reads back a different setting than it names, a score that is not reproducible, and a descent that ignores its own gain floor. 11 tests, all passing, plus the existing suite.

🤖 Generated with Claude Code

https://claude.ai/code/session_018b2bn5CKSzpjRV4nxv3c3y


Generated by Claude Code

The bench harness could measure a configuration but not compare two, so
choosing a default still came down to reading a table and forming an
opinion. This adds the missing half: one number per configuration, a
coordinate descent that moves one setting at a time against it, and the
statistics needed to tell a real improvement from a lucky sample.

LayoutScore is the judgement call written down. Eight normalised penalty
terms, weighted so overlaps dominate and hidden links and unreadable
starts come next, because those are the defects a reader cannot work
around. It is deliberately arguable, and the remarks say where to argue.

LayoutTuner sweeps one setting across explicit candidates with the rest
held, keeps the best, and repeats until a round finds nothing. Candidates
are scored in parallel into their own slots, so a run is several times
faster and still deterministic.

The measurement statistics turned out to matter more than the search. The
corpus score is a random variable: scored eight times over independent
families of starting arrangements, the same settings vary with a standard
deviation of 0.317 at twelve starts, 0.096 at twenty-four and 0.032 at
forty-eight. Two descents run without accounting for that reached values
disagreeing on six settings out of fifteen while scoring within 0.01 of
each other, which is what fitting to noise looks like. So:

- BenchOptions.StartOffset selects an independent family of arrangements,
  which is what makes a holdout check possible at all.
- LayoutTuner.DefaultMinimumGain requires a change to beat the incumbent
  by three times that deviation before it is kept.

GraphCorpus.TwoClasses is a second real document, and the first in the
corpus with two roots. Counter is a single class, so all of its edges are
short; two classes with calls crossing between them produce the long
edges that get drawn through whatever is parked in the middle, which is
the case that makes a link disappear.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018b2bn5CKSzpjRV4nxv3c3y
@matt-edmondson
matt-edmondson merged commit 9bc6bb8 into main Sep 9, 2026
12 checks passed
@matt-edmondson
matt-edmondson deleted the claude/layout-tuning branch September 9, 2026 04:56
@sonarqubecloud

sonarqubecloud Bot commented Sep 9, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants