Duplicate schools fix

[tomas-stefano]

Tell the duplicated schools apart, and keep what was found

Some providers hold the same school twice. The query we started from grouped site
rows by provider and urn but filtered code <> '-', which found 9 groups for 2026 and
hid the shape behind the other 95.

This adds a read-only classification run that records what it found as a DataHub
process summary. It decides nothing and merges nothing — picking a merge policy is the
next piece of work, and this is the evidence it gets picked from.

DataHub::DuplicateSchools::Executor.new(years: %w[2026]).execute
#=> DataHub::DuplicateSchoolsProcessSummary

What it found for 2026

104 groups · 119 surplus site rows · 100 surplus provider_school rows

The provider_school number is the one users feel: the schools list, both course school
pickers and the new search panel all read Provider::School now, so each surplus row is
a school listed twice.

The four kinds

Each shape is a Kind subclass that declares how to recognise itself, what it means, and
what to do about it. Matching order is precedence — MainSiteCollision is tested first,
DivergentNameTwin is the terminal fallback.

main_site_collision — 95 groups, 95 surplus rows

Matches a group containing the - row.
What it is: the provider's main site and the same school added again as a placement
school. BackfillMainSiteUrns gave all 494 main sites a urn, and
ProviderSchools::Creator#normal_provider_school excludes site_code: '-' when checking
for an existing row, so re-adding a school you already hold as your main site always makes
a second one. The provider sees Foo School (Main Site) and Foo School side by side.
Action: decide the policy — keep - and move courses onto it, or keep the named row.
Flags: main_site_at_risk 35, courses_on_both_sides 50.

⚠️ main_site_at_risk is the reason not to point the existing sites:deduplicate task at
these: pick_primary_site ranks by [site_statuses.count, -id], so in 35 of the 95 it
would discard the provider's main site — and it never touches provider_school /
course_school, so it would remove nothing a user can see.

clone — 4 groups, 11 surplus rows

Matches one code and one name across the group.
What it is: the same school added repeatedly under one code, from a bulk add that ran
about ten times for one provider on 2025-09-24 (492 sites in a minute). Invisible to
providers — provider_school's unique (provider_id, gias_school_id, site_code) index
already collapsed them, so this is legacy litter only.
Action: safe to merge unattended — no user-visible duplicate to remove.
Flags: gias_closed 1.

divergent_name_twin — 4 groups, 4 surplus rows

Matches more than one code and more than one name, no - row.
What it is: one urn under two provider-written names. Often deliberate labelling rather
than an accident — 13S holds urn 136906 as both George Abbot School (93 courses) and
Main site Secondary- one of our partner schools (88).
Action: ask the provider which name they meant before merging.
Flags: courses_on_both_sides 4, provider_authored_name 1.

split_code_twin — 1 group, 9 surplus rows

Matches more than one code, one name, no - row.
What it is: one school held under two codes, so it is listed twice and its courses are
split between them. Same bulk-add cause as clone, except Sites::CodeGenerator picks
codes randomly, so a re-run produced a different code. 1TZ holds urn 144834 as DK
(7 courses) and DN (11).
Action: merge onto the code holding more courses, moving the rest across.
Flags: courses_on_both_sides 1.

What gets recorded

• short_summary — years, groups, surplus site rows, surplus provider_school rows, and
  the same counts split by kind and by flag. Tallies rather than a field per kind, so the
  Kind list stays the single source of truth and a fifth shape needs no migration.
• full_summary — every group and every site behind those counts (88KB for 2026), so the
  CSV can be rebuilt without running against production again.

Notes for review

• Nothing is mutated. The only row written is the summary itself. Asserted by the
  executor spec (not_change on both Site and Provider::School), not just by eye.
• Kinds are one file each, so the inherited hook that used to collect them no longer
  fires — Zeitwerk would not have loaded a subclass before Kind.for asked for it. An
  explicit ORDER replaces it, which also makes precedence declared rather than an
  accident of load order.
• Analytics blocklist: jsonb_accessor keys are real model attributes, so the app
  refuses to boot until the new fields are declared. Added by hand rather than with
  dfe:analytics:regenerate_blocklist — that task runs with SUPPRESS_DFE_ANALYTICS_INIT=1,
  so Provider::School and Course::School are never autoloaded and it removes their
  existing entries. They are in neither config file, so accepting that would break boot.
  The diff is 7 added lines; rails dfe:analytics:check passes.
• Scope is 2026. 2027 does not exist in production yet. Cleaning 2026 before rollover
  means 2027 is created clean — on a rolled-over dev copy, 95 of the 100 duplicate pairs
  come straight across.

Next

Read the recorded summary, choose the main_site_collision policy, then write the merge
process — which must move provider_school / course_school rows, not just site /
course_site, and should record its own summary the same way.

[tomas-stefano]

Closing because I was only demonstrating the kinds and groups of the issue to the tech lead

[github-actions]

Review app for PR 6318 was deleted