Skip to content

Fill what a series is missing from the other sources it is attached to - #257

Closed
Squeaks72 wants to merge 1 commit into
AngeloSha:mainfrom
Squeaks72:pr/fill-missing-metadata
Closed

Squeaks72 wants to merge 1 commit into
AngeloSha:mainfrom
Squeaks72:pr/fill-missing-metadata

Conversation

@Squeaks72

Copy link
Copy Markdown
Contributor

Summary

  • A series is scanned from its files, so it carries only what its main source gave when they were written. A series whose main source never described it, or whose follower source has a description, genres, author or cover the main one lacks, kept the gap for good. bff/src/lib/metaFill.ts (fillMissingMeta) now fills it from the sources the series is attached to: its own source first, then its followers in the order they were added, skipping switched-off or failing sources, until nothing is missing.
  • Only empty fields are filled; nothing a scan, a source or a person set is replaced. Text (description, genres, author) goes to series_overrides, the one place a scan does not overwrite, and only where the field is still empty at write time. A cover goes to series_art.cover (the automatic cover, below any cover a person chose) and only for a series with no chapters yet, since otherwise the series already has its first page.
  • Two callers: the nightly library repair (a new stepMeta after the reading-directions step: 15 series a night, picked by seriesMissingMeta, stopping between series for a Cancel or a shutdown), and the manual follow route (POST /api/admin/series/:id/sources), which fires it in the background after the follow is stored. It never throws and never delays the answer.
  • No new route or setting, so no openapi change.

Type

  • Feature

Testing

bff and web typecheck clean; web npm test passes in full. New bff/test/metaFill.int.test.ts against a throwaway Postgres: a series missing everything is filled from its follower (trimmed, main source asked first), a second call asks nothing, a complete series gets no override row, a partly filled one gets only its empty fields (a person's genres and a scanned description are left alone), a failing source is skipped for the next, an unknown series is a no-op, seriesMissingMeta picks only series that still lack something, plus source checks that the repair step and the follow route call it. repair.int (53) and seriesSources.int (16) pass unchanged. Nothing was exercised against a real source or in a browser.

Checklist

  • Focused scope (one topic)
  • Matches the existing TypeScript style
  • Docs updated if user-facing (docs/USAGE.md)
  • No new default sources hardcoded to specific sites

🤖 Generated with Claude Code

A description, genres, author or cover that a series lacks is taken from the sources it is attached to (its own source first, then its followers), by the nightly repair and in the background after a manual follow. Only empty fields are filled: nothing a scan, a source or a person set is replaced.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@AngeloSha

Copy link
Copy Markdown
Owner

Thank you, this is a good idea, and it's in v0.73.0 as
Admin → Settings → Library housekeeping → Fill in missing details. I rebuilt it on the same pattern as the
chapter-name borrowing (#85), so a few things work differently from your version:

  • It has a layer of its own. Your version wrote into series_overrides, which is the admin's edit layer. A
    filled-in description there hid any better one a later scan brought. It also refilled genres someone had cleared
    on purpose ({} means "cleared", NULL means "inherit"), and it could change what the 18+ switch hides. The filled-in
    values now sit below the scan: a series shows them only where its own value and any edit are both empty, a scan
    that brings real data wins straight away, and the 18+ filter judges the genres a series actually shows.
  • It's off by default, like names. It asks only series that are kept updated, and only sources that aren't
    switched off, failing or cooling down (blockedNow). Each lookup gets the source's own time budget.
  • It's a proper repair step (details), with counters, a Tasks line and an REPAIR_DETAILS_MAX knob (15 by
    default, your number). A series nobody can describe is asked again after a month, not every night.
  • Text goes through cleanDescription and cleanGenres, as an add does. Followers in another language aren't
    asked, so a Spanish follower can't describe an English series in Spanish.
  • The manual follow still asks at once, past the monthly wait.
  • Switching it off takes back everything it filled in, covers included.

The tests are behaviour tests now: the series route, the 18+ filter, the cooldown and language rules, and the follow
route driven over HTTP.

@AngeloSha AngeloSha closed this Oct 16, 2026
Squeaks72 added a commit to Squeaks72/uchiyomi that referenced this pull request Oct 16, 2026
…ilds of AngeloSha#257 and AngeloSha#245)

Admin → Settings → Library housekeeping → Fill in missing details, off by default: the nightly repair's ninth step
(`details`) gives a series with no description, genres or author of its own the ones another place has, and a series
with no chapters yet a cover.

- Its own layer, series_auto_details, read only where the series' own value (lib_series, the scan) and the admin's
  edit (series_overrides) are both empty: lib/autoDetailsSql.ts is the one expression for the series source, OPDS,
  the Komga authors and the 18+ filter (which judges the genres a series shows), and the hero rule counts all three.
  AngeloSha#257 wrote into series_overrides: it shadowed better data a later scan brought, refilled genres cleared on purpose
  (`{}`), and could change what Show 18+ hides.
- AniList first (AngeloSha#245, ported AniList-first): one batched Media(id_in:) request per fifty series linked there with a
  checked link, in libraries whose lookups are on, in English. No Kitsu or MangaUpdates call. mediaById reads through
  readCapped.
- Then, as stepNames does: at most REPAIR_DETAILS_MAX (15) series that are kept updated, their own source and then
  the ones they follow in their language, only usable sources (standing, then blockedNow right before), each lookup
  bounded by the source's budget and the series by a minute; text through cleanDescription and cleanGenres.
- A series asked waits 30 days (sources_at, anilist_at); a manual follow asks at once. Counters, a Tasks line, the
  limits and the estimate. Switching it off deletes the layer and puts back any cover it set. Forget purges it.
- Behaviour tests (autoDetails.int): off by default, the order and the cleaning, below the scan, `{}` kept, the
  cooldown and language rules, monitored only, the 18+ filter, AniList's one batch, the month's wait, the manual
  follow over HTTP, the take-back. Three reintroduced.
- 5 new strings, translated in all eight languages.

Co-Authored-By: Squeaks72 <Squeaks72@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Squeaks72 pushed a commit to Squeaks72/uchiyomi that referenced this pull request Oct 16, 2026
…gine passwords

@Squeaks72 is credited for AngeloSha#257, AngeloSha#245 and AngeloSha#256. The upgrade note says what changes for sources that were ticked as
adult under the 18+ filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants