Skip to content

feat: integrity check jobs (missing files, untracked files, checksums) #24205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
insertish wants to merge 140 commits into
base: main
Choose a base branch
from
+5,645 −24
Open

feat: integrity check jobs (missing files, untracked files, checksums) #24205

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
f77f43a
stash: integrity checks
insertish Nov 26, 2025
4a7120c
refactor: batched integrity checks
insertish Nov 26, 2025
3414210
feat: checksum job
insertish Nov 27, 2025
15503b1
chore: open api
insertish Nov 27, 2025
1e941f3
feat: write integrity report to database
insertish Nov 27, 2025
929ad52
feat: add createdAt to integrity report table
insertish Nov 27, 2025
cc31b9c
feat: clean up old reports of checksum or missing files
insertish Nov 27, 2025
ef7d8e9
feat: check orphaned file reports are not out of date
insertish Nov 27, 2025
1744237
chore: open api
insertish Nov 27, 2025
9386023
feat: add config options & cron entries for checks
insertish Nov 27, 2025
2516319
fix: mock the new repository
insertish Nov 27, 2025
919eb83
revert: override migration db url
insertish Nov 27, 2025
4462683
chore: generate SQL queries
insertish Nov 27, 2025
03276de
fix: add integrity report repository to service depends.
insertish Nov 27, 2025
8db6132
fix: add mock for asset repo.
insertish Nov 27, 2025
0fdc7b4
feat: draft controller entry
insertish Nov 27, 2025
d3abed3
feat: view integrity report in maintenance page (cherry picked)
insertish Nov 27, 2025
ca358f4
feat: sub-pages for integrity reports
insertish Nov 28, 2025
c50118e
chore: remove old table comment
insertish Nov 28, 2025
13e9cf0
stash: moving computers because pnpm is cooked
insertish Nov 28, 2025
2779fce
feat: manually trigger integrity jobs
insertish Nov 28, 2025
e447ba8
chore: sort i18n
insertish Nov 28, 2025
4d7f7b8
feat: refresh missing & checksum
insertish Nov 28, 2025
0362d21
test: take baseline, check for each issue, check refreshes work
insertish Nov 28, 2025
c4ac8d9
stash: incomplete checksum outdated test
insertish Nov 28, 2025
01f96de
test: serialise the buffer over events
insertish Dec 1, 2025
1daf1b4
chore: lint
insertish Dec 1, 2025
db690bc
chore: generate SQL
insertish Dec 1, 2025
fec8923
test: increase timeouts
insertish Dec 1, 2025
06fcd54
feat: download csv report, download file, delete file
insertish Dec 1, 2025
042af30
chore: use checksum configuration
insertish Dec 1, 2025
806a288
feat: assetId, fileAssetId columns on integrity reports
insertish Dec 1, 2025
6cfd199
feat: ability to delete all reports (and corresponding objects)
insertish Dec 2, 2025
64cc64d
refactor: move all new queries into integrity repository
insertish Dec 2, 2025
6e752be
fix: don't process trashed/deleted assets for integrity
insertish Dec 2, 2025
e1a1662
chore: more compliant csv
insertish Dec 2, 2025
73a17bb
chore: generate SQL
insertish Dec 2, 2025
ae653f9
chore: lint
insertish Dec 2, 2025
7a215c1
fix: flip deletedAt filter
insertish Dec 2, 2025
5d5d421
fix: `path` -> `reportId` as `reportId`
insertish Dec 3, 2025
6e7854b
chore: sync SQL
insertish Dec 3, 2025
8fdec46
refactor: use separate queue for integrity checks
insertish Dec 17, 2025
7d71f99
test: split integrity out of maintenance
insertish Dec 17, 2025
21c26dd
refactor: split integrity out of maintenance controller/service
insertish Dec 17, 2025
08e5321
refactor: split maintenance dto for integrity checks
insertish Dec 17, 2025
0e75f38
merge: remote-tracking branch 'immich/main' into feat/integrity-check…
insertish Dec 17, 2025
b8feaec
merge: remote-tracking branch 'immich/main' into feat/integrity-check…
insertish Dec 17, 2025
05acf74
test: add new queue name
insertish Dec 17, 2025
16c2082
chore: use new buttons for admin pages
insertish Dec 17, 2025
f1c7f13
test: remove un-used variables
insertish Dec 17, 2025
b205350
chore: type issues
insertish Dec 17, 2025
31ac88f
test: cont. integrity service tests
insertish Dec 18, 2025
5028c56
feat: paginate integrity report results
insertish Dec 18, 2025
098563e
refactor: use storage repository stat instead of real stat
insertish Dec 18, 2025
b730662
test: service tests for orphaned files
insertish Dec 18, 2025
c045fa2
test: service tests for missing files
insertish Dec 18, 2025
92d23ce
fix: use plain read stream instead of real
insertish Dec 18, 2025
b1f3c75
test: add mock for plain read
insertish Dec 18, 2025
ff07b4f
test: service tests for checksum
insertish Dec 18, 2025
748ba67
chore: update sql
insertish Dec 18, 2025
8b1e299
chore: i18n pass
insertish Dec 18, 2025
2962c54
refactor: queue separate jobs for deleting integrity reports
insertish Dec 18, 2025
4ed3386
fix: use correct queue type in web
insertish Dec 18, 2025
06f81f4
chore: reload tables if job queue empties
insertish Dec 18, 2025
ff7453e
test: service tests for deleting reports
insertish Dec 18, 2025
82351f4
chore: add descriptions to routes
insertish Dec 18, 2025
042335f
fix: wrap and don't disappear UI on smaller screens
insertish Dec 18, 2025
bb4893d
test: e2e tests for get report and delete entries
insertish Dec 18, 2025
a81074f
fix: filter assets that are trashed
insertish Dec 18, 2025
a6cac7d
test: add tests for csv/download file
insertish Dec 18, 2025
2d48d05
chore: open api / sql
insertish Dec 18, 2025
66da3c1
chore: summary text
insertish Dec 19, 2025
d35c4f8
test: e2e for delete all jobs
insertish Dec 19, 2025
d4161a2
feat: run all jobs button
insertish Dec 19, 2025
b3ba880
chore: remove stray comments
insertish Dec 19, 2025
7e1e283
chore: remove debug code
insertish Dec 19, 2025
c4c5358
merge: remote-tracking branch 'immich/main' into feat/integrity-check…
insertish Jan 6, 2026
06ee275
chore: fmt
insertish Jan 6, 2026
d189722
refactor: use cursors instead of pages
insertish Jan 6, 2026
ed33f79
refactor: orphan -> untracked
insertish Jan 7, 2026
aaec6db
chore: i18n sort
insertish Jan 7, 2026
7e0e4bd
chore: sync sql
insertish Jan 7, 2026
5f35abb
test: correct service spec
insertish Jan 7, 2026
7f8b077
fix: accept UUIDv7 in URL param
insertish Jan 7, 2026
a9867c3
test: update e2e tests with refactoring changes
insertish Jan 7, 2026
0ef02ba
test: e2e web test
insertish Jan 7, 2026
1b15e4f
test: wait for files to be detected before deleting them
insertish Jan 7, 2026
300d409
chore: format e2e test
insertish Jan 7, 2026
728e018
chore: e2e lint
insertish Jan 8, 2026
1bab670
test: ensure directory we write to exists first
insertish Jan 8, 2026
e0624ad
Merge branch 'main' into feat/integrity-checks-izzy
insertish Jan 8, 2026
66e7517
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 13, 2026
aebde65
chore: open api
insertish Jan 13, 2026
26d2b41
chore: add index to asset.createdAt
insertish Jan 13, 2026
259e93d
chore: i18n pass on table
insertish Jan 13, 2026
bfd0ac2
chore: lint
insertish Jan 13, 2026
85f6490
refactor: use new ui Table component
insertish Jan 13, 2026
812419d
test: update concurrency
insertish Jan 13, 2026
67cc937
chore: pick a version in controller
insertish Jan 13, 2026
69b2e36
refactor: use new web service architecture (1/2)
insertish Jan 13, 2026
1c73f7f
refactor: split delete event into two
insertish Jan 13, 2026
ef6e31b
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 14, 2026
bf83507
refactor: use handleCreateJob
insertish Jan 14, 2026
a20458f
chore: lint fixes
insertish Jan 15, 2026
28443a0
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 15, 2026
e9dafef
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 16, 2026
b68c758
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 16, 2026
e1ba8e6
fix: incorrect merge on admin page layout
insertish Jan 16, 2026
ffdc901
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 21, 2026
bffbf02
chore: merge fixes
insertish Jan 21, 2026
ec8628f
Merge branch 'main' into feat/integrity-checks-izzy
insertish Jan 21, 2026
e22beb0
test: probably stuff that got lost in merge
insertish Jan 22, 2026
48ca31f
merge: remote-tracking branch 'origin/main' into feat/integrity-check…
insertish Jan 22, 2026
1b033c0
Merge remote-tracking branch 'origin/main' into feat/integrity-checks…
insertish Feb 6, 2026
7e5ab25
refactor: merge duplicate beforeAll blocks
insertish Feb 6, 2026
3d5db67
refactor: move `/report/:id` route definitions together
insertish Feb 6, 2026
08fb19b
refactor: rewrite countAll SQL to use GROUP BY count
insertish Feb 6, 2026
bbcb6b8
refactor: use `makeStream` instead of inline generator
insertish Feb 6, 2026
6c38417
refactor: move code out of try {} block
insertish Feb 6, 2026
cc7d8f2
refactor: prefer `type` over `interface`
insertish Feb 11, 2026
a1812e9
Merge remote-tracking branch 'origin/main' into feat/integrity-checks…
insertish Feb 11, 2026
477d432
chore: update readme/sql
insertish Feb 11, 2026
8cb63eb
refactor: review suggestions for web/../integrity.service.ts
insertish Feb 12, 2026
5ed0ff4
chore: update icon for integrity checks queue
insertish Feb 12, 2026
fbdeb04
refactor: use GET/Query for report fetch
insertish Feb 12, 2026
4908289
refactor: pass only userId to deleteIntegrityReport
insertish Feb 12, 2026
bf17d8c
refactor: arrow functions in svelte components
insertish Feb 12, 2026
c5fb13e
refactor: split refresh into private functions
insertish Feb 12, 2026
6b7ac81
refactor: split asset checksum check into private func.
insertish Feb 12, 2026
f6d88ff
refactor: use `makeStream` for 2nd case; update `makeStream` type
insertish Feb 12, 2026
a44a886
Merge remote-tracking branch 'origin/main' into feat/integrity-checks…
insertish Feb 24, 2026
f4ccf18
chore: openapi
insertish Feb 24, 2026
df51bea
chore: remove `uppercase` from UI
insertish Feb 24, 2026
e32ed6c
chore: change variable name casing
insertish Feb 24, 2026
0a35809
refactor: use medium tests for integrity service
insertish Feb 25, 2026
891a3c8
refactor: stricter repository typing
insertish Feb 25, 2026
1812248
Merge remote-tracking branch 'origin/main' into feat/integrity-checks…
insertish Feb 25, 2026
6f35ab9
fix: use @immich/sql-tools
insertish Feb 25, 2026
50eea11
chore: open api
insertish Feb 25, 2026
a7045b3
chore: sql query was renamed
insertish Feb 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions e2e/src/specs/maintenance/server/maintenance.e2e-spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ describe('/admin/maintenance', () => {
},
{
interval: 500,
timeout: 10_000,
timeout: 60_000,
},
)
.toBeTruthy();
Expand Down Expand Up @@ -190,7 +190,7 @@ describe('/admin/maintenance', () => {
},
{
interval: 500,
timeout: 10_000,
timeout: 60_000,
},
)
.toBeFalsy();
Expand Down
Loading
Loading