Skip to content

fix: prevent snapshot from tracking large files#12185

Open
goniz wants to merge 10 commits intoanomalyco:devfrom
goniz:snapshot-filter-bigf
Open

fix: prevent snapshot from tracking large files#12185
goniz wants to merge 10 commits intoanomalyco:devfrom
goniz:snapshot-filter-bigf

Conversation

@goniz
Copy link
Contributor

@goniz goniz commented Feb 4, 2026

This PR prevents the Snapshot.track() flow from adding (untracked) big files into the snapshot system (git).

It changes the code to first do git add -N . which stage the files without content.
Next it iterates over the staged files to find any files larger then defined threshold (1 gig). unstages them AND adds them to $gitdir/info/exclude to prevent them from getting staged again.
Finally it calls git add -- $stagedFiles to stage the files content as well.

To handle existing bloated snapshot dirs, the cleanup method was updated to detect if any existing object blob is bigger then threshold -> nukes the whole snapshot dir.
Leaving the big files causes the git gc --prune process to blow up on memory and hog the system.

Additionally, this PR cleans up the Snapshot code a bit.

Don't be scared from the lines added, I've added tests too 😆

Fixes #6845

@github-actions
Copy link
Contributor

github-actions bot commented Feb 4, 2026

The following comment was made by an LLM, it may be inaccurate:

Based on the search results, I found one potentially related PR:

Related PR:

You should verify if PR #10628 has already been merged or if it addresses the same root cause as PR #12185.

@goniz goniz force-pushed the snapshot-filter-bigf branch 4 times, most recently from 3f7b472 to 026c6ee Compare February 11, 2026 18:34
@goniz goniz changed the title feat: filter out 1GB+ files from snapshot fix: prevent snapshot from trackibg large files Feb 14, 2026
@goniz goniz changed the title fix: prevent snapshot from trackibg large files fix: prevent snapshot from tracking large files Feb 15, 2026
@goniz goniz force-pushed the snapshot-filter-bigf branch 4 times, most recently from 72f72fb to 5bbccec Compare February 17, 2026 17:50
@goniz goniz force-pushed the snapshot-filter-bigf branch from 5bbccec to c28f3ed Compare February 24, 2026 14:00
Copy link
Contributor Author

@goniz goniz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self review 😄

@goniz goniz force-pushed the snapshot-filter-bigf branch from c28f3ed to 70022bc Compare February 24, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HUGE snapshot folder

1 participant