Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Garbage Collection LFS Pointer deletes inuse LFS Objects #31271

Closed
wienans opened this issue Jun 6, 2024 · 1 comment · Fixed by #31790
Closed

Garbage Collection LFS Pointer deletes inuse LFS Objects #31271

wienans opened this issue Jun 6, 2024 · 1 comment · Fixed by #31790
Labels
issue/critical This issue should be fixed ASAP. If it is a PR, the PR should be merged ASAP topic/lfs type/bug
Milestone

Comments

@wienans
Copy link
Contributor

wienans commented Jun 6, 2024

Description

Hallo Community,

i noticed that on my Gitea Platform all old LFS Files are lost. Currently i only have LFS files which are 3 days old. And 3 days ago i manually run the Garbage Collection for all repositories and Garbage Collection LFS pointers.

I now reproduced this issue with setting the config as follows for that task and uploaded some lfs files an hour ago.

[cron.gc_lfs]
OLDER_THAN = 1h
LAST_UPDATED_MORE_THAN_AGO = 1h

All LFS Files are gone again. And the Log looks not to be conclusive at all...
The Admin pannel shows that the task was done at 12:11:43 and i see the gc_lfs but there is not more than that.

Gitea Version

1.21.11

Can you reproduce the bug on the Gitea demo site?

No

Log

2024/06/06 12:11:43 ...eb/routing/logger.go:102:func1() [I] router: completed GET /user/events for 192.168.1.132:56594, 200 OK in 14976.7ms @ events/events.go:18(events.Events)
2024/06/06 12:11:43 ...rvices/cron/tasks.go:152:GetTask() [I] Getting gc_lfs in &{{0 0} gc_lfs 0xc000918740 0x24a6980    0 0001-01-01 00:00:00 +0000 UTC}
2024/06/06 12:11:43 ...eb/routing/logger.go:102:func1() [I] router: completed POST /admin for 192.168.1.132:56634, 303 See Other in 6.0ms @ admin/admin.go:141(admin.DashboardPost)
2024/06/06 12:11:43 ...eb/routing/logger.go:102:func1() [I] router: completed GET /admin/monitor/cron for 192.168.1.132:56634, 200 OK in 23.1ms @ admin/admin.go:175(admin.CronTasks)
2024/06/06 12:11:43 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/actions/runner.v1.RunnerService/FetchTask for 192.168.3.47:46962, 200 OK in 60.9ms @ <autogenerated>:1(http.Handler.ServeHTTP-fm)
2024/06/06 12:11:45 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/actions/runner.v1.RunnerService/FetchTask for 192.168.3.47:46962, 200 OK in 55.4ms @ <autogenerated>:1(http.Handler.ServeHTTP-fm)
2024/06/06 12:11:47 ...eb/routing/logger.go:68:func1() [I] router: polling   GET /user/events for 192.168.1.132:56634, elapsed 3610.7ms @ events/events.go:18(events.Events)
2024/06/06 12:11:47 ...eb/routing/logger.go:102:func1() [I] router: completed GET /user/events for 192.168.1.132:56634, 200 OK in 3732.3ms @ events/events.go:18(events.Events)
2024/06/06 12:11:47 ...eb/routing/logger.go:102:func1() [I] router: completed GET /admin/monitor/cron for 192.168.1.132:56635, 200 OK in 18.0ms @ admin/admin.go:175(admin.CronTasks)
2024/06/06 12:11:47 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/actions/runner.v1.RunnerService/FetchTask for 192.168.3.47:46962, 200 OK in 50.6ms @ <autogenerated>:1(http.Handler.ServeHTTP-fm)
2024/06/06 12:11:49 ...eb/routing/logger.go:102:func1() [I] router: completed POST /api/actions/runner.v1.RunnerService/FetchTask for 192.168.3.47:46962, 200 OK in 52.4ms @ <autogenerated>:1(http.Handler.ServeHTTP-fm)
2024/06/06 12:11:50 ...eb/routing/logger.go:102:func1() [I] router: completed GET /wienand/GIT-lfs-test/settings/lfs for 192.168.1.132:56636, 200 OK in 146.5ms @ setting/lfs.go:42(setting.LFSFiles)
2024/06/06 12:11:51 ...eb/routing/logger.go:68:func1() [I] router: polling   GET /user/events for 192.168.1.132:56635, elapsed 3738.1ms @ events/events.go:18(events.Events)

Screenshots

No response

Git Version

2.42.0, Wire Protocol Version 2 Enabled

Operating System

Windows

How are you running Gitea?

Windows Server: gitea-1.21.11-gogit-windows-4.0-amd64

Database

MySQL/MariaDB

@wolfogre
Copy link
Member

wolfogre commented Aug 7, 2024

Confirmed that it can be reproduced. The problem arises with gogit enabled editions:

  • gitea-1.22.1-gogit-windows-4.0-386.exe
  • gitea-1.22.1-gogit-windows-4.0-amd64.exe

@wolfogre wolfogre added the issue/critical This issue should be fixed ASAP. If it is a PR, the PR should be merged ASAP label Aug 7, 2024
@lunny lunny added this to the 1.22.2 milestone Aug 8, 2024
GiteaBot pushed a commit to GiteaBot/gitea that referenced this issue Aug 9, 2024
Fix go-gitea#31271.

When gogit is enabled, `IsObjectExist` calls
`repo.gogitRepo.ResolveRevision`, which is not correct. It's for
checking references not objects, it could work with commit hash since
it's both a valid reference and a commit object, but it doesn't work
with blob objects.

So it causes go-gitea#31271 because it reports that all blob objects do not
exist.
wolfogre added a commit that referenced this issue Aug 9, 2024
Backport #31790 by @wolfogre

Fix #31271.

When gogit is enabled, `IsObjectExist` calls
`repo.gogitRepo.ResolveRevision`, which is not correct. It's for
checking references not objects, it could work with commit hash since
it's both a valid reference and a commit object, but it doesn't work
with blob objects.

So it causes #31271 because it reports that all blob objects do not
exist.

Co-authored-by: Jason Song <[email protected]>
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Nov 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
issue/critical This issue should be fixed ASAP. If it is a PR, the PR should be merged ASAP topic/lfs type/bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants