Git Bundles with LFS

[stu1811]

Gitea Version

1.15.3

Git Version

2.24.1

Operating System

CentOS 7.2

How are you running Gitea?

I'm running gitea locally via systemd.

Database

MySQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

I use git bundles to sync two remote sites. We grab all recent LFS files, tar them up, and extract on the remote site. This has worked for over a year now without any issues. Since 1.15.0 this process does not work anymore. I get a 404 error when I try to fetch the LFS files. I discovered a workaround where I extract the files into gitea/data/lfs and then manually insert a row into the lfs_meta_object table with the appropriate info, then I can fetch the new files. For some reason the extraction is no longer being added to the lfs_meta_object automatically

Screenshots

No response

[stu1811]

Issue is probably related to changes made under #14726

[stu1811]

Still an issue

[stu1811]

Weekly bump. Still need to manually edit DB.

[stu1811]

Weekly bump. I don't like manipulating the MySQL DB.

[stu1811]

I missed a week of bumping. Please fix so I don't have to manually edit the mysql db.

[techknowlogick]

@stu1811 we are all volunteers here, if you'd like to prioritize this you have a couple of options 1. Create a PR yourself, or 2. Fund a developer/put a bounty forward for this

[lunny]

I missed a week of bumping. Please fix so I don't have to manually edit the mysql db.

I think maybe the title is confusing. Did you mean you think this is a bug caused by #14726 ?

[stu1811]

@techknowlogick I realize this is volunteer project and everyone does awesome work. I was hoping to at least get it acknowledged and marked as a bug.

@lunny Yes, I belive #14726 caused the problem. Prior to 1.15.0 my process for transferring bundles between sites was the following:

Source Site

1. Generate bundle of repo
2. Tar up recent lfs files from gitea/data/lfs

Destination Site

1. Extract lfs tarball into data/gitea/lfs
2. Merge branches from bundle

Since 1.15.0 I cannot merge the branches from the bundle because gitea cannot find the new files in the LFS repo. I get a 404 error. Eventually I figured out the LFS files are tracked in the lfs_meta_object table. When I add the files to the table I can merge branches from the bundle.

[lunny]

Which route returned 404 for you? Could you give some logs?

[stu1811]

@lunny let me know if you need any other debug or logs.

I created a new sample repo. Added two new lfs files and cloned to another machine.

[ME@centipede lfs_test2]$ GIT_TRACE=1 git lfs fetch 
06:53:56.133146 git.c:702               trace: exec: git-lfs fetch
06:53:56.133281 run-command.c:663       trace: run_command: git-lfs fetch
06:53:56.136687 trace git-lfs: exec: git 'version'
06:53:56.140740 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
06:53:56.142412 trace git-lfs: exec: git 'config' '-l'
fetch: Fetching reference refs/heads/master
06:53:56.159177 trace git-lfs: filepathfilter: accepting "bigFileOfDoom"
06:53:56.159218 trace git-lfs: filepathfilter: accepting "bigFileOfDoom2"
06:53:56.159222 trace git-lfs: filepathfilter: accepting "foobar"
06:53:56.160122 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'           
06:53:56.162429 trace git-lfs: tq: running as batched queue, batch size of 100
06:53:56.162444 trace git-lfs: fetch bigFileOfDoom [a6f9d0f0c8028a60b37e948825c87c8f34f2a1cd59ceffeea64b09c39edc0dae]
06:53:56.162458 trace git-lfs: fetch bigFileOfDoom2 [22e4cab3cf50cf42248d5531ae71ba203b73400c578ec61ec218f60b54c8682d]
06:53:56.162504 trace git-lfs: tq: sending batch of size 2
06:53:56.162608 trace git-lfs: run_command: ssh -- git@localhost git-lfs-authenticate stu/lfs_test.git download
06:53:56.735919 trace git-lfs: api: batch 2 files
06:53:56.736267 trace git-lfs: HTTP: POST http://localhost:3000/stu/lfs_test.git/info/lfs/objects/batch
06:53:56.867359 trace git-lfs: HTTP: 200
06:53:56.867428 trace git-lfs: HTTP: {"objects":[{"oid":"a6f9d0f0c8028a60b37e948825c87c8f34f2a1cd59ceffeea64b09c39edc0dae","size":104000000,"error":{"code":404,"message":"Not Found"}},{"oid":"22e4cab3cf50cf42248d5531ae71ba203b73400c578ec61ec218f60b54c8682d","size":104000000,"error":{"code":404,"message":"Not Found"}}]}

06:53:56.867607 trace git-lfs: tq: starting transfer adapter "basic"                                                                                                                                               
[a6f9d0f0c8028a60b37e948825c87c8f34f2a1cd59ceffeea64b09c39edc0dae] Not Found: [404] Not Found                                                                                                                      
[22e4cab3cf50cf42248d5531ae71ba203b73400c578ec61ec218f60b54c8682d] Not Found: [404] Not Found
error: failed to fetch some objects from 'https://localhost/stu/lfs_test.git/info/lfs'

On the original host when I run a mysql query here is the lfs_meta_object table. On the machine I cloned to the table is empty. If I manually populate the table on the machine with the clone I can successfully do a git lfs fetch

MariaDB [giteadb]> select * from lfs_meta_object;
+----+------------------------------------------------------------------+-----------+---------------+--------------+
| id | oid                                                              | size      | repository_id | created_unix |
+----+------------------------------------------------------------------+-----------+---------------+--------------+
|  1 | a6f9d0f0c8028a60b37e948825c87c8f34f2a1cd59ceffeea64b09c39edc0dae | 104000000 |             3 |   1636716692 |
|  2 | 22e4cab3cf50cf42248d5531ae71ba203b73400c578ec61ec218f60b54c8682d | 104000000 |             3 |   1636717610 |
+----+------------------------------------------------------------------+-----------+---------------+--------------+
2 rows in set (0.000 sec)

You can also test this using the gitea web interface. Go to https://GITEA_HOST/TEAM/PROJECT/settings/lfs/show/LFS_SHA`. Both SHAs are reachable on the host system. They are not reachable on the cloned system.

[lunny]

Are you using Gitea v1.15.6?

[stu1811]

Our production system is 1.15.4 and the test system is 1.15.6