Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix missing signature key error when pulling Docker images with SERVE_DIRECT enabled #32365

Merged
merged 6 commits into from
Oct 31, 2024

Conversation

Zettat123
Copy link
Contributor

@Zettat123 Zettat123 commented Oct 29, 2024

Fix #28121

I did some tests and found that the missing signature key error is caused by an incorrect Content-Type header. Gitea correctly sets the Content-Type header when serving files.

headers := &containerHeaders{
ContentDigest: pfd.Properties.GetByName(container_module.PropertyDigest),
ContentType: pfd.Properties.GetByName(container_module.PropertyMediaType),
ContentLength: pfd.Blob.Size,
Status: http.StatusOK,
}

However, when SERVE_DIRECT is enabled, the Content-Type header may be set to an incorrect value by the storage service. To fix this issue, we can use query parameters to override response header values.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html

In this PR, I introduced a new parameter to the URL method to support additional parameters.

URL(path, name string, reqParams url.Values) (*url.URL, error)

Most S3-like services support specifying the content type when storing objects. However, Gitea always use application/octet-stream. Therefore, I believe we also need to improve the Save method to support storing objects with the correct content type.

minio.PutObjectOptions{
ContentType: "application/octet-stream",
// some storages like:
// * https://developers.cloudflare.com/r2/api/s3/api/
// * https://www.backblaze.com/b2/docs/s3_compatible_api.html
// do not support "x-amz-checksum-algorithm" header, so use legacy MD5 checksum
SendContentMd5: m.cfg.ChecksumAlgorithm == "md5",
},

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Oct 29, 2024
@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 29, 2024
@github-actions github-actions bot added modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code labels Oct 29, 2024
@@ -215,7 +215,7 @@ func servePackageFile(ctx *context.Context, params parameters, serveContent bool
return
}

s, u, _, err := packages_service.GetPackageBlobStream(ctx, pf, pb)
s, u, _, err := packages_service.GetPackageBlobStream(ctx, pf, pb, nil)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if other package managers also have content type issues, so I didn't specify a content type for them.

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Oct 29, 2024
@lunny lunny added lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. backport/v1.22 This PR should be backported to Gitea 1.22 and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Oct 29, 2024
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Oct 30, 2024
@lunny lunny added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Oct 31, 2024
@techknowlogick techknowlogick enabled auto-merge (squash) October 31, 2024 15:08
@techknowlogick techknowlogick merged commit 0690cb0 into go-gitea:main Oct 31, 2024
26 checks passed
@GiteaBot GiteaBot added this to the 1.24.0 milestone Oct 31, 2024
@GiteaBot
Copy link
Collaborator

I was unable to create a backport for 1.22. @Zettat123, please send one manually. 🍵

go run ./contrib/backport 32365
...  // fix git conflicts if any
go run ./contrib/backport --continue

@GiteaBot GiteaBot added backport/manual No power to the bots! Create your backport yourself! and removed reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. labels Oct 31, 2024
Zettat123 added a commit to Zettat123/gitea that referenced this pull request Nov 1, 2024
…RVE_DIRECT` enabled (go-gitea#32365)

Fix go-gitea#28121

I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.

https://github.com/go-gitea/gitea/blob/348d1d0f322ca57c459acd902f54821d687ca804/routers/api/packages/container/container.go#L712-L717
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
<img width="600px"
src="https://github.com/user-attachments/assets/f2ff90f0-f1df-46f9-9680-b8120222c555"
/>

In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.

```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```

---

Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.

https://github.com/go-gitea/gitea/blob/b7fb20e73e63b8edc9b90c52073e248bef428fcc/modules/storage/minio.go#L214-L221
lunny pushed a commit that referenced this pull request Nov 1, 2024
…RVE_DIRECT` enabled (#32365) (#32397)

Backport #32365

Fix #28121

I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.


https://github.com/go-gitea/gitea/blob/348d1d0f322ca57c459acd902f54821d687ca804/routers/api/packages/container/container.go#L712-L717
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html <img
width="600px"

src="https://github.com/user-attachments/assets/f2ff90f0-f1df-46f9-9680-b8120222c555"
/>

In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.

```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```
@lunny lunny modified the milestones: 1.24.0, 1.23.0 Nov 2, 2024
zjjhot added a commit to zjjhot/gitea that referenced this pull request Nov 6, 2024
* giteaofficial/main: (21 commits)
  Use 8 as default value for git lfs concurrency (go-gitea#32421)
  Fix milestone deadline and date related problems (go-gitea#32339)
  Only query team tables if repository is under org when getting assignees (go-gitea#32414)
  Refactor RepoRefByType (go-gitea#32413)
  Refactor template ctx and render utils (go-gitea#32422)
  Refactor DateUtils and merge TimeSince (go-gitea#32409)
  Refactor markup package (go-gitea#32399)
  Add some handy markdown editor features (go-gitea#32400)
  Make LFS http_client parallel within a batch. (go-gitea#32369)
  Refactor repo legacy (go-gitea#32404)
  Replace DateTime with proper functions (go-gitea#32402)
  Fix git error handling (go-gitea#32401)
  Fix created_unix for mirroring (go-gitea#32342)
  Replace DateTime with DateUtils (go-gitea#32383)
  improve performance of diffs (go-gitea#32393)
  Refactor tests to prevent from unnecessary preparations (go-gitea#32398)
  Add artifacts test fixture (go-gitea#30300)
  Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (go-gitea#32365)
  Fix a number of typescript issues (go-gitea#32308)
  Update go dependencies (go-gitea#32389)
  ...
@lunny lunny added the backport/done All backports for this PR have been created label Nov 21, 2024
TKaxv-7S added a commit to TKaxv-7S/gitea that referenced this pull request Dec 1, 2024
* SECURITY
  * Fix basic auth with webauthn (go-gitea#32531) (go-gitea#32536)
  * Refactor internal routers (partial backport, auth token const time comparing) (go-gitea#32473) (go-gitea#32479)
* PERFORMANCE
  * Remove transaction for archive download (go-gitea#32186) (go-gitea#32520)
* BUGFIXES
  * Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (go-gitea#32365) (go-gitea#32397)
  * Fix get reviewers fails when selecting user without pull request permissions unit (go-gitea#32415) (go-gitea#32616)
  * Fix adding index files to tmp directory (go-gitea#32360) (go-gitea#32593)
  * Fix PR creation on forked repositories via API (go-gitea#31863) (go-gitea#32591)
  * Fix missing menu tabs in organization project view page (go-gitea#32313) (go-gitea#32592)
  * Support HTTP POST requests to `/userinfo`, aligning to OpenID Core specification (go-gitea#32578) (go-gitea#32594)
  * Fix debian package clean up cron job (go-gitea#32351) (go-gitea#32590)
  * Fix GetInactiveUsers (go-gitea#32540) (go-gitea#32588)
  * Allow the actions user to login via the jwt token (go-gitea#32527) (go-gitea#32580)
  * Fix submodule parsing (go-gitea#32571) (go-gitea#32577)
  * Refactor find forks and fix possible bugs that weaken permissions check (go-gitea#32528) (go-gitea#32547)
  * Fix some places that don't respect org full name setting (go-gitea#32243) (go-gitea#32550)
  * Refactor push mirror find and add check for updating push mirror (go-gitea#32539) (go-gitea#32549)
  * Fix basic auth with webauthn (go-gitea#32531) (go-gitea#32536)
  * Fix artifact v4 upload above 8MB (go-gitea#31664) (go-gitea#32523)
  * Fix oauth2 error handle not return immediately (go-gitea#32514) (go-gitea#32516)
  * Fix action not triggered when commit message is too long (go-gitea#32498) (go-gitea#32507)
  * Fix `GetRepoLink` nil pointer dereference on dashboard feed page when repo is deleted with actions enabled (go-gitea#32501) (go-gitea#32502)
  * Fix `missing signature key` error when pulling Docker images with `SERVE_DIRECT` enabled (go-gitea#32397) (go-gitea#32397)
  * Fix the permission check for user search API and limit the number of returned users for `/user/search` (go-gitea#32310)
  * Fix SearchIssues swagger docs (go-gitea#32208) (go-gitea#32298)
  * Fix dropdown content overflow (go-gitea#31610) (go-gitea#32250)
  * Disable Oauth check if oauth disabled (go-gitea#32368) (go-gitea#32480)
  * Respect renamed dependencies of Cargo registry (go-gitea#32430) (go-gitea#32478)
  * Fix mermaid diagram height when initially hidden (go-gitea#32457) (go-gitea#32464)
  * Fix broken releases when re-pushing tags (go-gitea#32435) (go-gitea#32449)
  * Only provide the commit summary for Discord webhook push events (go-gitea#32432) (go-gitea#32447)
  * Only query team tables if repository is under org when getting assignees (go-gitea#32414) (go-gitea#32426)
  * Fix created_unix for mirroring (go-gitea#32342) (go-gitea#32406)
  * Respect UI.ExploreDefaultSort setting again (go-gitea#32357) (go-gitea#32385)
  * Fix broken image when editing comment with non-image attachments (go-gitea#32319) (go-gitea#32345)
  * Fix disable 2fa bug (go-gitea#32320) (go-gitea#32330)
  * Always update expiration time when creating an artifact (go-gitea#32281) (go-gitea#32285)
  * Fix null errors on conversation holder (go-gitea#32258) (go-gitea#32266) (go-gitea#32282)
  * Only rename a user when they should receive a different name (go-gitea#32247) (go-gitea#32249)
  * Fix checkbox bug on private/archive filter (go-gitea#32236) (go-gitea#32240)
  * Add a doctor check to disable the "Actions" unit for mirrors (go-gitea#32424) (go-gitea#32497)
  * Quick fix milestone deadline 9999 (go-gitea#32423)
  * Make `show stats` work when only one file changed (go-gitea#32244) (go-gitea#32268)
  * Make `owner/repo/pulls` handlers use "PR reader" permission (go-gitea#32254) (go-gitea#32265)
  * Update scheduled tasks even if changes are pushed by "ActionsUser" (go-gitea#32246) (go-gitea#32252)
* MISC
  * Remove unnecessary code: `GetPushMirrorsByRepoID` called on all repo pages (go-gitea#32560) (go-gitea#32567)
  * Improve some sanitizer rules (go-gitea#32534)
  * Update nix development environment vor v1.22.x (go-gitea#32495)
  * Add warn log when deleting inactive users (go-gitea#32318) (go-gitea#32321)
  * Update github.com/go-enry/go-enry to v2.9.1 (go-gitea#32295) (go-gitea#32296)
  * Warn users when they try to use a non-root-url to sign in/up (go-gitea#32272) (go-gitea#32273)

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEumb2f9c/cFjXEtMIw7fJG2Mvc4oFAmdEyeoACgkQw7fJG2Mv
# c4pythAAn57Z9Csfd8UrHbCd87SBlEGydhlng5Oc99pQIAvExR0hc9VFWjt5pFr4
# aXTajtzb/sDQkAPZEiL45CL471z+Ga81ixaKRfrBeMiSECB0wBaL4+XH94qQ3lw3
# /dNfQsc9bUnomGWQyEIbQ6mT85fJdvBD1nibUSH3b5P4WqOBHbY9YlehPmE96KY2
# 9k1IYvBvcfCjK6njVQ7m+sFOr7/Y2ZHe9FeN8hEf/1Bfnc75wtkeNyeXnlNe67Eo
# ViFzcA35WyTXw4NRY+TG/8xZEXHl8DuOuUdPoBqkpFw9TzxR2svO0QLzRIHgJP+t
# /Cdd16zZd6fQ+ET+DV8IaF2wlXdEgVDWs2aT04VDLGpSw9czxsUEUQ0ETWFFomXN
# //goTLu1B3fVQYrE9MK2vfUQGe2Su3ChGwNtNEK9bMQpO6sLFGRE0nPgBJMPJ0yA
# bfPhRlsVxnyEToqeKoC77wv0kPiOkzPfDm6sFLAt+tATcij5UlTU4nVXyXsELk14
# p5mtsTfaEqiH3U+JW0Drz8wV7nk8F599lZbYO92M3Z59bqC5TsOVYgqb1ODTpqQO
# 7gLdgdKmQbKWTPHLA9Hz+0/3bT1MirMRdtXW7TmgW83TuN37wOuElCmXmJTN2feY
# LG4k417kVrBwF+fdGPXo+T7H0MqxX1fTkVftG3C63sdaRQrUM1M=
# =jyQM
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue, Nov 26, 2024  3:03:06 AM
# gpg:                using RSA key BA66F67FD73F7058D712D308C3B7C91B632F738A
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: keydb_search failed: Connection timed out
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: Note: database_open 134217901 waiting for lock (held by 1152) ...
# gpg: keydb_search failed: Connection timed out
# gpg: Can't check signature: No public key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/done All backports for this PR have been created backport/manual No power to the bots! Create your backport yourself! backport/v1.22 This PR should be backported to Gitea 1.22 lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code size/M Denotes a PR that changes 30-99 lines, ignoring generated files. type/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Registry: Error response from daemon: missing signature key after upgrade to 1.21.0
6 participants