Skip to content

Fix artifacts v4 backend upload problems#36805

Merged
ChristopherHX merged 27 commits intogo-gitea:mainfrom
ChristopherHX:fix-various-artifact-v4-problems
Mar 5, 2026
Merged

Fix artifacts v4 backend upload problems#36805
ChristopherHX merged 27 commits intogo-gitea:mainfrom
ChristopherHX:fix-various-artifact-v4-problems

Conversation

@ChristopherHX
Copy link
Copy Markdown
Contributor

  • Use base64.RawURLEncoding to avoid equal sign
    • using the nodejs package they seem to get lost
  • Support uploads with unspecified length
  • Support uploads with a single named blockid
    • without requiring a blockmap

* Use base64.RawURLEncoding to avoid equal sign
  * using the nodejs package they seem to get lost
* Support uploads with unspecified length
* Support uploads with a single named blockid
  * without requiring a blockmap
@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Mar 2, 2026
@github-actions github-actions bot added modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code labels Mar 2, 2026
@ChristopherHX ChristopherHX added type/bug topic/gitea-actions related to the actions of Gitea backport/v1.25 This PR should be backported to Gitea 1.25 modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code and removed modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code labels Mar 2, 2026
@ChristopherHX
Copy link
Copy Markdown
Contributor Author

ChristopherHX commented Mar 2, 2026

Caused upload issues here: https://gitea.com/actions-oss/act-cli/issues/18 in the snapshot job.

I haven't found any docs how a query value like ?sig=AAAAAAAAA=&... can cause this to get only AAAAAAAAA back, which is obviously http 403.

RawURLEncoding omits the padding = symbols and fixed this, while I have no idea how to write a test for this specific thing.

In go tests + for the actual action I used for testing the = symbol has not been dropped while I read any logs of it. However using the package I tried this many times and all had a 403 reliable.

Defect code

      - name: Setup Node
        continue-on-error: true
        uses: actions/setup-node@v6
        with:
          node-version: 20
      - name: Install @actions/artifact@2.1.0
        continue-on-error: true
        run: npm install @actions/artifact@2.1.0
      - name: Upload All
        uses: actions/github-script@v8
        continue-on-error: true
        with:
          script: |
            // We do not use features depending on GITHUB_API_URL so we can hardcode it to avoid the GHES no support error
            process.env["GITHUB_SERVER_URL"] = "https://github.com";
            const {DefaultArtifactClient} = require('@actions/artifact');
            const aartifact = new DefaultArtifactClient();
            var artifacts = JSON.parse(process.env.ARTIFACTS);
            for(var artifact of artifacts) {
              if(artifact.type === "Binary") {
                const {id, size} = await aartifact.uploadArtifact(
                  // name of the artifact
                  `${artifact.name}-${artifact.target}`,
                  // files to include (supports absolute and relative paths)
                  [artifact.path],
                  process.cwd(),
                  {
                    // optional: how long to retain the artifact
                    // if unspecified, defaults to repository/org retention settings (the limit of this value)
                    retentionDays: 10
                  }
                );
                console.log(`Created artifact with id: ${id} (bytes: ${size}`);
              }
            }
        env:
          ARTIFACTS: ${{ steps.goreleaser.outputs.artifacts }}

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Addresses GitHub Actions artifacts v4 upload compatibility issues (notably around URL signatures, unknown-length uploads, and single-block blockid uploads) so more client implementations (eg Node) can upload reliably.

Changes:

  • Switch artifact v4 signed URL sig encoding/decoding to base64.RawURLEncoding (no = padding).
  • Extend chunk handling to support unknown/unspecified lengths and single blockid uploads without requiring a blocklist.
  • Expand integration coverage for these upload variations and adjust artifact v4 download test comments.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File Description
tests/integration/api_actions_artifact_v4_test.go Adds a table-driven upload test matrix covering blockid and “no length” scenarios; minor comment fix in download test.
routers/api/actions/artifactsv4.go Updates signature encoding, block upload handling, blocklist reading, and finalize logic for v4 artifacts.
routers/api/actions/artifacts_chunks.go Adjusts chunk save/append and chunk discovery logic to tolerate missing lengths and blocklist-less single-block uploads.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ChristopherHX ChristopherHX marked this pull request as draft March 2, 2026 16:32
@ChristopherHX
Copy link
Copy Markdown
Contributor Author

ChristopherHX commented Mar 2, 2026

Marked as draft, to allow appended uploads of multiple unsized chunks including chunks with blockid (https://learn.microsoft.com/en-us/rest/api/storageservices/put-block told me, this cannot happen for a client)

This comment was marked as resolved.

* Better error messages for decoding in log
@ChristopherHX ChristopherHX marked this pull request as ready for review March 2, 2026 20:10
@wxiaoguang
Copy link
Copy Markdown
Contributor

wxiaoguang commented Mar 3, 2026

The legacy "length" handling is quite tricky and unclear, I think we can clarify it to 6b0fe79

@wxiaoguang wxiaoguang force-pushed the fix-various-artifact-v4-problems branch from bd0fdd2 to 29dc6ee Compare March 5, 2026 01:13
@wxiaoguang wxiaoguang requested a review from Copilot March 5, 2026 01:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@imkuang
Copy link
Copy Markdown

imkuang commented Mar 5, 2026

Hello,

I just noticed this PR. Could it possibly be related to my issue? #36829

@ChristopherHX
Copy link
Copy Markdown
Contributor Author

ChristopherHX commented Mar 5, 2026

Yes potentially partially related

  • v6 upload problem could be is related, but this is not tested by me yet (verified the npm package, EDIT verified action v6 works in smoke test)
    • Since you wrote v5 works without GHES check, this version is more similar to the version I used during initial implementation
  • v7 is more this one: Feature non-zipped actions artifacts (action v7 / nodejs / npm v6.2.0) #36786 they added the mimetype to the json and the error in your log complains about the unknown field
    • This feature PR needs proper content-type delivery from blob storage to be ready for review, the azureblob backend lacks code to do this

@ChristopherHX
Copy link
Copy Markdown
Contributor Author

@wxiaoguang Just noticed you used the new new(<scalar>) syntax, removing backport label.

@ChristopherHX ChristopherHX removed the backport/v1.25 This PR should be backported to Gitea 1.25 label Mar 5, 2026
Copy link
Copy Markdown
Contributor

@wxiaoguang wxiaoguang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new() can be removed.

Feel free to merge and/or backport. Either is fine to me.

Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
@ChristopherHX ChristopherHX added backport/v1.25 This PR should be backported to Gitea 1.25 reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. labels Mar 5, 2026
@ChristopherHX
Copy link
Copy Markdown
Contributor Author

Merging and (1.25 backport (both verified) now)

@ChristopherHX ChristopherHX merged commit 867c4af into go-gitea:main Mar 5, 2026
26 checks passed
@GiteaBot GiteaBot added this to the 1.26.0 milestone Mar 5, 2026
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Mar 5, 2026
GiteaBot pushed a commit to GiteaBot/gitea that referenced this pull request Mar 5, 2026
* Use base64.RawURLEncoding to avoid equal sign
  * using the nodejs package they seem to get lost
* Support uploads with unspecified length
* Support uploads with a single named blockid
  * without requiring a blockmap

---------

Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
@GiteaBot GiteaBot added the backport/done All backports for this PR have been created label Mar 5, 2026
zjjhot added a commit to zjjhot/gitea that referenced this pull request Mar 6, 2026
* giteaofficial/main:
  Fix non-admins unable to automerge PRs from forks (go-gitea#36833)
  upgrade to github.com/cloudflare/circl 1.6.3, svgo 4.0.1, markdownlint-cli 0.48.0 (go-gitea#36837)
  Fix dump release asset bug (go-gitea#36799)
  build(deps): update material-icon-theme v5.32.0 (go-gitea#36832)
  Fix bug to check whether user can update pull request branch or rebase branch (go-gitea#36465)
  Fix forwarded proto handling for public URL detection (go-gitea#36810)
  Fix artifacts v4 backend upload problems (go-gitea#36805)
  Add a git grep search timeout (go-gitea#36809)
  fix(repo): unify DEFAULT_SHOW_FULL_NAME output in templates and dropdown (go-gitea#36597)
  Harden render iframe open-link handling (go-gitea#36811)
ChristopherHX added a commit that referenced this pull request Mar 6, 2026
Backport #36805 by @ChristopherHX

* Use base64.RawURLEncoding to avoid equal sign
  * using the nodejs package they seem to get lost
* Support uploads with unspecified length
* Support uploads with a single named blockid
  * without requiring a blockmap

Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
silverwind added a commit to silverwind/gitea that referenced this pull request Mar 6, 2026
* origin/main: (27 commits)
  Fix OAuth2 authorization code expiry and reuse handling (go-gitea#36797)
  Fix org permission API visibility checks for hidden members and private orgs (go-gitea#36798)
  Fix non-admins unable to automerge PRs from forks (go-gitea#36833)
  upgrade to github.com/cloudflare/circl 1.6.3, svgo 4.0.1, markdownlint-cli 0.48.0 (go-gitea#36837)
  Fix dump release asset bug (go-gitea#36799)
  build(deps): update material-icon-theme v5.32.0 (go-gitea#36832)
  Fix bug to check whether user can update pull request branch or rebase branch (go-gitea#36465)
  Fix forwarded proto handling for public URL detection (go-gitea#36810)
  Fix artifacts v4 backend upload problems (go-gitea#36805)
  Add a git grep search timeout (go-gitea#36809)
  fix(repo): unify DEFAULT_SHOW_FULL_NAME output in templates and dropdown (go-gitea#36597)
  Harden render iframe open-link handling (go-gitea#36811)
  [skip ci] Updated translations via Crowdin
  fix: /repos/{owner}/{repo}/actions/{runs,jobs} requiring owner permissions (go-gitea#36818)
  Fix CRAN package version validation to allow more than 4 version components (go-gitea#36813)
  Fix API not persisting pull request unit config when has_pull_requests is not set (go-gitea#36718)
  feat: Add Actions API rerun endpoints for runs and jobs (go-gitea#36768)
  Fix bug when pushing mirror with wiki (go-gitea#36795)
  Pull Request Pusher should be the author of the merge (go-gitea#36581)
  Delete non-exist branch should return 404 (go-gitea#36694)
  ...

# Conflicts:
#	routers/web/repo/issue_view.go
silverwind added a commit to silverwind/gitea that referenced this pull request Mar 8, 2026
* main: (26 commits)
  Clean up `refreshViewedFilesSummary` (go-gitea#36868)
  Remove `util.URLJoin` and replace all callers with direct path concatenation (go-gitea#36867)
  Optimize Docker build with dependency layer caching (go-gitea#36864)
  Fix URLJoin, markup render link reoslving, sign-in/up/linkaccount page common data (go-gitea#36861)
  Fix CodeQL code scanning alerts (go-gitea#36858)
  Refactor auth middleware (go-gitea#36848)
  Update Nix flake (go-gitea#36857)
  Update JS deps (go-gitea#36850)
  Load `mentionValues` asynchronously (go-gitea#36739)
  [skip ci] Updated translations via Crowdin
  Fix dbfs error handling (go-gitea#36844)
  Fix OAuth2 authorization code expiry and reuse handling (go-gitea#36797)
  Fix org permission API visibility checks for hidden members and private orgs (go-gitea#36798)
  Fix non-admins unable to automerge PRs from forks (go-gitea#36833)
  upgrade to github.com/cloudflare/circl 1.6.3, svgo 4.0.1, markdownlint-cli 0.48.0 (go-gitea#36837)
  Fix dump release asset bug (go-gitea#36799)
  build(deps): update material-icon-theme v5.32.0 (go-gitea#36832)
  Fix bug to check whether user can update pull request branch or rebase branch (go-gitea#36465)
  Fix forwarded proto handling for public URL detection (go-gitea#36810)
  Fix artifacts v4 backend upload problems (go-gitea#36805)
  ...

# Conflicts:
#	pnpm-lock.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/done All backports for this PR have been created backport/v1.25 This PR should be backported to Gitea 1.25 lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code topic/gitea-actions related to the actions of Gitea type/bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants