Skip to content

Add organization migration feature#36782

Open
TomRoyls wants to merge 14 commits intogo-gitea:mainfrom
TomRoyls:feat/org-migration
Open

Add organization migration feature#36782
TomRoyls wants to merge 14 commits intogo-gitea:mainfrom
TomRoyls:feat/org-migration

Conversation

@TomRoyls
Copy link
Copy Markdown

Summary

This PR adds the ability to migrate all repositories from an organization on a supported external platform (GitHub, GitLab, Gitea, Gogs, OneDev, Codebase, CodeCommit, GitBucket) into a target Gitea organization in a single operation.

What this PR does

  • Adds GetOrgRepositories(ctx, orgName, page, perPage) to the Downloader interface and implements it for all supported migration backends (GitHub, GitLab, Gitea, Gogs, OneDev, Codebase, CodeCommit, GitBucket)
  • Adds MigrateOrganization service function that pages through the source org's repos and calls MigrateRepository for each one, collecting per-repo success/failure results
  • Adds a web UI at /org/migrate with fields for source URL, service type, credentials, source/target org names, and content options (wiki, issues, labels, milestones, releases, PRs)
  • Adds REST API endpoint POST /api/v1/orgs/migrate (requires org-owner or admin) with MigrateOrgOptions input and OrgMigrationResult response
  • Adds integration tests covering authentication, authorization, validation errors, and error responses
  • Adds swagger type registrations for MigrateOrgOptions (input) and OrgMigrationResult (response)
  • Adds locale strings for all new UI labels

Usage

Web UI: Navigate to Organization Settings → Migrate (or /org/migrate). Fill in the source platform URL, select the service type, provide credentials, enter source and target org names, select which content to include, and submit. A success page lists migrated and failed repositories.

API:

POST /api/v1/orgs/migrate
Authorization: token <token>
Content-Type: application/json

{
  "clone_addr": "https://github.com",
  "service": 2,
  "auth_token": "ghp_...",
  "source_org_name": "my-github-org",
  "target_org_name": "my-gitea-org",
  "wiki": true,
  "issues": true,
  "labels": true,
  "milestones": true,
  "releases": true,
  "pull_requests": true
}

Response:

{
  "total_repos": 12,
  "migrated_repos": ["repo-a", "repo-b", ...],
  "failed_repos": [{"repo_name": "repo-c", "error": "..."}]
}

Testing

  • Integration tests: make test-sqlite#TestOrgMigrate
  • Manual: start Gitea locally, create an org, then call POST /api/v1/orgs/migrate with credentials for a test org on another platform

Notes

  • Individual repository failures do not abort the overall migration — the response includes a failed_repos list so callers can retry specific repos
  • The Gogs backend fetches all repos in one call (SDK limitation); for very large orgs this may use significant memory
  • The Codebase backend fetches all repos on each paginated call; this is a known inefficiency for large orgs
  • Mirrors are supported: set "mirror": true in the request

AI disclosure: This implementation was assisted by Claude (claude-sonnet-4-6). The code was reviewed and the PR description was written with AI assistance.

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Feb 28, 2026
@github-actions github-actions bot added modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code modifies/templates This PR modifies the template files labels Feb 28, 2026
Add ability to migrate all repositories from an organization on GitHub,
GitLab, Gitea, Gogs, and other supported platforms in a single operation.

New features:
- Add GetOrgRepositories to the Downloader interface and implement it for
  GitHub, GitLab, Gitea, Gogs, OneDev, Codebase, CodeCommit, and GitBucket
- Add MigrateOrganization service function that iterates org repos and calls
  MigrateRepository for each
- Add web UI at /org/migrate with service picker, credentials, org name, and
  content selection options
- Add REST API endpoint POST /api/v1/orgs/migrate with MigrateOrgOptions
  input and OrgMigrationResult response
- Add integration tests covering auth, authorization, validation, and error cases
- Add swagger type registrations for MigrateOrgOptions and OrgMigrationResult
- Add locale strings for all new UI labels
@bircni
Copy link
Copy Markdown
Member

bircni commented Feb 28, 2026

Any UI examples?
please also check first the tests locally 😄

- Update MigrateOrgOptions struct to use ptr(true) instead of new(bool) for IncludeSubGroups
- Fix codebase.go: fmt.Sprintf -> string concatenation
- Fix migrate.go: errors.New instead of fmt.Errorf
- Run make fmt
- Run make generate-swagger
    - Commit and push
@bircni
Copy link
Copy Markdown
Member

bircni commented Feb 28, 2026

Authorship: Codex (GPT-5.3)

Overall this is a strong feature addition, but I see a few blocking correctness issues before merge.

  1. Blocking: clone_addr format mismatch can trigger runtime panic in GitHub path parsing
    The new org migration UI/API describe clone_addr as a service/org URL, but org migration initializes the existing
    repo downloader factories with that value directly. The GitHub factory still assumes /owner/repo and indexes path
    segments without bounds checks, so host/org URLs can panic.
    Refs: services/migrations/migrate.go#L577, services/migrations/github.go#L49, templates/org/migrate.tmpl#L15,
    options/locale/locale_en-US.json#L2715
  2. Blocking: API path bypasses instance migration policy checks
    POST /orgs/migrate does not enforce mirror-disable policy, does not force private when Repository.ForcePrivate is
    enabled, and does not gate LFS with LFS.StartServer. Existing repo migration handlers do enforce these.
    Refs: routers/api/v1/org/migrate.go#L92, routers/web/org/migrate.go#L62, routers/web/org/migrate.go#L128, routers/
    api/v1/repo/migrate.go#L117
  3. Major: per-repo clone URL is reconstructed incorrectly
    Org migration builds each repo clone address via string concat (opts.CloneAddr + "/" + owner + "/" + name) instead
    of using provider-returned clone URL/full namespace. This is fragile for subgroups/subpaths and can produce wrong
    repo paths (notably GitLab with IncludeSubGroups=true).
    Refs: services/migrations/migrate.go#L602, services/migrations/gitlab.go#L789, services/migrations/gitlab.go#L801
  4. Major test gap: API “valid request” test allows 500s
    The integration test explicitly allows server errors for valid input, so the above runtime/path issues can slip
    through green CI.
    Refs: tests/integration/api_org_migrate_test.go#L74, tests/integration/api_org_migrate_test.go#L8

- Add policy checks to API handler:
  - Check mirror.DisableNewPull before allowing mirrors
  - Enforce Repository.ForcePrivate setting
  - Enforce LFS.StartServer setting
- Fix clone URL reconstruction to use repo.CloneURL from provider
  instead of string concatenation (fixes panic for GitHub/GitLab)
The perfsprint linter requires errors.New() when there are no format
arguments in fmt.Errorf().
- Add placeholder repo name to CloneAddr when creating downloader for org
  migration. The GitHub factory expects /owner/repo format but GetOrgRepositories
  only uses the orgName parameter. The placeholder prevents index out of bounds
  panic when factory parses the URL path.
- Fix test token scopes to include both WriteOrganization and WriteRepository
@TomRoyls
Copy link
Copy Markdown
Author

UI example:
org-migrate-filled

@silverwind
Copy link
Copy Markdown
Member

The tests contain references to _csrf but we removed that in #36183. Prompt your AI to rewrite the code against latest main branch.

@bircni
Copy link
Copy Markdown
Member

bircni commented Mar 1, 2026

In UI there is stuff odd

  • Why set "this repository will be a mirror" --> we are migrating an ORG
  • Same with items
  • Same with "Make repository private"

@silverwind
Copy link
Copy Markdown
Member

Also the cut-off "g" in source org name is a issue that I think we have solved already, but I could be wrong. Check other similar dropdowns if they are structurally identical.

@TomRoyls
Copy link
Copy Markdown
Author

TomRoyls commented Mar 3, 2026

The tests contain references to _csrf but we removed that in #36183. Prompt your AI to rewrite the code against latest main branch.

fixed it

In UI there is stuff odd

* Why set "this repository will be a mirror" --> we are migrating an ORG

* Same with items

* Same with "Make repository private"

added different text instead of reusing old one

Also the cut-off "g" in source org name is a issue that I think we have solved already, but I could be wrong. Check other similar dropdowns if they are structurally identical.

fixed it.

New screenshot:
Screenshot 2026-03-03 at 23-44-41 Migrate Organization - Gitea Git with a cup of tea

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@bircni
Copy link
Copy Markdown
Member

bircni commented Mar 10, 2026

This comment was written by Codex on behalf of @bircni

Findings for PR #36782 (#36782):

  1. Medium: the new web UI does not actually support the non-token auth flows that the feature claims to support. The form only renders auth_token and never renders au
    th_username / auth_password, while initOrgMigration() enables the migrate-items checkboxes solely from token presence. That means services such as Gogs, OneDev, Co
    debase, CodeCommit, and plain Git cannot use their normal auth path from the web flow, even though the API struct supports those fields. References: templates/org/
    migrate.tmpl#L33-L37 (https://github.com/go-gitea/gitea/pull/36782/files#diff-55eaf1a0cc3d88d3e9ca5a95cf7801e6f67ad8d2da059d8f7cfd6d1c38b4de69R33-R37), web_src/js/
    features/repo-migration.ts#L14-L25
    (https://github.com/go-gitea/gitea/pull/36782/files#diff-1a19857cde93c8bbf0ccf02029e7b3b9f1cc376a7e4f2e89af262f9e8c6538f7R14-R25), modules/structs/repo.go#L414-L418
    (https://github.com/go-gitea/gitea/pull/36782/files#diff-f77a0a7d63ea58a7704b5cbf8c605718cc88216340d64c8ebebd805f46bde7abR414-R418).
  2. Medium: the page advertises service types that the backend still hard-rejects. MigrateOrg builds the selector from SupportedFullGitService, which includes CodeComm
    it, but CodeCommitDownloader.GetOrgRepositories is implemented as ErrNotSupported. So a user can pick CodeCommit in the UI and only discover at submit time that org
    migration is unsupported. References: routers/web/org/migrate.go#L38-L40
    (https://github.com/go-gitea/gitea/pull/36782/files#diff-b74d5e3eaa1ad4f6fef3f4633cc9f17131a2fb0ee924c0275f7b5aa1148f6d3eR38-R40), modules/structs/repo.go#L413-L421
    (https://github.com/go-gitea/gitea/pull/36782/files#diff-f77a0a7d63ea58a7704b5cbf8c605718cc88216340d64c8ebebd805f46bde7abR413-R421), services/migrations/codecommit
    .go#L268-L270 (https://github.com/go-gitea/gitea/pull/36782/files#diff-4e5cfe4a8a9287c6c730d03ea33084fdc8f4bdb5e49db9f2810b520212d0704dR268-R270).
  3. Low: the generated OpenAPI for MigrateOrgOptions only marks clone_addr as required, even though both source_org_name and target_org_name are required by the handler
    and form. That will mislead generated clients and schema-based validation. References: modules/structs/repo.go#L407-L412
    (https://github.com/go-gitea/gitea/pull/36782/files#diff-f77a0a7d63ea58a7704b5cbf8c605718cc88216340d64c8ebebd805f46bde7abR407-R412), templates/swagger/v1_json.tmpl
    #L26497-L26502 (https://github.com/go-gitea/gitea/pull/36782/files#diff-2f4eaf34caef7618bbba6fef5fd6d6ea2b1bb4ba9b66108ec5fba54a826ffba6R26497-R26502).

- Add auth_username/auth_password fields to web UI for non-token auth
  services (Git, Gogs, OneDev, GitBucket, Codebase)
- Filter out CodeCommit from org migration dropdown since GetOrgRepositories
  is not supported for that service
- Mark source_org_name and target_org_name as required in OpenAPI schema
@TomRoyls TomRoyls force-pushed the feat/org-migration branch from da9aa0f to f612a23 Compare March 12, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. modifies/api This PR adds API routes or modifies them modifies/frontend modifies/go Pull requests that update Go code modifies/templates This PR modifies the template files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants