-
Notifications
You must be signed in to change notification settings - Fork 0
feat: Implement automated database migration flow for PostgREST services #79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
NgocAnhDo26
merged 57 commits into
main
from
users/hpn/refactorPostgREST-CI/CD-pipeline-with-ArgoCD-presync-migrations
May 31, 2026
Merged
Changes from all commits
Commits
Show all changes
57 commits
Select commit
Hold shift + click to select a range
f69be3a
feat(portal): add .NET 10 Web API backstage template with postgres wi…
PhuocHoan 4ea9c93
chore: update prereq scripts and operator test expectations
PhuocHoan 95e1f1a
portal: refine template descriptions and local CORS origins
PhuocHoan 7cf6573
Enhance database environment variable injection in Deployment (#67)
NgocAnhDo26 f67ca74
Users/hpn/implement postg rest auto api template (#65)
nghiaz160904 e7946e9
Upgrade dotnet template and postgres defaults to latest
PhuocHoan cf86730
Merge branch 'features/sprint4-backend-templates-db-injection' into u…
PhuocHoan fb2bfff
feat: add cross-platform support (Windows & Linux) for bootstrap proc…
PhamHoangKha1403 18cc2e1
Merge base branch and resolve template/setup conflicts
PhuocHoan 2edfa6a
Update dotnet template to SDK 10.0.202 and add PR68 evidence guide
PhuocHoan 1a68e88
Remove PR68 evidence guide from tracked files
PhuocHoan 2f1779e
fix tekton and portal local dev integration
PhuocHoan 8663db2
fix portal prettier formatting
PhuocHoan c53892c
feat(postgrest-template): Remove obsolete files and enhance template …
nghiaz160904 9a91d2e
feat(postgrest-template): Enhance PostgREST template with detailed RE…
nghiaz160904 3ce52d2
feat(postgrest-template): Remove obsolete GitOps files and update val…
nghiaz160904 b85b094
feat: Implement database migration trigger for PostgREST
nghiaz160904 1aedf76
temp: enhance PostgREST template with default values and new actions …
nghiaz160904 a0db3a4
Add initial migration
nghiaz160904 edd5a5a
feat: Enhance database migration support and improve webhook handling
nghiaz160904 ea40c07
chore: remove initial schema migration file
nghiaz160904 89ff59f
feat: Add database secret reference and validation for HeliosApp conf…
nghiaz160904 c30b5b9
Merge pull request #68 from helios-platform-team/users/hph/impl/templ…
NgocAnhDo26 8a5ef55
fix: Update secret validation to exclude database secrets due to auto…
nghiaz160904 37ac092
feat: Add database secret reference to WebhookIngress configuration
nghiaz160904 aa28e11
fix: Add databaseSecretRef to Tekton schema for improved secret manag…
nghiaz160904 b70dd0f
Merge pull request #70: branch 'bug/user/hpn/postgrest-template-fail…
VH3956 17539f6
setup hasura-graphql-template
VH3956 dfd936b
comment's review fix + change of workflow
VH3956 e0c011b
fix gitops namespace
VH3956 6ef3a7b
feat: Enhance Hasura GraphQL template with database secret reference …
nghiaz160904 25360f8
adding Docker workflow
VH3956 917bf8f
fix: Add new steps in taskfile to resolve startup problem, fix duplic…
hoangphuc841 eeba8db
fix: update application catalog metadata and fix an issue with compon…
hoangphuc841 7f837e9
fix: resolve db-migrate trigger and postgrest template issues
hoangphuc841 6cb1c63
- fix(crd): remove hardcoded 'api-db-secret' kubebuilder default from
hoangphuc841 2641c6a
update gitosPath + readme
VH3956 f6b8f1b
fix(test): updates tests for the recent changes
hoangphuc841 02e186d
chore: regenerate the crd from the go type annotations to match CI ch…
hoangphuc841 1c1fab8
fix(cue): repalce unsafe [:32] string slice with safe conditional let…
hoangphuc841 7fd9b97
fix(template): updated jwt references after switching to secret field
hoangphuc841 6bdddb2
fix(lint): extract repeated postgres string into dbTypePostgres const…
hoangphuc841 1c8e862
fix(template): update jwtSecret description and enforce length constr…
nghiaz160904 aa57f62
fix: accessing DB url + add:database migration
VH3956 4c363f9
fix: using PGRST_DB_URI + update cue logic for kube secret injections
VH3956 be601cd
fix: update manifest namespace
VH3956 b6faeb3
Merge pull request #71 from helios-platform-team/user/nhphuc/test-and…
NgocAnhDo26 062708c
Merge branch 'features/sprint4-backend-templates-db-injection' into u…
nghiaz160904 dd7acd2
Merge pull request #64 from helios-platform-team/users/ndvh/impl/hasu…
nghiaz160904 9450115
feat: Implement automated database migration flow for PostgREST services
9afc93c
refactor: Simplify client calls in PreSyncReconciler methods
72724d2
Merge branch 'main' into users/hpn/refactorPostgREST-CI/CD-pipeline-w…
nghiaz160904 f03ab17
fix: Fixed according to CodeRabbit reviews.
c85adb1
fix: lint
ed8571e
feat: Add system secrets provisioning to DatabaseReconciler and updat…
05db557
refactor: Enhance error handling and improve annotations in PreSync a…
cd9514e
test: Add Docker credentials and GitOps bot secrets to unit tests
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,13 +8,68 @@ import ( | |
| "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" | ||
| ) | ||
|
|
||
| // GenerateArgoApplication creates an ArgoCD Application manifest. | ||
| // GenerateArgoApplication creates an ArgoCD Application manifest with PreSync hooks | ||
| // if the HeliosApp has a database trait for automatic database migrations. | ||
| func GenerateArgoApplication(heliosApp *appv1alpha1.HeliosApp) (*unstructured.Unstructured, error) { | ||
| appName := heliosApp.Name + "-argocd" | ||
| targetNamespace := cmp.Or(heliosApp.Spec.ArgoCDNamespace, "argocd") | ||
| project := cmp.Or(heliosApp.Spec.ArgoCDProject, "default") | ||
| gitOpsBranch := cmp.Or(heliosApp.Spec.GitOpsBranch, "main") | ||
|
|
||
| spec := map[string]any{ | ||
| "project": project, | ||
| "source": map[string]any{ | ||
| "repoURL": shared.RewriteGiteaURL(heliosApp.Spec.GitOpsRepo), | ||
| "targetRevision": gitOpsBranch, | ||
| "path": heliosApp.Spec.GitOpsPath, | ||
| }, | ||
| "destination": map[string]any{ | ||
| "server": "https://kubernetes.default.svc", | ||
| "namespace": heliosApp.Namespace, | ||
| }, | ||
| "syncPolicy": map[string]any{ | ||
| "automated": map[string]any{ | ||
| "prune": true, | ||
| "selfHeal": true, | ||
| }, | ||
| "syncOptions": []any{ | ||
| "CreateNamespace=true", | ||
| }, | ||
| }, | ||
| "ignoreDifferences": []any{ | ||
| map[string]any{ | ||
| "group": "apps", | ||
| "kind": "Deployment", | ||
| "jqPathExpressions": []any{ | ||
| `.spec.template.spec.containers[].env[]? | select(.name | test("^DB_"))`, | ||
| }, | ||
| }, | ||
| }, | ||
| } | ||
|
|
||
| // Add PreSync hook if database trait exists | ||
| if HasDatabaseTrait(heliosApp) { | ||
| spec["syncPolicy"] = map[string]any{ | ||
| "automated": map[string]any{ | ||
| "prune": true, | ||
| "selfHeal": true, | ||
| }, | ||
| "syncOptions": []any{ | ||
| "CreateNamespace=true", | ||
| }, | ||
| } | ||
|
|
||
| // Add PreSync hook to application | ||
| // Note: PreSync Job is created and managed by PreSyncReconciler | ||
| // This is referenced via Job annotations, not stored in Application spec | ||
| syncPolicy := spec["syncPolicy"].(map[string]any) | ||
| syncPolicy["syncOptions"] = append( | ||
| syncPolicy["syncOptions"].([]any), | ||
| "SkipDryRunOnMissingResource=true", | ||
| ) | ||
| spec["syncPolicy"] = syncPolicy | ||
| } | ||
|
Comment on lines
+50
to
+71
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win Code thêm Lines 52-60 gán lại ♻️ Đề xuất đơn giản hóa // Add PreSync hook if database trait exists
if HasDatabaseTrait(heliosApp) {
- spec["syncPolicy"] = map[string]any{
- "automated": map[string]any{
- "prune": true,
- "selfHeal": true,
- },
- "syncOptions": []any{
- "CreateNamespace=true",
- },
- }
-
- // Add PreSync hook to application
- // Note: PreSync Job is created and managed by PreSyncReconciler
- // This is referenced via Job annotations, not stored in Application spec
- syncPolicy := spec["syncPolicy"].(map[string]any)
- syncPolicy["syncOptions"] = append(
- syncPolicy["syncOptions"].([]any),
+ // Add SkipDryRunOnMissingResource for PreSync Job support
+ syncPolicy := spec["syncPolicy"].(map[string]any)
+ syncPolicy["syncOptions"] = append(
+ syncPolicy["syncOptions"].([]any),
"SkipDryRunOnMissingResource=true",
)
- spec["syncPolicy"] = syncPolicy
}🤖 Prompt for AI Agents |
||
|
|
||
| app := map[string]any{ | ||
| "apiVersion": "argoproj.io/v1alpha1", | ||
| "kind": "Application", | ||
|
|
@@ -26,36 +81,7 @@ func GenerateArgoApplication(heliosApp *appv1alpha1.HeliosApp) (*unstructured.Un | |
| "app.kubernetes.io/managed-by": "helios-operator", | ||
| }, | ||
| }, | ||
| "spec": map[string]any{ | ||
| "project": project, | ||
| "source": map[string]any{ | ||
| "repoURL": shared.RewriteGiteaURL(heliosApp.Spec.GitOpsRepo), | ||
| "targetRevision": gitOpsBranch, | ||
| "path": heliosApp.Spec.GitOpsPath, | ||
| }, | ||
| "destination": map[string]any{ | ||
| "server": "https://kubernetes.default.svc", | ||
| "namespace": heliosApp.Namespace, | ||
| }, | ||
| "syncPolicy": map[string]any{ | ||
| "automated": map[string]any{ | ||
| "prune": true, | ||
| "selfHeal": true, | ||
| }, | ||
| "syncOptions": []any{ | ||
| "CreateNamespace=true", | ||
| }, | ||
| }, | ||
| "ignoreDifferences": []any{ | ||
| map[string]any{ | ||
| "group": "apps", | ||
| "kind": "Deployment", | ||
| "jqPathExpressions": []any{ | ||
| `.spec.template.spec.containers[].env[]? | select(.name | test("^DB_"))`, | ||
| }, | ||
| }, | ||
| }, | ||
| }, | ||
| "spec": spec, | ||
| } | ||
|
|
||
| return &unstructured.Unstructured{Object: app}, nil | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 422
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 747
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 6353
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 11237
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 121
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 7837
🏁 Script executed:
Repository: helios-platform-team/helios-platform
Length of output: 3046
Pin tag cho image
controllerthay vì dùnglatestTrong
apps/operator/config/manager/kustomization.yamlđang set:Ngoài ra
apps/operator/MakefilecóIMG ?= controller:latestvàmake deploy/installchạykustomize edit set image controller=${IMG}, nên nếu không truyềnIMGthì bản deploy sẽ luôn dùnglatest, làm giảm reproducibility/khó rollback-debug ở production. Nên pin version cụ thể (hoặc bắt buộc truyềnIMGtừ CI) thay vì mặc địnhlatest.🤖 Prompt for AI Agents