Move idempotency_records schema to v4 and add H2 support#3386
Move idempotency_records schema to v4 and add H2 support#3386singhpk234 merged 4 commits intoapache:mainfrom
Conversation
singhpk234
left a comment
singhpk234
left a comment
There was a problem hiding this comment.
LGTM, thanks @huaxingao
github-project-automation
bot
moved this from PRs In Progress
to Ready to merge
in Basic Kanban Board
| realm_id TEXT NOT NULL, | ||
| idempotency_key TEXT NOT NULL, | ||
| operation_type TEXT NOT NULL, | ||
| resource_id TEXT NOT NULL, |
There was a problem hiding this comment.
Should we allow resource id nullable? For example, if we are using table entity id as resource id, the entity id will be generated only after the table creation succeeded. When table creation failed(return 4xx), there may be no entity id generated.
There was a problem hiding this comment.
Thanks for the comment! You’re right that if resource_id were a generated entity id, it could be null on 4xx failures (no entity created). In this implementation, resource_id is intentionally request-derived and stable (a normalized identifier like tables/<ns>.<name>), not an internal entity id. That means it’s always available even on failures. Keeping it NOT NULL ensures the idempotency key is always tied to one specific resource.
I added a comment to resource_id TEXT NOT NULL to make it clear
resource_id TEXT NOT NULL, -- normalized request-derived resource identifier (not a generated entity id)
There was a problem hiding this comment.
That makes sense. I think the connection tables like grant_records and policy_mapping_record will need resource id to cover two normalized identifiers, e.g., a table identifier and a policy identifier, correct?
github-project-automation
bot
moved this from Ready to merge
to Done
in Basic Kanban Board
|
Thanks @huaxingao for the change and @flyrain for the review |
|
Thanks @singhpk234 @flyrain |
|
|
||
| MERGE INTO version (version_key, version_value) | ||
| KEY (version_key) | ||
| VALUES ('version', 3); |
There was a problem hiding this comment.
Should this version be set to 4? @singhpk234 @dimas-b
There was a problem hiding this comment.
@obelix74 Thanks for catching this problem. I will fix this in next PR.
* Replace custom token-bucket implementation with Guava's `RateLimiter` (apache#3507) Addresses the issues discussed on the dev mailing-list discussion https://lists.apache.org/thread/gkyw7m4fcbjbzhcrlrp4kcq5lr05r0m4, opting to use Guava as the easiest replacement here. * Move idempotency_records schema to v4 and add H2 support (apache#3386) * Move idempotency_records schema to v4 and add H2 support * address comments and fix test failures * fix format * add comment to resource_id * (nit): Getting started examples with mc/s5cmd to aws cli (apache#3526) * Switch mc/s3cmd to aws cli * Switch mc/s3cmd to aws cli * Add support for no KMS with s3-compatible backend (apache#3501) * chore(deps): update amazon/aws-cli docker tag to v2.33.7 (apache#3558) * Update doc for helm around rateLimiter (apache#3562) * Disable renoavte update for python version (apache#3560) * Fix the Keycloak getting-started example for 26.5+ (apache#3568) The example was failing because Keycloak 26.5 introduced stricter validation rules for session lifespan and timeout. * NoSQL: Add to runtime-service (apache#3396) * NoSQL: Add to runtime-service This change adds the NoSQL persistence to polaris-runtime-service. * chore(deps): update amazon/aws-cli docker tag to v2.33.8 (apache#3575) * Add spark sql integration test for Hudi (apache#3194) * Fix ozone getting started example (apache#3574) * Fix Ozone getting started example * Fix Ozone getting started example * Change AWS CLI image to weekly (apache#3578) * fix(deps): update dependency com.diffplug.spotless:spotless-plugin-gradle to v8.2.1 (apache#3576) * chore(deps): update registry.access.redhat.com/ubi9/openjdk-21-runtime docker tag to v1.24-2.1769108682 (apache#3588) * removed references of BEFORE/AFTER_COMMIT_VIEW (apache#3554) * nits - post-merge fixes * Last merged commit 2b0ca21 --------- Co-authored-by: Huaxin Gao <huaxin.gao11@gmail.com> Co-authored-by: Yong Zheng <yongzheng0809@gmail.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Alexandre Dutra <adutra@apache.org> Co-authored-by: Rahil C <32500120+rahil-c@users.noreply.github.com> Co-authored-by: Innocent Djiofack <djiofack007@gmail.com>
4 participants
Moves the idempotency_records table definition and indexes from schema-v3 to schema-v4 for Postgres, and adds the matching idempotency_records table and realm‑scoped expiry index to H2 schema-v4. This keeps both JDBC backends aligned on the v4 schema for idempotency persistence
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)