Skip to content

Optimize KMS key reconciliation process#298

Merged
jimlambrt merged 5 commits intomainfrom
ICU-17643
Jan 14, 2026
Merged

Optimize KMS key reconciliation process#298
jimlambrt merged 5 commits intomainfrom
ICU-17643

Conversation

@gc-tejas
Copy link
Copy Markdown
Contributor

@gc-tejas gc-tejas commented Dec 17, 2025

This pull request enhances the KMS key reconciliation process by optimizing how missing data keys are detected and handled across multiple scopes and purposes. The main improvement is the introduction of a new repository method that efficiently identifies which scopes lack specific data keys, enabling batch processing and reducing unnecessary lookups. Comprehensive tests have also been added to ensure correctness, including edge cases and large batches. This addresses ICU-17643 and ICU-17641

Key improvements to key reconciliation logic:

  • Added a new repository method, ListScopesMissingDataKey, which efficiently returns a map of missing data keys for each purpose across a list of scopes, supporting batching for large requests.
  • Refactored the Kms.ReconcileKeys method to use the new repository method, allowing it to only process scopes and purposes that are actually missing data keys, improving performance and clarity.

Testing improvements:

  • Added comprehensive tests for ListScopesMissingDataKey, covering cases such as missing keys, missing root keys, all keys present, mixed scenarios, and large batch operations.
  • Extended key reconciliation tests to cover scenarios with multiple scopes and mixed missing keys, as well as large batch processing.

Supporting changes:

  • Added query, postgresScopesMissingDataKeyQuery sqliteScopesMissingDataKeyQuery, to efficiently find missing data keys for given scopes and purposes.
  • Added necessary imports and minor code cleanups to support new functionality and tests.

@hashicorp-cla-app
Copy link
Copy Markdown

hashicorp-cla-app Bot commented Dec 17, 2025

CLA assistant check
All committers have signed the CLA.

Comment thread extras/kms/repository.go Outdated
Comment thread extras/kms/repository.go Outdated
@gc-tejas gc-tejas requested a review from tmessi December 17, 2025 12:10
@gc-tejas gc-tejas marked this pull request as ready for review December 17, 2025 12:11
ChandraSGadde
ChandraSGadde previously approved these changes Dec 18, 2025
Comment thread extras/kms/repository.go Outdated
@jimlambrt jimlambrt merged commit c729b81 into main Jan 14, 2026
20 checks passed
@jimlambrt jimlambrt deleted the ICU-17643 branch January 14, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants