fix(policy): Return the correct total during list responses. #2836

c-r33d · 2025-10-27T15:53:19Z

Proposed Changes

1.) Correct all instances where Total is not being calculated properly within our list responses.

Checklist

I have added or updated unit tests
I have added or updated integration tests (if appropriate)
I have added or updated documentation

Testing Instructions

gemini-code-assist · 2025-10-27T15:53:49Z

Summary of Changes

Hello @c-r33d, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves an issue where the pagination Total field in list responses for various policy-related entities was inaccurately reported. The solution involves optimizing the underlying SQL queries to use a window function for precise total row counting, which is then verified by a suite of new, detailed integration tests covering different listing scenarios and filters.

Highlights

Pagination Total Count Fix: Refactored SQL queries for all list operations across policy entities (attributes, attribute values, namespaces, resource mapping groups, and resource mappings) to correctly calculate the total count for pagination responses.
SQL Query Optimization: Replaced the less efficient WITH counted AS (...) CROSS JOIN counted pattern with the COUNT(*) OVER() window function directly within the main SELECT statement, ensuring accurate total counts in a single pass.
Comprehensive Integration Tests: Added new integration tests for ListAttributeValues, ListAttributes, ListResourceMappingGroups, and ListResourceMappings to thoroughly validate the correctness of the pagination Total field under various filtering and state conditions.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Count was not quite right, Window function sets it straight, Tests now sing with joy.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request correctly fixes an issue where the Total count in paginated list responses was not being calculated properly when filters were applied. The change from using a CTE with CROSS JOIN to the COUNT(*) OVER() window function is a significant improvement, ensuring the total reflects the filtered dataset. The addition of new integration tests is also a great way to verify this fix and prevent future regressions. I've found one minor issue in a new test case.

service/integration/attribute_values_test.go

github-actions · 2025-10-27T15:58:11Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	184.393749ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	99.718015ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	370.31378ms
Throughput	270.04 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	39.466460382s
Average Latency	393.328053ms
Throughput	126.69 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	27.771354504s
Average Latency	276.650195ms
Throughput	180.04 requests/second

github-actions · 2025-10-27T16:00:50Z

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	178.406555ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric	Value
Approved Decision Requests	1000
Denied Decision Requests	0
Total Time	101.528935ms

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	360.303014ms
Throughput	277.54 requests/second

TDF3 Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	39.099305333s
Average Latency	389.671149ms
Throughput	127.88 requests/second

NANOTDF Benchmark Results:

Metric	Value
Total Requests	5000
Successful Requests	5000
Failed Requests	0
Concurrent Requests	50
Total Time	28.382373482s
Average Latency	282.835134ms
Throughput	176.17 requests/second

github-actions · 2025-10-27T16:09:26Z

X-Test Results

opentdf~~platform~~WJ8KJH.dockerbuild
cukes-report
✅ js-v0.4.34
✅ java-v0.4.34
✅ go-v0.4.34
bats-test-results
✅ js-pull-2836
✅ java-pull-2836
✅ go-pull-2836

github-actions · 2025-10-27T16:10:43Z

X-Test Results

opentdf~~platform~~VYAJGA.dockerbuild
cukes-report
✅ java-v0.4.34
✅ js-v0.4.34
✅ go-v0.4.34
bats-test-results
✅ js-pull-2836
✅ java-pull-2836
✅ go-pull-2836

jakedoublev

🔥

c-r33d · 2025-10-29T16:11:01Z

/backport

c-r33d · 2025-10-29T16:13:46Z

/backport

### Proposed Changes 1.) Correct all instances where `Total` is not being calculated properly within our list responses. ### Checklist - [ ] I have added or updated unit tests - [ ] I have added or updated integration tests (if appropriate) - [ ] I have added or updated documentation ### Testing Instructions (cherry picked from commit 5c1ec9c)

opentdf-automation · 2025-10-29T16:14:03Z

Successfully created backport PR for release/service/v0.11:

fix(policy): Return the correct total during list responses. [backport to release/service/v0.11] #2843

### Proposed Changes 1.) Correct all instances where `Total` is not being calculated properly within our list responses. ### Checklist - [ ] I have added or updated unit tests - [ ] I have added or updated integration tests (if appropriate) - [ ] I have added or updated documentation ### Testing Instructions (cherry picked from commit 5c1ec9c)

…t to release/service/v0.11] (#2843) # Description Backport of #2836 to `release/service/v0.11`. Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>

fix(policy): Return the correct total during list responses.

04bcb25

c-r33d requested review from a team as code owners October 27, 2025 15:53

github-actions bot added comp:db DB component comp:policy Policy Configuration ( attributes, subject mappings, resource mappings, kas registry) size/m labels Oct 27, 2025

gemini-code-assist bot reviewed Oct 27, 2025

View reviewed changes

service/integration/attribute_values_test.go Outdated Show resolved Hide resolved

fix attribute values list response test.

07fcc21

jakedoublev approved these changes Oct 28, 2025

View reviewed changes

c-r33d added this pull request to the merge queue Oct 28, 2025

Merged via the queue into main with commit 5c1ec9c Oct 28, 2025
62 of 64 checks passed

c-r33d deleted the fix/DSPX-1788-list-rpcs branch October 28, 2025 13:51

opentdf-automation bot mentioned this pull request Oct 29, 2025

chore(main): release service 0.12.0 #2841

Open

c-r33d added the backport/service/v0.10 label Oct 29, 2025

c-r33d added backport release/service/v0.11 and removed backport/service/v0.10 labels Oct 29, 2025

opentdf-automation bot mentioned this pull request Oct 29, 2025

fix(policy): Return the correct total during list responses. [backport to release/service/v0.11] #2843

Merged

fix(policy): Return the correct total during list responses. #2836

fix(policy): Return the correct total during list responses. #2836

Uh oh!

Conversation

c-r33d commented Oct 27, 2025

Proposed Changes

Checklist

Testing Instructions

Uh oh!

gemini-code-assist bot commented Oct 27, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

github-actions bot commented Oct 27, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

github-actions bot commented Oct 27, 2025

Benchmark authorization.GetDecisions Results:

Benchmark authorization.v2.GetMultiResourceDecision Results:

Standard Benchmark Metrics Skipped or Failed

Bulk Benchmark Results

TDF3 Benchmark Results:

NANOTDF Benchmark Results:

Uh oh!

github-actions bot commented Oct 27, 2025

X-Test Results

Uh oh!

github-actions bot commented Oct 27, 2025

X-Test Results

Uh oh!

jakedoublev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

c-r33d commented Oct 29, 2025

Uh oh!

c-r33d commented Oct 29, 2025

Uh oh!

opentdf-automation bot commented Oct 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants