feat(policy): enhance subject mappings with subject condition sets #321

jakedoublev · 2024-03-06T18:35:02Z

This PR includes:

subject condition sets simplification of DB schema in ERD/migration/protos
support of subject condition sets after schema rework in protos
DB layer/storage of subject condition sets relation to subject mappings
SCS service RPCs and support for creation/updation within subject mappings
data layer integration tests
make sure every time pgx.Rows are accessed that there is a deferred Close call
make sure extending metadata checks that previously set labels exist before trying to extend them
move tables out of db package and into individual db packages that consume the shared db client interface and set their own

Closes #295

…dard/custom actions in protos and unify fixtures to that schema

…lize one to many relation

…tion sets

…tting/listing subject mappings (and test for it)

…icking with pattern of putting any name within labels

internal/db/db.go

internal/db/errors.go

jakedoublev · 2024-03-06T20:44:21Z

Moving this back to draft and will merge #314 first, then handle any conflicts, so that subject mappings conform to the API behavior standardized across the policy services.

…ubject condition sets, and reflect that in tests and service rpcs

… in the protos

…metadata behavior decisions

…ta update behavior is to extend but no metadata labels were actually previously available

…the subject condition sets table

internal/db/db.go

internal/db/errors.go

internal/db/marshalHelpers.go

internal/fixtures/fixtures.go

🤖 I have created a release *beep* *boop* --- ## [0.1.0](protocol/go-v0.1.0...protocol/go/v0.1.0) (2024-04-22) ### Features * **attr value lookup by fqn:** adds GetAttributesByFqns rpc in attributes service [#243](#243) ([#250](#250)) ([b810d33](b810d33)) * **auth:** add authorization via casbin ([#417](#417)) ([292f2bd](292f2bd)) * **authorization service:** Gets the attributes from the in-memory service connection inside the GetDecisions request ([#273](#273)) ([ce57117](ce57117)) * **authorization:** entitlements ([#247](#247)) ([42c4f27](42c4f27)) * **core:** exposes new well-known configuration endpoint ([#299](#299)) ([d52cd21](d52cd21)) * **idp-add-on:** PLAT-3005 Add keycloak idp add on and idp add on protos ([#233](#233)) ([2365e61](2365e61)) * **kas:** authorization decisions ([#431](#431)) ([82e8895](82e8895)) * **PLAT-2950:** Update buf generated interface code for java ([#240](#240)) ([d7e2642](d7e2642)) * **policy object selectors:** adds initial selector protos, moves policy object type messages to top-level to avoid circular imports, and provides subject mappings in response to GetAttributeValuesByFqns ([#372](#372)) ([e9d9241](e9d9241)) * **policy subject mappings condition sets / migrations:** adds DB schema, fixes migrate down command, adds migrate up command, bumps goose ([#286](#286)) ([4d7a032](4d7a032)) * **policy:** adds support for match subject request to get entitlements without FQN scopes ([#347](#347)) ([63c34a5](63c34a5)) * **policy:** enhance and expand metadata and normalize API ([#314](#314)) ([9389f3b](9389f3b)) * **policy:** enhance subject mappings with subject condition sets ([#321](#321)) ([df692eb](df692eb)) * **policy:** list attrs by namespace ([#479](#479)) ([92d8f8c](92d8f8c)) * **policy:** list attrs by namespace name ([#487](#487)) ([04e723f](04e723f)) * **policy:** rework attribute value members ([#398](#398)) ([1cb7d0c](1cb7d0c)) * **policy:** support attribute value creation ([#454](#454)) ([432ee6b](432ee6b)) * **policy:** update fixtures, proto comments, and proto field names to reflect use of jq selector syntax within Conditions of Subject Sets ([#523](#523)) ([16f40f7](16f40f7)) ### Bug Fixes * **authorization:** remove access pdp internal AttributeInstance type and use policy proto generated struct types instead ([#485](#485)) ([8435f59](8435f59)) * **policy:** Adds policy package infix ([#280](#280)) ([57e8ef9](57e8ef9)) * **protos:** authorization service's ResourceAttribute message should map to updated platform policy schema ([#238](#238)) ([bf381dc](bf381dc)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- ## [0.1.0](opentdf/platform@protocol/go-v0.1.0...protocol/go/v0.1.0) (2024-04-22) ### Features * **attr value lookup by fqn:** adds GetAttributesByFqns rpc in attributes service [#243](opentdf/platform#243) ([#250](opentdf/platform#250)) ([b810d33](opentdf/platform@b810d33)) * **auth:** add authorization via casbin ([#417](opentdf/platform#417)) ([292f2bd](opentdf/platform@292f2bd)) * **authorization service:** Gets the attributes from the in-memory service connection inside the GetDecisions request ([#273](opentdf/platform#273)) ([ce57117](opentdf/platform@ce57117)) * **authorization:** entitlements ([#247](opentdf/platform#247)) ([42c4f27](opentdf/platform@42c4f27)) * **core:** exposes new well-known configuration endpoint ([#299](opentdf/platform#299)) ([d52cd21](opentdf/platform@d52cd21)) * **idp-add-on:** PLAT-3005 Add keycloak idp add on and idp add on protos ([#233](opentdf/platform#233)) ([2365e61](opentdf/platform@2365e61)) * **kas:** authorization decisions ([#431](opentdf/platform#431)) ([82e8895](opentdf/platform@82e8895)) * **PLAT-2950:** Update buf generated interface code for java ([#240](opentdf/platform#240)) ([d7e2642](opentdf/platform@d7e2642)) * **policy object selectors:** adds initial selector protos, moves policy object type messages to top-level to avoid circular imports, and provides subject mappings in response to GetAttributeValuesByFqns ([#372](opentdf/platform#372)) ([e9d9241](opentdf/platform@e9d9241)) * **policy subject mappings condition sets / migrations:** adds DB schema, fixes migrate down command, adds migrate up command, bumps goose ([#286](opentdf/platform#286)) ([4d7a032](opentdf/platform@4d7a032)) * **policy:** adds support for match subject request to get entitlements without FQN scopes ([#347](opentdf/platform#347)) ([63c34a5](opentdf/platform@63c34a5)) * **policy:** enhance and expand metadata and normalize API ([#314](opentdf/platform#314)) ([9389f3b](opentdf/platform@9389f3b)) * **policy:** enhance subject mappings with subject condition sets ([#321](opentdf/platform#321)) ([df692eb](opentdf/platform@df692eb)) * **policy:** list attrs by namespace ([#479](opentdf/platform#479)) ([92d8f8c](opentdf/platform@92d8f8c)) * **policy:** list attrs by namespace name ([#487](opentdf/platform#487)) ([04e723f](opentdf/platform@04e723f)) * **policy:** rework attribute value members ([#398](opentdf/platform#398)) ([1cb7d0c](opentdf/platform@1cb7d0c)) * **policy:** support attribute value creation ([#454](opentdf/platform#454)) ([432ee6b](opentdf/platform@432ee6b)) * **policy:** update fixtures, proto comments, and proto field names to reflect use of jq selector syntax within Conditions of Subject Sets ([#523](opentdf/platform#523)) ([16f40f7](opentdf/platform@16f40f7)) ### Bug Fixes * **authorization:** remove access pdp internal AttributeInstance type and use policy proto generated struct types instead ([#485](opentdf/platform#485)) ([8435f59](opentdf/platform@8435f59)) * **policy:** Adds policy package infix ([#280](opentdf/platform#280)) ([57e8ef9](opentdf/platform@57e8ef9)) * **protos:** authorization service's ResourceAttribute message should map to updated platform policy schema ([#238](opentdf/platform#238)) ([bf381dc](opentdf/platform@bf381dc)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>

jakedoublev added 18 commits March 1, 2024 12:56

remove unused function tableName from db package

e7a7300

new protos and gen code/docs

810a9ba

store ACTIONS as JSONB instead of VARCHAR array to support oneof stan…

8d9129c

…dard/custom actions in protos and unify fixtures to that schema

checkpoint after schema relation update to remove pivot table and uti…

5c1e3ed

…lize one to many relation

cleanup and diagram notes update

d546c4d

make sure to always close rows

42415e4

sync up new create/update behavior with policy api norms

0ed6cbc

map to new types

1e3cfe3

db layer and integration tests so far (passing)

1303e2d

fully working integration tests for subjectmappings and subject condi…

2d147b9

…tion sets

remove extraneous error check

8aa4351

follow new policy proto norms for subject condition sets

1e24496

Merge branch 'main' into policy/subject-sets

4def80e

provide back active state of attribute values with attributes when ge…

1a55c66

…tting/listing subject mappings (and test for it)

cleanup

25e87e1

clean subjectmappings proto comments

f9db1b2

remove name as a column and proto field on subject condition sets, st…

ab555ce

…icking with pattern of putting any name within labels

cleanup

32cd6ae

jakedoublev changed the title ~~feat(policy subject mappings/subject sets): enhance subject mappings with subject condition sets~~ feat(policy): enhance subject mappings with subject condition sets Mar 6, 2024

jakedoublev linked an issue Mar 6, 2024 that may be closed by this pull request

Add data schema and service logic to support Subject Mapping SubjectSet enhancements #171

Closed

jakedoublev marked this pull request as ready for review March 6, 2024 19:38

jakedoublev requested a review from a team as a code owner March 6, 2024 19:38

jrschumacher reviewed Mar 6, 2024

View reviewed changes

internal/db/db.go Outdated Show resolved Hide resolved

pflynn-virtru reviewed Mar 6, 2024

View reviewed changes

internal/db/errors.go Outdated Show resolved Hide resolved

pflynn-virtru previously approved these changes Mar 6, 2024

View reviewed changes

jakedoublev marked this pull request as draft March 6, 2024 20:43

make delete, update, create only return id for subject mappings and s…

71f46dd

…ubject condition sets, and reflect that in tests and service rpcs

jakedoublev dismissed pflynn-virtru’s stale review via 71f46dd March 6, 2024 21:47

Merge branch 'main' into policy/subject-sets

d1c26f3

jakedoublev added 14 commits March 6, 2024 17:31

move kasr db out of internal db package

10756da

one more test

3f726f3

Merge branch 'main' into policy/subject-sets

1a08410

Merge branch 'main' into policy/subject-sets

865b067

Merge branch 'main' into policy/subject-sets

c122774

sync up create/update/delete response with rest of policy API updates…

d2cd8f5

… in the protos

sync up subject condition set RPCs as well

77bf8ba

match rpc handlers to proto expectations

a871c40

tiny test fixes to namespaces after merging

795e626

further sync up with Policy API proto/rpc behavior

b781aac

return all values on create, id on update/delete, unify comments and …

025e113

…metadata behavior decisions

update tests and fix potential nil pointer exception where the metada…

592ab92

…ta update behavior is to extend but no metadata labels were actually previously available

fix conflict around migration names and add the timestamp columns to …

96ca42f

…the subject condition sets table

Merge branch 'main' into policy/subject-sets

92cc666

jakedoublev marked this pull request as ready for review March 11, 2024 15:55

remove extraneous comment

327c1f4

pflynn-virtru previously approved these changes Mar 11, 2024

View reviewed changes

jrschumacher reviewed Mar 11, 2024

View reviewed changes

internal/db/db.go Outdated Show resolved Hide resolved

internal/db/errors.go Outdated Show resolved Hide resolved

internal/db/marshalHelpers.go Show resolved Hide resolved

internal/fixtures/fixtures.go Outdated Show resolved Hide resolved

handle key not in fixtures map edge case

ef50138

jakedoublev dismissed pflynn-virtru’s stale review via ef50138 March 11, 2024 17:44

jrschumacher approved these changes Mar 11, 2024

View reviewed changes

jakedoublev added this pull request to the merge queue Mar 11, 2024

Merged via the queue into main with commit df692eb Mar 11, 2024

jakedoublev deleted the policy/subject-sets branch March 11, 2024 18:25

jakedoublev mentioned this pull request Mar 11, 2024

ADR: Policy API performance, open ended querying for entitlements (MatchedSubjectMappings) has known performance concerns at scale #365

Closed

opentdf-automation bot mentioned this pull request Apr 18, 2024

chore(main): release protocol/go 0.1.0 #615

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(policy): enhance subject mappings with subject condition sets #321

feat(policy): enhance subject mappings with subject condition sets #321

Uh oh!

jakedoublev commented Mar 6, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

jakedoublev commented Mar 6, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

feat(policy): enhance subject mappings with subject condition sets #321

feat(policy): enhance subject mappings with subject condition sets #321

Uh oh!

Conversation

jakedoublev commented Mar 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jakedoublev commented Mar 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jakedoublev commented Mar 6, 2024 •

edited

Loading

jakedoublev commented Mar 6, 2024 •

edited

Loading