Skip to content

Implement semantic search API for unified resources#31881

Merged
xacrimon merged 12 commits intomasterfrom
joel/smart-search-scrap
Oct 2, 2023
Merged

Implement semantic search API for unified resources#31881
xacrimon merged 12 commits intomasterfrom
joel/smart-search-scrap

Conversation

@xacrimon
Copy link
Copy Markdown
Contributor

@xacrimon xacrimon commented Sep 14, 2023

This PR implements a new API on the Assist gRPC service called SearchUnifiedResources. This API is gated behind assist being enabled together with all other Assist APIs and is off unless Assist is otherwise enabled || you're on Cloud Team plan.

This API searches the unified resource cache already present on the Auth server and returns resources of one or more kinds that the user has access to, ordered by semantic closeness. Otherwise it behaves very similar to existing resource fetch APIs. Since the kind of resource the user wants is not always obvious, this API is multi-kind i.e it supports returning resources of more than one kind and we can piggyback off embedding distance to guess the most reasonable resource kind.

@xacrimon xacrimon requested a review from jakule September 20, 2023 14:54
@xacrimon xacrimon marked this pull request as ready for review September 20, 2023 14:54
@github-actions github-actions Bot requested review from r0mant and tcsc September 20, 2023 14:55
@xacrimon xacrimon force-pushed the joel/smart-search-scrap branch from 2210aa1 to ddcf2f6 Compare September 20, 2023 15:07
@xacrimon xacrimon self-assigned this Sep 20, 2023
@xacrimon xacrimon force-pushed the joel/smart-search-scrap branch 2 times, most recently from 521d88d to 8238b3c Compare September 21, 2023 01:46
@xacrimon
Copy link
Copy Markdown
Contributor Author

Huh, this is odd. If I reset the go.mod to be equiv to master I keep getting weird failures and go.mod tidy wants to revert versions?

@xacrimon
Copy link
Copy Markdown
Contributor Author

xacrimon commented Sep 21, 2023

Above should be fixed now. Still trying to repro and fix the flaky test locally but can't get the same failure to occur.

Copy link
Copy Markdown
Contributor

@jakule jakule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above should be fixed now. Still trying to repro and fix the flaky test locally but can't get the same failure to occur.

The flaky test detector is taming out because this test takes 10 seconds to execute.

Comment thread lib/auth/assist/assistv1/test/service_test.go Outdated
Comment thread lib/auth/assist/assistv1/test/service_test.go Outdated
Comment thread lib/auth/assist/assistv1/test/service_test.go Outdated
@jakule jakule requested a review from avatus September 21, 2023 23:59
@xacrimon xacrimon requested a review from jakule September 27, 2023 13:49
@xacrimon xacrimon force-pushed the joel/smart-search-scrap branch from 25b1b44 to 50948fe Compare September 27, 2023 13:54
@avatus
Copy link
Copy Markdown
Contributor

avatus commented Sep 28, 2023

Doesn't relate directly to this PR, but I'm proposing some changes to the unified resource cache here that might be useful to your interactions with it

@xacrimon
Copy link
Copy Markdown
Contributor Author

Thanks for the heads up @avatus !

Comment thread api/proto/teleport/assist/v1/assist.proto Outdated
Comment thread lib/auth/assist/assistv1/test/service_test.go Outdated
Joel and others added 2 commits October 2, 2023 14:59
@xacrimon xacrimon enabled auto-merge October 2, 2023 13:11
@xacrimon xacrimon disabled auto-merge October 2, 2023 13:11
@xacrimon xacrimon enabled auto-merge October 2, 2023 13:14
@xacrimon xacrimon added this pull request to the merge queue Oct 2, 2023
Merged via the queue into master with commit fb77d4f Oct 2, 2023
@xacrimon xacrimon deleted the joel/smart-search-scrap branch October 2, 2023 20:16
xacrimon pushed a commit that referenced this pull request Oct 2, 2023
* Implement semantic search API for unified resources

* use require.eventually to complete test faster

* reset mod and sum

* Update lib/auth/assist/assistv1/test/service_test.go

Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>

* use correct proto numbering

* reset e

* make grpc

---------

Co-authored-by: Jakub Nyckowski <jakub.nyckowski@goteleport.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants