[v17] Backport Azure integration in Discovery Service to v17 by mvbrock · Pull Request #51725 · gravitational/teleport

mvbrock · 2025-01-31T20:26:18Z

Per https://github.com/gravitational/access-graph/issues/1455, enterprise users running Teleport v17 will want to use this new feature. This PR backports the merge commits from each of the following PRs to the v17 branch:

Changelog: Backport new Azure integration functionality to v17, which allows the Discovery Service to fetch Azure resources and send them to the Access Graph.

public-teleport-github-review-bot · 2025-02-01T03:06:40Z

@mvbrock - this PR will require admin approval to merge due to its size. Consider breaking it up into a series smaller changes.

tigrato

@mvbrock before merging, can you please test and ensure everything works as expected?

* Adding Azure integration gRPC messages and RPC methods * Make derive * Update proto/accessgraph/v1alpha/azure.proto Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * Update proto/accessgraph/v1alpha/azure.proto Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * Update proto/accessgraph/v1alpha/azure.proto Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * More PR feedback and generating protobuf code * Make derive * Adding identities field to principals, condition to role assignments, and role name to role definitions * Rebase conflicts * Did not fully fetch from origin/master when rebasing * Removing azure config field and keeping poll_interval as-is * Correct from parent branch * Apply suggestions from code review Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * Adding doc comments to access graph proto * Adding object type to principals * Adding location to Azure virtual machines * Update proto/accessgraph/v1alpha/access_graph_service.proto Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * Moving Azure Discovery protobuf config to the Azure Discovery PR * Make grpc --------- Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>

* Protobuf and configuration for Access Graph Azure Discovery * Adding godoc and removing Integration field from fileconf

…nctionality (#50366) * Protobuf and configuration for Access Graph Azure Discovery * Adding the Azure sync module functions along with new cloud client functionality * Forgot to decouple role definitions fetching function from the fetcher * Moving reconciliation to the upstream azure sync PR * Moving reconciliation test to the upstream azure sync PR * Updating go.sum * Fixing rebase after protobuf gen * Nolinting until upstream PRs * Updating to use existing msgraph client * Adding protection around nil values * PR feedback * Updating principal fetching to incorporate metadata from principal subtypes * Updating opts to not leak URL parameters * Conformant package name * Using variadic options * PR feedback * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * Also returning expanded principals for improved readability * Removing ptrToList * PR feedback * Rebase go.sum stuff * Go mod tidy * Linting * Linting * Collecting errors from fetching memberships and using a WithContext error group * Fixing go.mod * Update lib/msgraph/paginated.go Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * e ref update * e ref update * Fixing method * Fetching group members from groups rather than memberships of each principal * Linting --------- Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>

#50367) * Protobuf and configuration for Access Graph Azure Discovery * Adding the Azure sync module functions along with new cloud client functionality * Moving reconciliation to the upstream azure sync PR * Moving reconciliation test to the upstream azure sync PR * Fixing rebase after protobuf gen * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * Go mod tidy * Fixing go.mod * Update lib/msgraph/paginated.go Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * e ref update * Adding the Azure sync module functions along with new cloud client functionality * Protobuf and configuration for Access Graph Azure Discovery * Adding Azure sync functionality which can be called by the Azure fetcher * Protobuf update * Update sync process to use msgraph client * Conformant package name * Invoking membership expansion * Setting principals before expansion * Removing msgraphclient * Update e ref * Linting * PR feedback * Adding test names to reconciliation tests * Adding channel buffer * Going back to just reading from channel * Linting * PR feedback * PR feedback * PR feedback * Apply suggestions from code review Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * Fixing flaky test * Lint * Fix imports --------- Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>

* Protobuf and configuration for Access Graph Azure Discovery * Adding the Azure sync module functions along with new cloud client functionality * Fixing rebase after protobuf gen * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * Go mod tidy * Fixing go.mod * Update lib/msgraph/paginated.go Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * Adding the Azure sync module functions along with new cloud client functionality * Protobuf and configuration for Access Graph Azure Discovery * Adding Azure sync functionality which can be called by the Azure fetcher * Protobuf update * Invoking membership expansion * Setting principals before expansion * Removing msgraphclient * Linting * PR feedback * PR feedback * Adding the Azure sync module functions along with new cloud client functionality * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * PR feedback * Adding the Azure sync module functions along with new cloud client functionality * Protobuf and configuration for Access Graph Azure Discovery * Invoking the Azure fetcher in the Discovery service * Protobuf gen fix * Conformant package name * Removing msgraphclient (again?) * Rebase fixes * More cleanup * PR feedback --------- Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>

* Protobuf and configuration for Access Graph Azure Discovery * Fixing rebase after protobuf gen * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * Go mod tidy * Fixing go.mod * Update lib/msgraph/paginated.go Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * Protobuf and configuration for Access Graph Azure Discovery * Adding Azure sync functionality which can be called by the Azure fetcher * Protobuf update * Linting * PR feedback * PR feedback * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * PR feedback * Protobuf and configuration for Access Graph Azure Discovery * Protobuf gen fix * Rebase fixes * More cleanup * e ref update * Invoking token generation and returning the response * Quick test with a message to make sure RPC is invoked * Skeleton of new Azure OIDC RPC call * Fetching the Azure OIDC token during fetcher creation and establishing a credential assertion approach * PR feedback; restricting token requests to auth, discovery, and proxy roles. * Lint * Fixing mocks * Fix imports * Fix test * Rebase fxes * Adding back OIDC fetching, accidentally removed it during rebase * e ref * Lint * Fix imports --------- Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>

* Protobuf and configuration for Access Graph Azure Discovery * Fixing rebase after protobuf gen * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * Go mod tidy * Fixing go.mod * Update lib/msgraph/paginated.go Co-authored-by: Tiago Silva <tiago.silva@goteleport.com> * PR feedback * Protobuf and configuration for Access Graph Azure Discovery * Adding Azure sync functionality which can be called by the Azure fetcher * Protobuf update * Linting * PR feedback * PR feedback * Updating to use existing msgraph client * PR feedback * Using variadic options * Removing memberOf expansion * Expanding memberships by calling memberOf on each user * PR feedback * Rebase go.sum stuff * PR feedback * Protobuf and configuration for Access Graph Azure Discovery * Invoking the Azure fetcher in the Discovery service * Protobuf gen fix * Rebase fixes * More cleanup * PR feedback * Invoking token generation and returning the response * Fetching the Azure OIDC token during fetcher creation and establishing a credential assertion approach * PR feedback; restricting token requests to auth, discovery, and proxy roles. * Lint * Rebase fxes * Adding back OIDC fetching, accidentally removed it during rebase * Initial refactoring to include Azure status reporting * Converging status sync between AWS and Azure * Fixing test * Sending usage stats * Fix imports * Add godocs and correct a few comments * Removing the usage events for now --------- Co-authored-by: Tiago Silva <tiago.silva@goteleport.com>

* Initial command to create the managed identity and role * Adding permissions and applying command params * Adding graph permissions to the MSI * Updating parameters * Adding some details and cleaning up comments * Fixing go.sum * Linting * License * PR feedback * Decoupling sync config with an interface for testing * Tweaks to test mocking * PR feedback * Rebase adjustments * PR feedback * Switch to empty struct maps instead of bool maps for set representation * Godocs * Adding user agent to Azure SDK requests * Linting

mvbrock changed the title ~~Mvbrock/backport azure integration/v17~~ [v17] Backport Azure integration in Discovery Service to v17 Jan 31, 2025

mvbrock marked this pull request as ready for review February 1, 2025 03:06

github-actions Bot requested review from creack, marcoandredinis, rosstimothy and tigrato February 1, 2025 03:06

github-actions Bot added backport discovery kubernetes-access size/xl labels Feb 1, 2025

marcoandredinis approved these changes Feb 3, 2025

View reviewed changes

tigrato approved these changes Feb 3, 2025

View reviewed changes

r0mant approved these changes Feb 5, 2025

View reviewed changes

mvbrock and others added 13 commits February 5, 2025 13:30

Post cherry-pick grpc

18a4578

Protobuf and configuration for Access Graph Azure Discovery (#50364)

5d4e804

* Protobuf and configuration for Access Graph Azure Discovery * Adding godoc and removing Integration field from fileconf

Post cherry-pick fixes

9ce9d38

Moving armcompute back to v3

d84c484

Post cherry-pick make grpc

8c0c514

Post rebase make grpc

431253a

mvbrock force-pushed the mvbrock/backport-azure-integration/v17 branch from ec2f4c7 to 431253a Compare February 5, 2025 19:31

public-teleport-github-review-bot Bot removed request for creack and rosstimothy February 5, 2025 19:32

mvbrock added this pull request to the merge queue Feb 5, 2025

Merged via the queue into branch/v17 with commit a4ae30e Feb 5, 2025

mvbrock deleted the mvbrock/backport-azure-integration/v17 branch February 5, 2025 20:25

camscale mentioned this pull request Feb 13, 2025

Release 17.2.7 #52158

Merged

BrewTestBot mentioned this pull request Feb 14, 2025

teleport 17.2.7 Homebrew/homebrew-core#207667

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v17] Backport Azure integration in Discovery Service to v17#51725

[v17] Backport Azure integration in Discovery Service to v17#51725
mvbrock merged 13 commits intobranch/v17from
mvbrock/backport-azure-integration/v17

mvbrock commented Jan 31, 2025 •

edited

Loading

Uh oh!

public-teleport-github-review-bot Bot commented Feb 1, 2025

Uh oh!

tigrato left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

mvbrock commented Jan 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

public-teleport-github-review-bot Bot commented Feb 1, 2025

Uh oh!

tigrato left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mvbrock commented Jan 31, 2025 •

edited

Loading