-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Adding the Azure sync module functions along with new cloud client functionality #50366
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
34 commits
Select commit
Hold shift + click to select a range
2540355
Protobuf and configuration for Access Graph Azure Discovery
mvbrock 64a4c18
Adding the Azure sync module functions along with new cloud client fu…
mvbrock 666b482
Forgot to decouple role definitions fetching function from the fetcher
mvbrock 64ee74c
Moving reconciliation to the upstream azure sync PR
mvbrock f4ed0e3
Moving reconciliation test to the upstream azure sync PR
mvbrock 597ba79
Updating go.sum
mvbrock 1edb6de
Fixing rebase after protobuf gen
mvbrock 0f7a405
Nolinting until upstream PRs
mvbrock 1fa09cf
Updating to use existing msgraph client
mvbrock 3b70c0b
Adding protection around nil values
mvbrock 69486a2
PR feedback
mvbrock 861b562
Updating principal fetching to incorporate metadata from principal su…
mvbrock 37e3dda
Updating opts to not leak URL parameters
mvbrock cce7007
Conformant package name
mvbrock d498564
Using variadic options
mvbrock 6726772
PR feedback
mvbrock bacbe49
Removing memberOf expansion
mvbrock fa9d8a6
Expanding memberships by calling memberOf on each user
mvbrock 8f07e78
Also returning expanded principals for improved readability
mvbrock f075bc8
Removing ptrToList
mvbrock 2b5b928
PR feedback
mvbrock 0572557
Rebase go.sum stuff
mvbrock 4959c11
Go mod tidy
mvbrock 8ac788c
Linting
mvbrock 194ae3e
Linting
mvbrock 1eacf07
Collecting errors from fetching memberships and using a WithContext e…
mvbrock 4cf25c4
Fixing go.mod
mvbrock ae6bba6
Update lib/msgraph/paginated.go
mvbrock aafeb05
PR feedback
mvbrock 8b3cd5d
e ref update
mvbrock c2319c6
e ref update
mvbrock acf1415
Fixing method
mvbrock 3c90b4d
Fetching group members from groups rather than memberships of each pr…
mvbrock c7cd683
Linting
mvbrock File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| /* | ||
| * Teleport | ||
| * Copyright (C) 2024 Gravitational, Inc. | ||
| * | ||
| * This program is free software: you can redistribute it and/or modify | ||
| * it under the terms of the GNU Affero General Public License as published by | ||
| * the Free Software Foundation, either version 3 of the License, or | ||
| * (at your option) any later version. | ||
| * | ||
| * This program is distributed in the hope that it will be useful, | ||
| * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| * GNU Affero General Public License for more details. | ||
| * | ||
| * You should have received a copy of the GNU Affero General Public License | ||
| * along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| */ | ||
|
|
||
| package azure | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/Azure/azure-sdk-for-go/sdk/azcore" | ||
| "github.com/Azure/azure-sdk-for-go/sdk/azcore/arm" | ||
| "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2" | ||
| "github.com/gravitational/trace" | ||
| ) | ||
|
|
||
| // RoleAssignmentsClient wraps the Azure API to provide a high level subset of functionality | ||
| type RoleAssignmentsClient struct { | ||
| cli *armauthorization.RoleAssignmentsClient | ||
| } | ||
|
|
||
| // NewRoleAssignmentsClient creates a new client for a given subscription and credentials | ||
| func NewRoleAssignmentsClient(subscription string, cred azcore.TokenCredential, options *arm.ClientOptions) (*RoleAssignmentsClient, error) { | ||
| clientFactory, err := armauthorization.NewClientFactory(subscription, cred, options) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
| roleDefCli := clientFactory.NewRoleAssignmentsClient() | ||
| return &RoleAssignmentsClient{cli: roleDefCli}, nil | ||
| } | ||
|
|
||
| // ListRoleAssignments returns role assignments for a given scope | ||
| func (c *RoleAssignmentsClient) ListRoleAssignments(ctx context.Context, scope string) ([]*armauthorization.RoleAssignment, error) { | ||
| pager := c.cli.NewListForScopePager(scope, nil) | ||
| var roleDefs []*armauthorization.RoleAssignment | ||
| for pager.More() { | ||
| page, err := pager.NextPage(ctx) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
| roleDefs = append(roleDefs, page.Value...) | ||
| } | ||
| return roleDefs, nil | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| /* | ||
| * Teleport | ||
| * Copyright (C) 2024 Gravitational, Inc. | ||
| * | ||
| * This program is free software: you can redistribute it and/or modify | ||
| * it under the terms of the GNU Affero General Public License as published by | ||
| * the Free Software Foundation, either version 3 of the License, or | ||
| * (at your option) any later version. | ||
| * | ||
| * This program is distributed in the hope that it will be useful, | ||
| * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| * GNU Affero General Public License for more details. | ||
| * | ||
| * You should have received a copy of the GNU Affero General Public License | ||
| * along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| */ | ||
|
|
||
| package azure | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/Azure/azure-sdk-for-go/sdk/azcore" | ||
| "github.com/Azure/azure-sdk-for-go/sdk/azcore/arm" | ||
| "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/authorization/armauthorization/v2" | ||
| "github.com/gravitational/trace" | ||
| ) | ||
|
|
||
| // RoleDefinitionsClient wraps the Azure API to provide a high level subset of functionality | ||
| type RoleDefinitionsClient struct { | ||
| cli *armauthorization.RoleDefinitionsClient | ||
| } | ||
|
|
||
| // NewRoleDefinitionsClient creates a new client for a given subscription and credentials | ||
| func NewRoleDefinitionsClient(subscription string, cred azcore.TokenCredential, options *arm.ClientOptions) (*RoleDefinitionsClient, error) { | ||
| clientFactory, err := armauthorization.NewClientFactory(subscription, cred, options) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
| roleDefCli := clientFactory.NewRoleDefinitionsClient() | ||
| return &RoleDefinitionsClient{cli: roleDefCli}, nil | ||
| } | ||
|
|
||
| // ListRoleDefinitions returns role definitions for a given scope | ||
| func (c *RoleDefinitionsClient) ListRoleDefinitions(ctx context.Context, scope string) ([]*armauthorization.RoleDefinition, error) { | ||
| pager := c.cli.NewListPager(scope, nil) | ||
| var roleDefs []*armauthorization.RoleDefinition | ||
| for pager.More() { | ||
| page, err := pager.NextPage(ctx) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
| roleDefs = append(roleDefs, page.Value...) | ||
| } | ||
| return roleDefs, nil | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| /* | ||
| * Teleport | ||
| * Copyright (C) 2025 Gravitational, Inc. | ||
| * | ||
| * This program is free software: you can redistribute it and/or modify | ||
| * it under the terms of the GNU Affero General Public License as published by | ||
| * the Free Software Foundation, either version 3 of the License, or | ||
| * (at your option) any later version. | ||
| * | ||
| * This program is distributed in the hope that it will be useful, | ||
| * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| * GNU Affero General Public License for more details. | ||
| * | ||
| * You should have received a copy of the GNU Affero General Public License | ||
| * along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| */ | ||
|
|
||
| package azuresync | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/gravitational/trace" | ||
| "golang.org/x/sync/errgroup" | ||
|
|
||
| accessgraphv1alpha "github.com/gravitational/teleport/gen/proto/go/accessgraph/v1alpha" | ||
| "github.com/gravitational/teleport/lib/msgraph" | ||
| ) | ||
|
|
||
| const parallelism = 10 //nolint:unused // invoked in a dependent PR | ||
|
|
||
| // expandMemberships adds membership data to AzurePrincipal objects by querying the Graph API for group memberships | ||
| func expandMemberships(ctx context.Context, cli *msgraph.Client, principals []*accessgraphv1alpha.AzurePrincipal) ([]*accessgraphv1alpha.AzurePrincipal, error) { //nolint:unused // invoked in a dependent PR | ||
| // Map principals by ID | ||
| var principalsMap = make(map[string]*accessgraphv1alpha.AzurePrincipal) | ||
| for _, principal := range principals { | ||
| principalsMap[principal.Id] = principal | ||
| } | ||
| // Iterate through the Azure groups and add the group ID as a membership for its corresponding principal | ||
| eg, _ := errgroup.WithContext(ctx) | ||
| eg.SetLimit(parallelism) | ||
| errCh := make(chan error, len(principals)) | ||
| for _, principal := range principals { | ||
| if principal.ObjectType != "group" { | ||
| continue | ||
| } | ||
| group := principal | ||
| eg.Go(func() error { | ||
| err := cli.IterateGroupMembers(ctx, group.Id, func(member msgraph.GroupMember) bool { | ||
| if memberPrincipal, ok := principalsMap[*member.GetID()]; ok { | ||
| memberPrincipal.MemberOf = append(memberPrincipal.MemberOf, group.Id) | ||
| } | ||
| return true | ||
| }) | ||
| if err != nil { | ||
| errCh <- err | ||
| } | ||
| return nil | ||
| }) | ||
| } | ||
| _ = eg.Wait() | ||
| close(errCh) | ||
| return principals, trace.NewAggregateFromChannel(errCh, ctx) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| /* | ||
| * Teleport | ||
| * Copyright (C) 2024 Gravitational, Inc. | ||
| * | ||
| * This program is free software: you can redistribute it and/or modify | ||
| * it under the terms of the GNU Affero General Public License as published by | ||
| * the Free Software Foundation, either version 3 of the License, or | ||
| * (at your option) any later version. | ||
| * | ||
| * This program is distributed in the hope that it will be useful, | ||
| * but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| * GNU Affero General Public License for more details. | ||
| * | ||
| * You should have received a copy of the GNU Affero General Public License | ||
| * along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| */ | ||
|
|
||
| package azuresync | ||
|
|
||
| import ( | ||
| "context" | ||
|
|
||
| "github.com/gravitational/trace" | ||
| "google.golang.org/protobuf/types/known/timestamppb" | ||
|
|
||
| accessgraphv1alpha "github.com/gravitational/teleport/gen/proto/go/accessgraph/v1alpha" | ||
| "github.com/gravitational/teleport/lib/msgraph" | ||
| ) | ||
|
|
||
| type dirObjMetadata struct { //nolint:unused // invoked in a dependent PR | ||
| objectType string | ||
| } | ||
|
|
||
| type queryResult struct { //nolint:unused // invoked in a dependent PR | ||
| metadata dirObjMetadata | ||
| dirObj msgraph.DirectoryObject | ||
| } | ||
|
|
||
| // fetchPrincipals fetches the Azure principals (users, groups, and service principals) using the Graph API | ||
| func fetchPrincipals(ctx context.Context, subscriptionID string, cli *msgraph.Client) ([]*accessgraphv1alpha.AzurePrincipal, error) { //nolint: unused // invoked in a dependent PR | ||
| // Fetch the users, groups, and service principals as directory objects | ||
| var queryResults []queryResult | ||
| err := cli.IterateUsers(ctx, func(user *msgraph.User) bool { | ||
| res := queryResult{metadata: dirObjMetadata{objectType: "user"}, dirObj: user.DirectoryObject} | ||
| queryResults = append(queryResults, res) | ||
| return true | ||
| }) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
| err = cli.IterateGroups(ctx, func(group *msgraph.Group) bool { | ||
| res := queryResult{metadata: dirObjMetadata{objectType: "group"}, dirObj: group.DirectoryObject} | ||
| queryResults = append(queryResults, res) | ||
| return true | ||
| }) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
| err = cli.IterateServicePrincipals(ctx, func(servicePrincipal *msgraph.ServicePrincipal) bool { | ||
| res := queryResult{metadata: dirObjMetadata{objectType: "servicePrincipal"}, dirObj: servicePrincipal.DirectoryObject} | ||
| queryResults = append(queryResults, res) | ||
| return true | ||
| }) | ||
| if err != nil { | ||
| return nil, trace.Wrap(err) | ||
| } | ||
|
|
||
| // Return the users, groups, and service principals as protobuf messages | ||
| var fetchErrs []error | ||
| var pbPrincipals []*accessgraphv1alpha.AzurePrincipal | ||
| for _, res := range queryResults { | ||
| if res.dirObj.ID == nil || res.dirObj.DisplayName == nil { | ||
| fetchErrs = append(fetchErrs, | ||
| trace.BadParameter("nil values on msgraph directory object: %v", res.dirObj)) | ||
| continue | ||
| } | ||
| pbPrincipals = append(pbPrincipals, &accessgraphv1alpha.AzurePrincipal{ | ||
| Id: *res.dirObj.ID, | ||
| SubscriptionId: subscriptionID, | ||
| LastSyncTime: timestamppb.Now(), | ||
| DisplayName: *res.dirObj.DisplayName, | ||
| ObjectType: res.metadata.objectType, | ||
| }) | ||
| } | ||
| return pbPrincipals, trace.NewAggregate(fetchErrs...) | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.