Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improved validating groups membership cli command #816

Merged
merged 18 commits into from
Jan 30, 2024
Merged

Improved validating groups membership cli command #816

merged 18 commits into from
Jan 30, 2024

Conversation

mwojtyczka
Copy link
Contributor

@mwojtyczka mwojtyczka commented Jan 19, 2024 •
edited
Loading

Changes

  • Corrected the fields returned by the cli in the spec. Currently the cli does not display the group names correctly.
  • Added group members count to the output.
  • Fix the issue that account group members count is always 0.

Functionality

  • return members count in the output generated by existing cli command: databricks labs ucx validate-groups-membership

Tests

  • manually tested
  • updated unit tests
  • run existing integration tests

@mwojtyczka
added new fields
3912590 
corrected output
@mwojtyczka mwojtyczka requested review from a team and larsgeorge-db January 19, 2024 11:47
@mwojtyczka mwojtyczka changed the title Improve validate groups membership cli command Improved validating groups membership cli command Jan 19, 2024
@codecov Codecov
Copy link

codecov bot commented Jan 19, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (606dd72) 85.67% compared to head (cdc43f9) 85.75%.
Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #816      +/-   ##
==========================================
+ Coverage   85.67%   85.75%   +0.08%     
==========================================
  Files          42       42              
  Lines        5311     5330      +19     
  Branches      969      971       +2     
==========================================
+ Hits         4550     4571      +21     
+ Misses        542      541       -1     
+ Partials      219      218       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mwojtyczka mwojtyczka added the pr/do-not-merge this pull request is not ready to merge label Jan 19, 2024
@mwojtyczka mwojtyczka added pr/do-not-merge this pull request is not ready to merge and removed pr/do-not-merge this pull request is not ready to merge labels Jan 19, 2024
william-conti
william-conti suggested changes Jan 19, 2024
View reviewed changes
Copy link
Contributor

@william-conti william-conti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, only minor change that is debatable

src/databricks/labs/ucx/workspace_access/groups.py Outdated Show resolved Hide resolved
@mwojtyczka mwojtyczka removed the pr/do-not-merge this pull request is not ready to merge label Jan 19, 2024
@mwojtyczka mwojtyczka added the pr/do-not-merge this pull request is not ready to merge label Jan 19, 2024
@mwojtyczka mwojtyczka removed the pr/do-not-merge this pull request is not ready to merge label Jan 29, 2024
nfx
nfx approved these changes Jan 30, 2024
View reviewed changes
src/databricks/labs/ucx/workspace_access/groups.py
retry_on_internal_error = retried(on=[InternalError], timeout=self._verify_timeout)
get_account_group = retry_on_internal_error(self._get_account_group)
for ws_group in migrated_groups:
# Users with the same display name but different email will be deduplicated!
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that a bug?🐜

Copy link
Contributor Author

@mwojtyczka mwojtyczka Jan 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a limitation of the scim groups API which only return display name. Email address is not present for users.
We could fetch user info separately and compare by email address but a user can theoretically have multiple email addresses so it is also problematic.

nfx
nfx approved these changes Jan 30, 2024
View reviewed changes
Copy link
Collaborator

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@nfx nfx merged commit 07f03f7 into main Jan 30, 2024
7 checks passed
@nfx nfx deleted the fix/group_membership_validation branch January 30, 2024 09:08
nfx added a commit that referenced this pull request Feb 1, 2024
@nfx
Release v0.11.1
202f52a 
* Added "what" property for migration to scope down table migrations ([#856](#856)).
* Added job count in the assessment dashboard ([#858](#858)).
* Adopted `installation` package from `databricks-labs-blueprint` ([#860](#860)).
* Debug logs to print only the first 96 bytes of SQL query by default, tunable by `debug_truncate_bytes` SDK configuration property ([#859](#859)).
* Extract command codes and unify the checks for spark_conf, cluster_policy, init_scripts ([#855](#855)).
* Improved installation failure with actionable message ([#840](#840)).
* Improved validating groups membership cli command ([#816](#816)).

Dependency updates:

 * Updated databricks-labs-blueprint requirement from ~=0.1.0 to ~=0.2.4 ([#867](#867)).
@nfx nfx mentioned this pull request Feb 1, 2024
Merged
nfx added a commit that referenced this pull request Feb 1, 2024
@nfx
Release v0.11.1 (#870)
e11494c 
* Added "what" property for migration to scope down table migrations
([#856](#856)).
* Added job count in the assessment dashboard
([#858](#858)).
* Adopted `installation` package from `databricks-labs-blueprint`
([#860](#860)).
* Debug logs to print only the first 96 bytes of SQL query by default,
tunable by `debug_truncate_bytes` SDK configuration property
([#859](#859)).
* Extract command codes and unify the checks for spark_conf,
cluster_policy, init_scripts
([#855](#855)).
* Improved installation failure with actionable message
([#840](#840)).
* Improved validating groups membership cli command
([#816](#816)).

Dependency updates:

* Updated databricks-labs-blueprint requirement from ~=0.1.0 to ~=0.2.4
([#867](#867)).
dmoore247 pushed a commit that referenced this pull request Mar 23, 2024
@mwojtyczka @dmoore247
Improved validating groups membership cli command (#816)
ed3365e
dmoore247 pushed a commit that referenced this pull request Mar 23, 2024
@nfx @dmoore247
Release v0.11.1 (#870)
0fb6c98 
* Added "what" property for migration to scope down table migrations
([#856](#856)).
* Added job count in the assessment dashboard
([#858](#858)).
* Adopted `installation` package from `databricks-labs-blueprint`
([#860](#860)).
* Debug logs to print only the first 96 bytes of SQL query by default,
tunable by `debug_truncate_bytes` SDK configuration property
([#859](#859)).
* Extract command codes and unify the checks for spark_conf,
cluster_policy, init_scripts
([#855](#855)).
* Improved installation failure with actionable message
([#840](#840)).
* Improved validating groups membership cli command
([#816](#816)).

Dependency updates:

* Updated databricks-labs-blueprint requirement from ~=0.1.0 to ~=0.2.4
([#867](#867)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nfx nfx nfx approved these changes

@larsgeorge-db larsgeorge-db Awaiting requested review from larsgeorge-db

@william-conti william-conti Awaiting requested review from william-conti

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mwojtyczka @nfx @william-conti