Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSIP-54][Master] Use ClusterManager to manage the cluster in master #16263

Merged

Conversation

ruanwenjun
Copy link
Member

Purpose of the pull request

close #16262

Brief change log

  • Create ClusterManager/MasterClusters/WorkerClusters to manage the cluster metadata
  • Create WorkerLoadBalancer to deal with worker load balancer.
  • Support to config the weight of cpu/memory/threadpool usage in DynamicWeightedRoundRobinWorkerLoadBalancer

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 33.25123% with 271 lines in your changes missing coverage. Please review.

Project coverage is 41.07%. Comparing base (565a917) to head (c7dd75d).
Report is 5 commits behind head on dev.

Current head c7dd75d differs from pull request most recent head 7bef33a

Please upload reports for the commit 7bef33a to get more accurate results.

Files Patch % Lines
...duler/server/master/cluster/MasterSlotManager.java 0.00% 32 Missing ⚠️
...cheduler/server/master/cluster/MasterClusters.java 0.00% 26 Missing ⚠️
...r/runner/GlobalTaskDispatchWaitingQueueLooper.java 0.00% 26 Missing ⚠️
...ster/cluster/AbstractClusterSubscribeListener.java 8.69% 21 Missing ⚠️
...cheduler/server/master/cluster/WorkerClusters.java 61.81% 21 Missing ⚠️
...rver/master/cluster/WorkerGroupChangeNotifier.java 0.00% 21 Missing ⚠️
...e/dolphinscheduler/common/utils/MapComparator.java 0.00% 17 Missing ⚠️
...er/server/master/cluster/ClusterStateMonitors.java 0.00% 14 Missing ⚠️
.../loadbalancer/WorkerLoadBalancerConfiguration.java 0.00% 12 Missing ⚠️
...r/master/runner/dispatcher/BaseTaskDispatcher.java 14.28% 12 Missing ⚠️
... and 16 more
Additional details and impacted files
@@             Coverage Diff              @@
##                dev   #16263      +/-   ##
============================================
- Coverage     41.17%   41.07%   -0.11%     
+ Complexity     5093     5083      -10     
============================================
  Files          1391     1396       +5     
  Lines         43803    43674     -129     
  Branches       4645     4621      -24     
============================================
- Hits          18038    17938     -100     
+ Misses        23997    23976      -21     
+ Partials       1768     1760       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@SbloodyS SbloodyS added this to the 3.3.0 milestone Jul 3, 2024
@SbloodyS SbloodyS changed the title [DSIP-49][Master] Use ClusterManager to manage the cluster in master [DSIP-54][Master] Use ClusterManager to manage the cluster in master Jul 3, 2024
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorServerNodeManager branch from 7bef33a to 61210f3 Compare July 3, 2024 02:17

@Data
@SuperBuilder
@EqualsAndHashCode(callSuper = true)

Check notice

Code scanning / CodeQL

Missing Override annotation Note

This method overrides
BaseServerMetadata.canEqual
; it is advisable to add an Override annotation.

@Data
@SuperBuilder
@EqualsAndHashCode(callSuper = true)

Check notice

Code scanning / CodeQL

Missing Override annotation Note

This method overrides
BaseServerMetadata.canEqual
; it is advisable to add an Override annotation.
@ruanwenjun ruanwenjun added the DSIP label Jul 3, 2024
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorServerNodeManager branch 5 times, most recently from 9fa1ea9 to 55750ba Compare July 3, 2024 12:45
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorServerNodeManager branch from 55750ba to a8ed188 Compare July 3, 2024 13:24
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_refactorServerNodeManager branch from a8ed188 to f26012b Compare July 4, 2024 02:32
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍🏻

Copy link

sonarcloud bot commented Jul 5, 2024

Please retry analysis of this Pull-Request directly on SonarCloud

@ruanwenjun ruanwenjun merged commit 8f6df8d into apache:dev Jul 5, 2024
63 of 64 checks passed
@ruanwenjun ruanwenjun deleted the dev_wenjun_refactorServerNodeManager branch July 5, 2024 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DSIP-54][Master] Use ClusterManager to manage the cluster in master
4 participants