[improve] [broker] Fix broker restart logic

[wolfstudy]

Motivation

Currently, when we execute the logic of Broker Close, we first shut down the Broker Service, and then close the LoadManagerTask. This will cause the Broker to be assigned to a new bundle during the shutdown of the Broker, resulting in an inelegant shutdown of the Broker. , eventually triggering brokerShutdownTimeoutMs to forcibly shut down the Broker, causing the Client to fail to produce or consume messages over time.

When shutting down the Broker, we will first remove the temporary node of the current Broker from zookeeper, which is the logic we expect. We expect that the broker will not allocate new bundles during active shutdown of the broker. However, since the task of loadManagerExecutor has been regularly executed to report its own Load information to zookeeper, this will cause the Broker node to be shut down to be re-registered by loadManagerExecutor. At this time, the Leader Broker believes that the Broker node to be shut down can also allocate a new bundle, and will redistribute the new bundle information.

Interestingly, in the code logic of Pulsar Service shutdown, the comments clearly indicate that we expect LoadMangerTask to be closed before Broker Service, but in real code implementation, it is the opposite.

 // cancel loadShedding task and shutdown the loadManager executor before shutting down the broker

Modifications

Adjust the order of component shutdown when Broker is shut down so that LoadManagerTask is shut down before Broker Service

• doc
• doc-required
• doc-not-needed
• doc-complete

[github-actions]

@wolfstudy Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

[codelipenghui]

/pulsarbot run-failure-checks

[HQebupt]

Good catch！

[wolfstudy]

/pulsarbot run-failure-checks

[codecov-commenter]

Codecov Report

Merging #20113 (5a4dff4) into master (c4aec66) will increase coverage by 39.78%.
The diff coverage is 100.00%.

@@              Coverage Diff              @@
##             master   #20113       +/-   ##
=============================================
+ Coverage     33.16%   72.94%   +39.78%     
- Complexity    12225    31911    +19686     
=============================================
  Files          1499     1868      +369     
  Lines        114340   138318    +23978     
  Branches      12415    15214     +2799     
=============================================
+ Hits          37918   100897    +62979     
+ Misses        71484    29398    -42086     
- Partials       4938     8023     +3085     

Flag	Coverage Δ
inttests	24.18% <0.00%> (?)
systests	24.87% <0.00%> (?)
unittests	72.23% <100.00%> (+39.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...n/java/org/apache/pulsar/broker/PulsarService.java	81.63% <100.00%> (+25.30%)	⬆️
...e/pulsar/io/elasticsearch/ElasticSearchConfig.java	90.67% <100.00%> (ø)

... and 1540 files with indirect coverage changes