[SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

[zhongyu09]

What changes were proposed in this pull request?

In AdaptiveSparkPlanExec.getFinalPhysicalPlan, when newStages are generated, sort the new stages by class type to make sure BroadcastQueryState precede others.
It can make sure the broadcast job are submitted before map jobs to avoid waiting for schedule and cause broadcast timeout.

Why are the changes needed?

When enable AQE, in getFinalPhysicalPlan, spark traversal the physical plan bottom up and create query stage for materialized part by createQueryStages and materialize those new created query stages to submit map stages or broadcasting. When ShuffleQueryStage are materializing before BroadcastQueryStage, the map job and broadcast job are submitted almost at the same time, but map job will hold all the computing resources. If the map job runs slow (when lots of data needs to process and the resource is limited), the broadcast job cannot be started(and finished) before spark.sql.broadcastTimeout, thus cause whole job failed (introduced in SPARK-31475).
The workaround to increase spark.sql.broadcastTimeout doesn't make sense and graceful, because the data to broadcast is very small.

Does this PR introduce any user-facing change?

NO

How was this patch tested?

1. Add UT
2. Test the code using dev environment in https://issues.apache.org/jira/browse/SPARK-33933

[zhongyu09]

Hi, @cloud-fan @maryannxue @Ngone51 @JkSelf, could you please take a look?

[dongjoon-hyun]

Could you rebase this PR once more to trigger GitHub Action, @zhongyu09 ?

[dongjoon-hyun]

ok to test

[dongjoon-hyun]

@zhongyu09 . Please open this PR to the master branch first.

To prevent a future regression, we need to fix the branch in an order: master -> branch-3.1 -> branch-3.0.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38184/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38184/

[SparkQA]

Test build #133595 has finished for PR 30962 at commit e9ae1e8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhongyu09]

Closing this PR and create another(#30998) based on master, sorry for inconvenience.

[dongjoon-hyun]

Thank you, @zhongyu09 .