Automatically switch to upload_graph when the graph is large

[liferoad]

Now the job graph is larger than 10 MB, it will be automatically uploaded by enabling upload_graph for Java SDK.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[liferoad]

I plan to manually test this later (might add one integration test). So no need to merge this now. Thanks.

[liferoad]

Tested these changes internally. The test passed.

[liferoad]

R: @Abacn

[github-actions]

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

[kennknowles]

Let us get a test externally. It should be very easy - just a single small test method with a big graph.

[Abacn]

For the integration test, /runners/google-cloud-dataflow-java/src/test could be a choice to settle these tests. Although currently there is only unit tests, this path is added to the integration test class path:

beam/runners/google-cloud-dataflow-java/build.gradle

Line 138 in f3df03d

googleCloudPlatformIntegrationTest project(project.path)

[liferoad]

Let us get a test externally. It should be very easy - just a single small test method with a big graph.

Added one unit test.

[kennknowles]

LGTM except it isn't really a breaking change.

[Abacn]

Thanks! the code changes LGTM. Had a comment about CHANGES.md item

[kennknowles]

Once it is green again (for the record) will merge. Or any committer can merge.

[liferoad]

Run Java PreCommit

[liferoad]

Run Java PreCommit

[liferoad]

Note "org.apache.beam.runners.dataflow.worker.StreamingDataflowWorkerTest.testActiveThreadMetric[*]" seems flaky. I tried several times to get all greens.

[Abacn]

Note "org.apache.beam.runners.dataflow.worker.StreamingDataflowWorkerTest.testActiveThreadMetric[*]" seems flaky. I tried several times to get all greens.

Yes this is unrelated. Tracking bug: #28957 caused by #28513 there is a pending PR include a fix: #29041 (comment)

[kennknowles]

I think it is OK to disable the test or even better to roll back the change so we do not have untested stuff in the codebase.