HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call

[aswinshakil]

What changes were proposed in this pull request?

Currently the client will get an IO_EXCEPTION result code when requesting the merkle tree of a peer that has not been generated yet. If the container is not open, have the peer build the merkle tree from metadata and return this instead. This change builds merkle tree if it's not available.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12990

How was this patch tested?

Existing tests.

[errose28]

Just a minor typo in the log message otherwise LGTM.

[ivandika3]

@errose28 @aswinshakil This seems to break master. Please kindly help to rectify it.

[errose28]

Good catch @ivandika3 and @adoroszlai. Looks like the order of events was:

• CI green on this PR with master
• HDDS-12824. Optimize container checksum read during datanode startup #8604 is merged, which updates master in such a way that compilation for this PR fails but doesn't generate a merge conflict
• This PR is merged based on the previous green CI
• The first CI run of this PR with the latest master changes happens on master itself and fails

I'm not sure if there's a way to prevent these sort of issues entirely, since there's always going to be a gap between when a PR's CI completes and when it is merged, during which any other change could show up in master. I guess if reviewers are aware of related changes in master they can re-trigger the CI before merge.

[ivandika3]

Thanks @adoroszlai for helping to resolve the issue promptly and @errose28 for tracing the issue.

I'm not sure if there's a way to prevent these sort of issues entirely, since there's always going to be a gap between when a PR's CI completes and when it is merged, during which any other change could show up in master.

Yes, IMO it's an acceptable tradeoff between occasional compile issue and needing to merge to latest master (which will slow down merge). I think it's OK if we can rectify the issue quickly (e.g. checking master branch frequently + checking for notification for CI failures on master when the PR is merged).

@adoroszlai By the way, can I check whether any committer / PMC has the privilege to push directly to master to fix compile issue or is there any SOP that we need to follow?

[adoroszlai]

whether any committer / PMC has the privilege to push directly to master

Yes.

a way to prevent these sort of issues entirely

Not completely, but here's a basic sanity check to be run locally before merge:

git fetch origin
gh pr checkout 8881
git merge --no-commit origin/master \
  && mvn -DskipRecon -DskipShade -DskipTests clean verify \
  && ./hadoop-ozone/dev-support/checks/checkstyle.sh \
  && ./hadoop-ozone/dev-support/checks/pmd.sh

(It uses GitHub CLI, but can also be done using few more lower level commands.)