HDDS-13423. Logging reason behind triggering onDemand scan on containers #8854
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
errose28
added
the
scanners
Tejaskriya
reviewed
Jul 29, 2025
Contributor
Tejaskriya
left a comment
Tejaskriya
left a comment
There was a problem hiding this comment.
Thanks for working on this @sreejasahithi, please find some suggestions below.
...tainer-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java
Outdated
Show resolved
Hide resolved
...vice/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OnDemandContainerScanner.java
Outdated
Show resolved
Hide resolved
...er-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestContainerSet.java
Outdated
Show resolved
Hide resolved
Tejaskriya
approved these changes
Jul 30, 2025
Contributor
Tejaskriya
left a comment
Tejaskriya
left a comment
There was a problem hiding this comment.
Thanks for updating this, LGTM overall, please resolve the merge conflicts
aryangupta1998
approved these changes
Jul 30, 2025
Contributor
aryangupta1998
left a comment
aryangupta1998
left a comment
There was a problem hiding this comment.
Thanks for the patch @sreejasahithi, LGTM!
adoroszlai
reviewed
Jul 31, 2025
Contributor
adoroszlai
left a comment
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @sreejasahithi for the patch.
I think we should:
- also include reason in "skip" messages
- make "skip" and "schedule" messages more consistent (
on-demand scan for containervs.OnDemandScan for Container)
sreejasahithi
commented
Jul 31, 2025
Comment on lines
82
to
85
| if (!helper.shouldScanMetadata(container)) { | ||
| if (LOG.isDebugEnabled()) { | ||
| LOG.debug("Skipping on-demand scan for container {}.", container.getContainerData().getContainerID()); | ||
| } |
Contributor
Author
There was a problem hiding this comment.
Removed the log message here because the detailed reason is already logged in the respective methods
adoroszlai
reviewed
Aug 1, 2025
| }); | ||
| } else { | ||
| if (LOG.isDebugEnabled()) { | ||
| LOG.debug("Skipping OnDemandScan for Container {}, Reason: {}", containerId, "Already scheduled."); |
Contributor
There was a problem hiding this comment.
Shouldn't we log the original reason?
Contributor
|
Thanks @sreejasahithi for the patch, @aryangupta1998, @Tejaskriya for the review. |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Aug 12, 2025
* master: (55 commits) HDDS-13525. Rename configuration property to ozone.om.compaction.service.enabled (apache#8928) HDDS-13519. Reconciliation should continue if a peer datanode is unreachable (apache#8908) HDDS-13566. Fix incorrect authorizer class in ACL documentation (apache#8931) HDDS-13084. Trigger on-demand container scan when a container moves from open to unhealthy. (apache#8904) HDDS-13432. Accelerating Namespace Usage Calculation in Recon using - Materialised Approach (apache#8797) HDDS-13557. Bump jline to 3.30.5 (apache#8920) HDDS-13556. Bump assertj-core to 3.27.4 (apache#8919) HDDS-13543. [Docs] Design doc for OM bootstrapping process with snapshots. (apache#8900) HDDS-13541. Bump sonar-maven-plugin to 5.1.0.4751 (apache#8911) HDDS-13101. Remove duplicate information in datanode list output (apache#8523) HDDS-13528. Handle null paths when the NSSummary is initializing (apache#8901) HDDS-12990. (addendum) Generate tree from metadata when it does not exist during getContainerChecksumInfo call (apache#8881) HDDS-13086. Block duplicate reconciliation requests for the same container and datanode within the datanode. (apache#8905) HDDS-12990. Generate tree from metadata when it doesn't exist during getContainerChecksumInfo call (apache#8881) HDDS-12824. Optimize container checksum read during datanode startup (apache#8604) HDDS-13522. Rename axisLabel for No. of delete request received (apache#8879) HDDS-12196. Document ozone repair cli (apache#8849) HDDS-13514. Intermittent failure in TestNSSummaryMemoryLeak (apache#8889) HDDS-13423. Log reason for triggering on-demand container scan (apache#8854) HDDS-13466. Disable flaky TestOmSnapshotFsoWithNativeLibWithLinkedBuckets ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Currently there are no logs present which tell why an on-demand container scan was triggered.
This PR adds the reason for on-demand container scan at the respective locations.
What is the link to the Apache JIRA
HDDS-13423
How was this patch tested?
https://github.com/sreejasahithi/ozone/actions/runs/16463299349