HDDS-12983. Validator Registry Changes for Supporting Version based validations #8404

swamirishi · 2025-05-06T20:44:18Z

What changes were proposed in this pull request?

Making changes to the ValidatorRegistry class to be able to pull the validators registered and inturn being able to get all the underlying validator methods based on the usage of the registered feature validator based on a specified version, request processing phase and request type efficiently.
This is the last PR for completing the request validator fixes after breaking down
#6932 into smaller PRs

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12983

How was this patch tested?

Adding unit tests

…alidations Change-Id: I939888c989eaec0a62884aa564e2f24a594122ef

Change-Id: Icec10ce7bb452753db0936212b0757c01279b533

Change-Id: I9a010c86d0128da8140fa33b46eaeef9da0dc283

adoroszlai

Thanks @swamirishi for the patch.

adoroszlai · 2025-05-14T09:07:42Z

...e-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java

+  public List<Method> validationsFor(RequestType requestType,
+                                     RequestProcessingPhase phase,
+                                     Map<Class<? extends Annotation>, ? extends Versioned> requestVersions) {


Please do not format method signature like this. Whenever visibility / return type / method name / other modifiers are changed, we would have to reindent all parameters.

adoroszlai · 2025-05-14T09:09:08Z

...e-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java

+      Validator validator, String methodName, Class<ReturnValue> returnValueType) {
+    try {
+      return (ReturnValue) validator.getClass().getMethod(methodName).invoke(validator);


Use returnValueType (currently unused) to cast the result, instead of using unchecked cast (ReturnValue).

adoroszlai · 2025-05-14T09:12:13Z

...e-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java

      processingPhase = RequestProcessingPhase.PRE_PROCESS,
-      requestType = Type.CreateBucket
+      requestType = Type.CreateBucket,
+      applyBefore =  ClientVersion.BUCKET_LAYOUT_SUPPORT


nit:

Suggested change

applyBefore = ClientVersion.BUCKET_LAYOUT_SUPPORT

applyBefore = ClientVersion.BUCKET_LAYOUT_SUPPORT

adoroszlai · 2025-05-14T09:51:55Z

.../ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/package-info.java

+ * One validator can be applied in multiple, E.g.
+ * {@link org.apache.hadoop.ozone.om.request.validation.OMClientVersionValidator},
+ * {@link org.apache.hadoop.ozone.om.request.validation.OMLayoutVersionValidator}


I don't think this sentence makes sense.

updated the sentence lmk if it is ok now

adoroszlai · 2025-05-14T09:56:44Z

...nager/src/test/java/org/apache/hadoop/ozone/om/request/validation/TestValidatorRegistry.java

+    ValidatorRegistry<OzoneManagerProtocolProtos.Type> registry =
+        new ValidatorRegistry<>(OzoneManagerProtocolProtos.Type.class, PACKAGE,
+        Arrays.stream(VersionExtractor.values()).map(VersionExtractor::getValidatorClass).collect(Collectors.toSet()),
+        REQUEST_PROCESSING_PHASES);


Can we avoid duplicating this logic in each test case?

adoroszlai · 2025-05-14T10:01:10Z

...e-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java

+     * @return All the items which has an index value greater than given index value.
+     */
+    public List<T> getItemsEqualToIdx(IDX indexValue) {


Javadoc mismatch ("greater than" vs. EqualTo)

adoroszlai · 2025-05-14T10:02:10Z

...e-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java

+   */
+  private static final class IndexedItems<T, IDX extends Comparable<IDX>> {
+    private final List<T> items;
+    private final TreeMap<IDX, Integer> indexMap;


Declare as interface: NavigableMap.

adoroszlai · 2025-05-14T10:02:56Z

...e-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/ValidatorRegistry.java

+        .sorted((validatorMethodPair1, validatorMethodPair2) ->
+            Integer.compare(
+                this.getApplyBeforeVersion(validatorMethodPair1.getKey()).version(),
+                this.getApplyBeforeVersion(validatorMethodPair2.getKey()).version()))


Extract this lambda to a static method.

adoroszlai · 2025-05-14T10:06:32Z

...-manager/src/main/java/org/apache/hadoop/ozone/om/request/validation/RequestValidations.java

+    List<Method> validations = registry.validationsFor(request.getCmdType(), PRE_PROCESS,
+        this.getVersions(request));


nit: remove this. and do not wrap line.

Change-Id: Ibf524d42e3d8535377105a07f8ca3213c8c62199

Change-Id: I67634e5517239bdec3965ed8819f9af6ecf577dd

Change-Id: Idf26bbc7e83877a71f46bafdf0216d70733a92ee

Change-Id: Idecefb392dbf5c01227d7c3702db1781b9edb073

Change-Id: I174cea546d6fd021e58966b3a5bca3672e9d567e # Conflicts: # hadoop-ozone/client/pom.xml # hadoop-ozone/interface-storage/pom.xml # hadoop-ozone/recon-codegen/pom.xml # hadoop-ozone/s3gateway/pom.xml

Change-Id: I132006b2b5a638e7f19d9ec1beb33b332d105e98

github-actions · 2025-11-12T00:05:28Z

This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.

github-actions · 2025-11-19T00:05:44Z

Thank you for your contribution. This PR is being closed due to inactivity. If needed, feel free to reopen it.

Change-Id: Ie1076419cae849563efc8a14ac97e5c0c83828a9 # Conflicts: # hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequest.java # hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java # hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java # hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java # hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRenameRequest.java

Change-Id: I461d46461905d041a613b101182d24e18a143d5c

adoroszlai · 2025-12-19T13:19:41Z

...-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMDirectoryCreateRequest.java

+  @OMLayoutVersionValidator(
      processingPhase = RequestProcessingPhase.PRE_PROCESS,
-      requestType = Type.CreateDirectory
+      requestType = Type.CreateDirectory,
+      applyBefore = OMLayoutFeature.ERASURE_CODED_STORAGE_SUPPORT


Please simply replace the two changed lines instead of moving things around.

- @RequestFeatureValidator( - conditions = ValidationCondition.CLUSTER_NEEDS_FINALIZATION, + @OMLayoutVersionValidator( + applyBefore = OMLayoutFeature.ERASURE_CODED_STORAGE_SUPPORT,

adoroszlai · 2025-12-19T13:51:16Z

hadoop-hdds/client/pom.xml

-                    <bannedImport>org.apache.hadoop.ozone.om.request.validation.RequestFeatureValidator</bannedImport>
+                    <bannedImport>org.apache.hadoop.ozone.om.request.validation.OMLayoutVersionValidator</bannedImport>
+                    <bannedImport>org.apache.hadoop.ozone.om.request.validation.OMClientVersionValidator</bannedImport>


Created #9530 to reduce the change that this PR needs to make in POM files. The other PR:

adds the new annotations, since they were already introduced in HDDS-11981

more importantly, removes RequestFeatureValidator from modules where it's not even accessible

So this PR will only need to update POMs for the removal of RequestFeatureValidator, and only in fewer modules.

merged with master branch

Change-Id: Ie6717a5c5e5e6821c2ec8d7abc7a5d4a20d6767f # Conflicts: # hadoop-hdds/client/pom.xml # hadoop-hdds/common/pom.xml # hadoop-hdds/container-service/pom.xml # hadoop-hdds/framework/pom.xml # hadoop-hdds/server-scm/pom.xml # hadoop-ozone/cli-admin/pom.xml # hadoop-ozone/cli-shell/pom.xml # hadoop-ozone/common/pom.xml # hadoop-ozone/csi/pom.xml # hadoop-ozone/freon/pom.xml # hadoop-ozone/insight/pom.xml # hadoop-ozone/recon/pom.xml # hadoop-ozone/tools/pom.xml # pom.xml

Change-Id: Ia2f6792669628c67088b8b5a273fb271476e8848

swamirishi · 2025-12-29T12:50:44Z

@adoroszlai I have addressed all your review comments. Can you take a look at the change now.

HDDS-12983. Validator Registry Changes for Supporting Version based v…

2fb399c

…alidations Change-Id: I939888c989eaec0a62884aa564e2f24a594122ef

swamirishi requested review from adoroszlai and fapifta May 6, 2025 20:44

swamirishi added 2 commits May 6, 2025 16:51

HDDS-12983. Revert module move change

16dd2d2

Change-Id: Icec10ce7bb452753db0936212b0757c01279b533

HDDS-12983. Fix Validator registry

de26a00

Change-Id: I9a010c86d0128da8140fa33b46eaeef9da0dc283

adoroszlai reviewed May 14, 2025

View reviewed changes

swamirishi added 3 commits May 20, 2025 08:06

HDDS-12983. Change key to validator map

bf98388

Change-Id: Ibf524d42e3d8535377105a07f8ca3213c8c62199

Merge remote-tracking branch 'apache/master' into HEAD

36cd0d5

Change-Id: I67634e5517239bdec3965ed8819f9af6ecf577dd

HDDS-12983. Address review comments

d4b9b33

Change-Id: Idf26bbc7e83877a71f46bafdf0216d70733a92ee

swamirishi requested a review from sumitagrawl May 20, 2025 15:23

swamirishi added 3 commits May 20, 2025 11:41

HDDS-12983. Fix checkstyle issue

64e0f58

Change-Id: Idecefb392dbf5c01227d7c3702db1781b9edb073

Merge remote-tracking branch 'apache/master' into HEAD

4e18baa

Change-Id: I174cea546d6fd021e58966b3a5bca3672e9d567e # Conflicts: # hadoop-ozone/client/pom.xml # hadoop-ozone/interface-storage/pom.xml # hadoop-ozone/recon-codegen/pom.xml # hadoop-ozone/s3gateway/pom.xml

Resolve merge conflicts

3ffac6c

Change-Id: I132006b2b5a638e7f19d9ec1beb33b332d105e98

github-actions bot added the stale label Nov 12, 2025

github-actions bot closed this Nov 19, 2025

swamirishi reopened this Dec 17, 2025

HDDS-12983. Fix checkstyle

3743515

Change-Id: I461d46461905d041a613b101182d24e18a143d5c

github-actions bot removed the stale label Dec 18, 2025

adoroszlai reviewed Dec 19, 2025

View reviewed changes

swamirishi added 2 commits December 29, 2025 07:38

HDDS-12983. Address review comments

a3a696c

Change-Id: Ia2f6792669628c67088b8b5a273fb271476e8848

swamirishi requested a review from adoroszlai December 29, 2025 14:15

	applyBefore = ClientVersion.BUCKET_LAYOUT_SUPPORT
	applyBefore = ClientVersion.BUCKET_LAYOUT_SUPPORT

		List<Method> validations = registry.validationsFor(request.getCmdType(), PRE_PROCESS,
		this.getVersions(request));

HDDS-12983. Validator Registry Changes for Supporting Version based validations #8404

Are you sure you want to change the base?

HDDS-12983. Validator Registry Changes for Supporting Version based validations #8404

Uh oh!

Conversation

swamirishi commented May 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 12, 2025

Uh oh!

github-actions bot commented Nov 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

swamirishi commented Dec 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

swamirishi commented May 6, 2025 •

edited

Loading