Skip to content

Conversation

@ArafatKhan2198
Copy link
Contributor

What changes were proposed in this pull request?

This issue is related to the use of GSON in the codebase and its compatibility with Java 17. The proposed solution is to replace GSON with Jackson to reduce dependencies.

Affected Modules:

The replacement of GSON with Jackson is considered for several modules in the project, including:

  • KeyValueContainerMetadataInspector and TestKeyValueContainerMetadataInspector in the hadoop-hdds/container-service module.
  • HddsConfServlet and EventQueue in the hadoop-hdds/framework module, along with its corresponding tests.
  • Several classes in hadoop-ozone/common, especially related to multitenancy and access policies AccessPolicy
    & RangerAccessPolicy.
  • Multiple integration tests and tools within the hadoop-ozone module, such as TestReconWithOzoneManager, TestOzoneShellHA, and various classes in the tools sub-module related to Ozone Shell, tenant management, and debug tools.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-10122?filter=-1

How was this patch tested?

Currently Draft State will be adding more classes into it.

@adoroszlai
Copy link
Contributor

@ArafatKhan2198 please check test failures

@ArafatKhan2198 ArafatKhan2198 marked this pull request as ready for review February 13, 2024 08:41
@ArafatKhan2198 ArafatKhan2198 marked this pull request as draft February 13, 2024 08:42
@ArafatKhan2198 ArafatKhan2198 marked this pull request as ready for review February 19, 2024 10:52
@ArafatKhan2198 ArafatKhan2198 marked this pull request as draft February 19, 2024 10:53
@ArafatKhan2198 ArafatKhan2198 marked this pull request as ready for review February 20, 2024 08:36
@adoroszlai
Copy link
Contributor

Thanks @ArafatKhan2198 for continuing work on this. There are still some test failures.

@ArafatKhan2198 ArafatKhan2198 marked this pull request as draft February 20, 2024 08:53
@ArafatKhan2198 ArafatKhan2198 marked this pull request as ready for review March 17, 2024 08:30
@ArafatKhan2198
Copy link
Contributor Author

@ArafatKhan2198
Copy link
Contributor Author

@adoroszlai Could you please take a look

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ArafatKhan2198 for the patch.

To make the change more manageable, I suggest splitting it to separate patches:

  1. ozone tenant
  2. ozone admin namespace
  3. ozone debug
  4. rest of the change, including EventQueue (still has GSON), and removal of gson as a dependency

Created HDDS-10538 with sub-tasks for this purpose.

(Please see few inline comments, too.)

String subPath = subPathDU.get("path").toString();
// differentiate key from other types
if (!(boolean)subPathDU.get("isKey")) {
LinkedTreeMap<?, ?> subPathDU = (LinkedTreeMap<?, ?>) o;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LinkedTreeMap is part of GSON. I guess if we reach this code, ClassCastException will be thrown. Probably needs to be Map, but please confirm by testing it manually.

Comment on lines +388 to +389
ArrayList<LinkedTreeMap> taskStatusList = objectMapper.readValue(
taskStatusResponse, new TypeReference<ArrayList<LinkedTreeMap>>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we replace LinkedTreeMap with Map here, too?

Comment on lines +190 to +195
ObjectMapper mapper = new ObjectMapper();
// For pretty-printing
mapper.enable(SerializationFeature.INDENT_OUTPUT);
// To serialize null values
mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
String jsonReport = mapper.writeValueAsString(containerJson);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try to avoid creating a new ObjectMapper for each serialization (also in ~17 other files). Use JsonUtils:

public static String toJsonStringWithDefaultPrettyPrinter(Object obj)
throws IOException {
return WRITER.writeValueAsString(obj);
}
public static String toJsonString(Object obj) throws IOException {
return MAPPER.writeValueAsString(obj);
}
public static ArrayNode createArrayNode() {
return MAPPER.createArrayNode();
}
public static ObjectNode createObjectNode(Object next) {
return MAPPER.valueToTree(next);
}

@adoroszlai
Copy link
Contributor

Thanks @ArafatKhan2198 for splitting this.

@adoroszlai adoroszlai closed this Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants