Skip to content

HDDS-7739. EC: Increase the information in the RM sending command log message #4153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adoroszlai merged 1 commit into from
Jan 9, 2023
Merged

HDDS-7739. EC: Increase the information in the RM sending command log message #4153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/protocol/DatanodeDetails.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,15 @@ public String getHostName() {
return hostName;
}

/**
* Returns the Hostname and IP of the datanode separated by a slash.
*
* Eg: datanode001.corp/192.168.0.123
*/
public String getHostNameAndIP() {
return getHostName() + "/" + getIpAddress();
Copy link
Contributor

@adoroszlai adoroszlai Jan 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to make this (or UUID + this) the default toString, and renaming the existing more verbose toString to something like toDebugString. This way we could avoid having to convert manually in all places, and use the verbose one explicitly if needed.

Copy link
Contributor Author

@sodonnel sodonnel Jan 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with adding UUID into the string too. It would solve the problem with mini-cluster where all the nodes are on the same host / IP. I wonder if anything would break if we changed the default datanode details to string? Almost everywhere it is logged, we don't call about all the verbose info it currently includes. We really just want to the host / IP / ID.

}

/**
* Sets a DataNode Port.
*
Expand Down
10 changes: 10 additions & 0 deletions ...ervice/src/main/java/org/apache/hadoop/ozone/protocol/commands/CloseContainerCommand.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,14 @@ public long getContainerID() {
public PipelineID getPipelineID() {
return pipelineID;
}

@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append(getType())
.append(": containerID: ").append(getContainerID())
.append(", pipelineID: ").append(getPipelineID())
.append(", force: ").append(force);
return sb.toString();
}
}
10 changes: 10 additions & 0 deletions ...rvice/src/main/java/org/apache/hadoop/ozone/protocol/commands/DeleteContainerCommand.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,4 +105,14 @@ public static DeleteContainerCommand getFromProtobuf(
public int getReplicaIndex() {
return replicaIndex;
}

@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append(getType())
.append(": containerID: ").append(getContainerID())
.append(", replicaIndex: ").append(getReplicaIndex())
.append(", force: ").append(force);
return sb.toString();
}
}
17 changes: 17 additions & 0 deletions ...c/main/java/org/apache/hadoop/ozone/protocol/commands/ReconstructECContainersCommand.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,23 @@ public ECReplicationConfig getEcReplicationConfig() {
return ecReplicationConfig;
}

@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append(getType())
.append(": containerID: ").append(containerID)
.append(", replicationConfig: ").append(ecReplicationConfig)
.append(", sources: [").append(getSources().stream()
.map(a -> a.dnDetails.getHostNameAndIP()
+ " replicaIndex: " + a.getReplicaIndex())
.collect(Collectors.joining(", "))).append("]")
.append(", targets: [").append(getTargetDatanodes().stream()
.map(DatanodeDetails::getHostNameAndIP)
.collect(Collectors.joining(", "))).append("]")
.append(", missingIndexes: ").append(
Arrays.toString(missingContainerIndexes));
return sb.toString();
}
/**
* To store the datanode details with replica index.
*/
Expand Down
13 changes: 13 additions & 0 deletions ...ce/src/main/java/org/apache/hadoop/ozone/protocol/commands/ReplicateContainerCommand.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,4 +109,17 @@ public List<DatanodeDetails> getSourceDatanodes() {
public int getReplicaIndex() {
return replicaIndex;
}

@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append(getType());
sb.append(": containerId: ").append(getContainerID());
sb.append(", replicaIndex: ").append(getReplicaIndex());
sb.append(", sourceNodes: [");
sb.append(sourceDatanodes.stream()
.map(DatanodeDetails::getHostNameAndIP)
.collect(Collectors.joining(", "))).append("]");
return sb.toString();
}
}
4 changes: 2 additions & 2 deletions ...cm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,8 +423,8 @@ public void sendDeleteCommand(final ContainerInfo container, int replicaIndex,
public void sendDatanodeCommand(SCMCommand<?> command,
ContainerInfo containerInfo, DatanodeDetails target)
throws NotLeaderException {
LOG.info("Sending command of type {} for container {} to {}",
command.getType(), containerInfo, target);
LOG.info("Sending command [{}] for container {} to {}",
command, containerInfo, target);
command.setTerm(getScmTerm());
command.setDeadline(clock.millis() +
Math.round(rmConf.eventTimeout * rmConf.commandDeadlineFactor));
Expand Down