HDDS-6313. Remove replicas in ContainerStateMap when a container is deleted

[sodonnel]

What changes were proposed in this pull request?

In ContainerStateMap, there are several maps to hold various details, eg:

private final Map<ContainerID, ContainerInfo> containerMap;
private final Map<ContainerID, Set<ContainerReplica>> replicaMap;

When we add a new container, we add an entry to both of these sets. When a container is removed, we don’t see to remove from replicaMap (see below). There doesn’t seem to be any way to remove the replicas later once the containerMap entry is gone, so removing the container is leaking the replicas.

  public void removeContainer(final ContainerID id) {
    Preconditions.checkNotNull(id, "ContainerID cannot be null");
    if (contains(id)) {
      // Should we revert back to the original state if any of the below
      // remove operation fails?
      final ContainerInfo info = containerMap.remove(id);
      lifeCycleStateMap.remove(info.getState(), id);
      ownerMap.remove(info.getOwner(), id);
      repConfigMap.remove(info.getReplicationConfig(), id);
      typeMap.remove(info.getReplicationType(), id);
      // Flush the cache of this container type.
      flushCache(info);
      LOG.trace("Container {} removed from ContainerStateMap.", id);
    }
  } 

You cannot remove the replicas anyway later, as the methods check if the container exists first, which it no longer will, eg:

public void removeContainerReplica(final ContainerID containerID,
    final ContainerReplica replica) {
  Preconditions.checkNotNull(containerID);
  Preconditions.checkNotNull(replica);
  if (contains(containerID)) {
    replicaMap.get(containerID).remove(replica);
  }
} 

Note that deleting a container seems to be a rare operation (eg delete it manually from the CLI). Empty containers are currently marked as deleted, but as far as I can tell, they are not actually removed from SCM.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-6313

How was this patch tested?

Existing tests

[swagle]

+1 LGTM

[JacksonYao287]

LGTM +1！

[adoroszlai]

Nice catch.