Skip to content
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
bb3704c
HDDS-8214. Recon - OM DB Insights - Container Level Info.
Mar 30, 2023
823ead5
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 4, 2023
f8e0d79
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 4, 2023
8dacd12
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 4, 2023
168e651
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 4, 2023
cace42c
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 5, 2023
f628968
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 5, 2023
9fc1eaf
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 5, 2023
285ddb1
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 17, 2023
05141a3
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 17, 2023
7091198
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
fb210c3
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
eb23a87
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
3e1e87b
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
72f6c40
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
18abcb9
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
410b846
HDDS-8214. Recon - OM DB Insights - Key Level Info.
Apr 19, 2023
737734c
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 5, 2023
25171b5
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 11, 2023
a169db1
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 11, 2023
90c5a15
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 15, 2023
6135735
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 15, 2023
5ae1ee4
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 15, 2023
76dd31a
HDDS-8214. Recon - OM DB Insights - Key Level Info.
May 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -481,4 +481,47 @@ private List<ContainerBlockMetadata> getBlocks(
return blockIds;
}

/** This method retrieves set of keys/files/dirs which are mapped to
* containers in DELETED state in SCM. */
@GET
@Path("/mismatch/keys")
public Response getDeletedContainerKeysInfo(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unable to get use case where getting keys only for deleted container from SCM. I think we should return all keys which is not present in SCM (including deleted key) which may have data loss.
Additionally need see if gap between SCM and DN if DN do not have data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unable to get use case where getting keys only for deleted container from SCM. I think we should return all keys which is not present in SCM (including deleted key) which may have data loss. Additionally need see if gap between SCM and DN if DN do not have data.

As discussed, Please conclude on the use case.

@DefaultValue(DEFAULT_FETCH_COUNT) @QueryParam(RECON_QUERY_LIMIT)
int limit,
@DefaultValue(StringUtils.EMPTY) @QueryParam(RECON_QUERY_PREVKEY)
String prevKeyPrefix) {
List<KeysResponse> keysResponseList = new ArrayList<>();
try {
Map<Long, ContainerMetadata> omContainers =
reconContainerMetadataManager.getContainers(-1, 0);
List<ContainerInfo> deletedStateSCMContainers =
containerManager.getContainers(HddsProtos.LifeCycleState.DELETED);
List<Long> deletedStateSCMContainerIds =
deletedStateSCMContainers.stream()
.map(containerInfo -> containerInfo.getContainerID()).collect(
Collectors.toList());

List<Long> omContainerIdsMappedToDeletedSCMContainers =
omContainers.entrySet().stream()
.filter(
map -> deletedStateSCMContainerIds.contains(map.getKey()))
.map(map -> map.getKey()).collect(Collectors.toList());

omContainerIdsMappedToDeletedSCMContainers.forEach(containerId -> {
Response keysForContainer = getKeysForContainer(containerId, limit,
prevKeyPrefix);
KeysResponse keysResponse = (KeysResponse) keysForContainer.getEntity();
keysResponseList.add(keysResponse);
});
} catch (IOException ex) {
throw new WebApplicationException(ex,
Response.Status.INTERNAL_SERVER_ERROR);
} catch (IllegalArgumentException e) {
throw new WebApplicationException(e, Response.Status.BAD_REQUEST);
} catch (Exception ex) {
throw new WebApplicationException(ex,
Response.Status.INTERNAL_SERVER_ERROR);
}
return Response.ok(keysResponseList).build();
}
}
Loading