Skip to content

HDDS-7926. [hsync] Recon throws ClassCastException. #4266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ChenSammi merged 9 commits into from
Mar 1, 2023
Merged

HDDS-7926. [hsync] Recon throws ClassCastException. #4266

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
0332cb2
HDDS-7926. [hsync] Recon throws ClassCastException.
ArafatKhan2198 Feb 10, 2023
c449415
Made review changes
ArafatKhan2198 Feb 13, 2023
84a1e60
Made changes to ignore the RepeatedOmKeyInfo object
ArafatKhan2198 Feb 19, 2023
4146f15
Added a check for classCast exception in NSSummaryTaskWithLegacy
ArafatKhan2198 Feb 27, 2023
394aa7a
Fixed bugs
ArafatKhan2198 Feb 27, 2023
1ca9fdc
Fixed bugs and failing UT's
ArafatKhan2198 Feb 27, 2023
3e91e0c
Made code changes as per review
ArafatKhan2198 Feb 28, 2023
f376d69
Fixed the log messages
ArafatKhan2198 Feb 28, 2023
1b9a97d
Changed log level to warn
ArafatKhan2198 Mar 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 35 additions & 21 deletions hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/FileSizeCountTask.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,37 +122,51 @@ public Pair<String, Boolean> process(OMUpdateEventBatch events) {
final Collection<String> taskTables = getTaskTables();

while (eventIterator.hasNext()) {
OMDBUpdateEvent<String, OmKeyInfo> omdbUpdateEvent = eventIterator.next();
OMDBUpdateEvent<String, Object> omdbUpdateEvent = eventIterator.next();
// Filter event inside process method to avoid duping
if (!taskTables.contains(omdbUpdateEvent.getTable())) {
continue;
}
String updatedKey = omdbUpdateEvent.getKey();
OmKeyInfo omKeyInfo = omdbUpdateEvent.getValue();
Object value = omdbUpdateEvent.getValue();
Object oldValue = omdbUpdateEvent.getOldValue();

try {
switch (omdbUpdateEvent.getAction()) {
case PUT:
handlePutKeyEvent(omKeyInfo, fileSizeCountMap);
break;
if (value instanceof OmKeyInfo) {
OmKeyInfo omKeyInfo = (OmKeyInfo) value;
OmKeyInfo omKeyInfoOld = (OmKeyInfo) oldValue;

case DELETE:
handleDeleteKeyEvent(updatedKey, omKeyInfo, fileSizeCountMap);
break;
try {
switch (omdbUpdateEvent.getAction()) {
case PUT:
handlePutKeyEvent(omKeyInfo, fileSizeCountMap);
break;

case UPDATE:
handleDeleteKeyEvent(updatedKey, omdbUpdateEvent.getOldValue(),
fileSizeCountMap);
handlePutKeyEvent(omKeyInfo, fileSizeCountMap);
break;
case DELETE:
handleDeleteKeyEvent(updatedKey, omKeyInfo, fileSizeCountMap);
break;

default: LOG.trace("Skipping DB update event : {}",
omdbUpdateEvent.getAction());
case UPDATE:
if (omKeyInfoOld != null) {
handleDeleteKeyEvent(updatedKey, omKeyInfoOld, fileSizeCountMap);
handlePutKeyEvent(omKeyInfo, fileSizeCountMap);
} else {
LOG.warn("Update event does not have the old keyInfo for {}.",
updatedKey);
}
break;

default:
LOG.trace("Skipping DB update event : {}",
omdbUpdateEvent.getAction());
}
} catch (Exception e) {
LOG.error("Unexpected exception while processing key {}.",
updatedKey, e);
return new ImmutablePair<>(getTaskName(), false);
}
} catch (Exception e) {
LOG.error("Unexpected exception while processing key {}.",
updatedKey, e);
return new ImmutablePair<>(getTaskName(), false);
} else {
LOG.warn("Unexpected value type {} for key {}. Skipping processing.",
value.getClass().getName(), updatedKey);
}
}
writeCountsToDB(false, fileSizeCountMap);
Expand Down
14 changes: 10 additions & 4 deletions ...zone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/NSSummaryTaskWithLegacy.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,16 @@ public boolean processWithLegacy(OMUpdateEventBatch events) {
String updatedKey = omdbUpdateEvent.getKey();

try {
OMDBUpdateEvent<String, OmKeyInfo> keyTableUpdateEvent =
(OMDBUpdateEvent<String, OmKeyInfo>) omdbUpdateEvent;
OmKeyInfo updatedKeyInfo = keyTableUpdateEvent.getValue();
OmKeyInfo oldKeyInfo = keyTableUpdateEvent.getOldValue();
OMDBUpdateEvent<String, ?> keyTableUpdateEvent = omdbUpdateEvent;
Object value = keyTableUpdateEvent.getValue();
Object oldValue = keyTableUpdateEvent.getOldValue();
if (!(value instanceof OmKeyInfo)) {
LOG.warn("Unexpected value type {} for key {}. Skipping processing.",
value.getClass().getName(), updatedKey);
continue;
}
OmKeyInfo updatedKeyInfo = (OmKeyInfo) value;
OmKeyInfo oldKeyInfo = (OmKeyInfo) oldValue;

// KeyTable entries belong to both Legacy and OBS buckets.
// Check bucket layout and if it's OBS
Expand Down