Partial WAL entry set causes persistent failure

[ewencp]

See https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/confluent-platform/cHeTBYAZBH8/qgrKXjAmCwAJ for a description of the issue.

If a node dies having only written 5 of the 10 entries in a WAL commit block for files, recovery just fails persistently. We have the correct behavior in the sense that we do not try to apply the WAL entries until we hit the endMarker for a commit. But the FSWAL.apply() code just bails if it hits EOF (or any IOException). Instead of just bailing, it should recover by ignoring the bad set of entries, giving up on committing data, truncate the log, and proceed from the last known safe point based on offsets seen in files in HDFS.

[lomignet]

Wouldn't this commit Eneco@f038915 help?