HDDS-6130. [Ozone-Streaming] When releaseBuffers will get “Couldn 't find the required future”

[captainzmc]

What changes were proposed in this pull request?

See: https://issues.apache.org/jira/browse/HDDS-6130

How was this patch tested?

Existing tests: testPutBlockAtBoundary

[sadanand48]

Thanks @captainzmc for the PR. I just realised that this problem is due to the wrongly computed totalAckDataLength. Since we are not doing watchForCommit() after periodic putBlock() i.e in doFlushIfNeeded() , it might happen that the same set of buffers are inserted into the commitInfoMap more than once because of which totalAckData is wrongly computed and we get this error. I think for now we can just add watchForCommit() after doing putBlock in here which should solve the issue right away,

[captainzmc]

I think for now we can just add watchForCommit() after doing putBlock in here which should solve the issue right away,

Thanks @sadanand48 for your advice, I have updated the PR and the testPutBlockAtBoundary can be used to verify this fix.

Through testing, I found that only add watchForCommit() is not enough. ExecutePutBlock() is only an asynchronous to execute updateCommitInfoMap. we should add waitOnFlushFutures() for ensure that updateCommitInfoMap is executed. Without waitOnFlushFutures() we'll have the same problem.

[captainzmc]

@sadanand48
Oops! I found that our cleanup of futureMap in releaseBuffers was redundant. We have called the commitWatcher cleanup in close. Since futureMap doesn't store much and doesn't affect the client's memory, we can simply clean up it on close.
Let's just delete this part so it looks simpler and cleaner.

[szetszwo]

@captainzmc , you are right that the error messages are inaccurate.

After the change, the futureMap is no longer useful. Let's remove it; see https://issues.apache.org/jira/secure/attachment/13037847/2939_review.patch

[szetszwo]

+1 the change looks good.