Account memory for OrcDeletedRows - fix memory accounting for ORC ACID delete delta

[losipiuk]

No description provided.

[findepi]

sizeOfObjectArray(rowCount) + rowCount * RowId.INSTANCE_SIZE

[alexjo2144]

Is it worth doing setting this as we loop vs just doing it once at the end?
Also, can we safely ignore the return value of setBytes?

[losipiuk]

The benefit of doing that in the loop (I think) is that we can pause some other query for a while, allowing us to get to the end of loop without OOM. Other than that it does not change much I think.
@findepi am I missing something?

[findepi]

we should assume that building deletedRows list can exhaust node memory.
if we set memory usage only at the end, we still do not protect ourselves against node OOM.

Also, can we safely ignore the return value of setBytes?

we cannot block here (can we?).
we could use trySetBytes here, perhaps.

[losipiuk]

AC

[losipiuk]

we cannot block here (can we?).

It does not look like. Maybe with a significant refactoring.

we could use trySetBytes here, perhaps.

I was thinking about that. But then what if it returns that we cannot allocate. Throw exception ourselves?

[findepi]

we could use trySetBytes here, perhaps.

I was thinking about that. But then what if it returns that we cannot allocate. Throw exception ourselves?

yes, we would throw then.

cc @sopel39 for LocalMemoryContext.setBytes vs trySetBytes.

[sopel39]

add a comment why

[losipiuk]

dropped as data is not passed to operator context immediately.

[sopel39]

nit: extra space before as