feat: use effective memory size for memory management purpose

[yjshen]

Which issue does this PR close?

Closes #9417.

Rationale for this change

ArrayData::get_slice_memory_size serves our requirements for memory management better.

What changes are included in this PR?

Introduce an EffectiveSize trait in common::utils, and substitute batch.get_array_memory_size() usages with batch.get_effective_memory_size() for codes related to memory management.

Are these changes tested?

Existing tests and also the case noted in the issue.

Are there any user-facing changes?

No.

[Dandandan]

I don't think this is really what we want for memory tracking? If we slice a batch, we still hold on to the original array (zero copy).
The slice itself has minimal metadata overhead, but the data remains in the original batch.

[Dandandan]

So in the situation we slice a batch and use it once, this will underreport memory usage. In the situation we will keep both the original batch and the sliced one in different operators, this will still overreport memory usage.

[yjshen]

By checking the current codebase, I think we are using the slice for several cases:

1. Slice large/mono batch into slices based on batch_size conf. (agg output, file scan output, join output)
2. Partitioning existing batch based on certain criteria. (window/sort batch partitioning)
3. limit (Is this the only one that only part of the batch can be used?)

For both cases 1 and 2, since we will use all sliced small batches in subsequent operators, using the effective size would make overreporting more reasonable, I suppose. The overreport be reduced from N times to 2 times compared to the actual batch size?

For case 3, you are correct that it will result in underreporting, but considering that a limit operation would most likely be on the upper level of a DAG, would memory usage be less of a concern?

Another approach to avoid overreporting could be adding a tag to the slice-generated batch. It should report size 0 for the sliced batch since the original, large batch should already be memory-tracked elsewhere.

[alamb]

I have a different suggested approach which I wrote up on #9417 (comment)