Support for killing individual task if cluster is out of memory

[losipiuk]

Description

Support for killing individual task if cluster is out of memory.
Individual tasks will be killed for queries which are run with task-level retries.
Currently only supported if default low memory killer policy is used (total-reservation-on-blocked-nodes).

Is this change a fix, improvement, new feature, refactoring, or other?

improvement to new feature

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

core query engine

How would you describe this change to a non-technical end user or system administrator?

N/A

Documentation

(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

( ) No release notes entries required.
(x) Release notes entries required with the following suggested text:

# Section
* Add support for killing individual tasks of queries run with ``retry-mode`` set to ``TASK`` low memory killer.
  Requires ``query.low-memory-killer.policy`` config option to be set to ``total-reservation-on-blocked-nodes``. ({issue}`11129`)

[arhimondr]

Looks good to me % comments

[losipiuk]

AC

[losipiuk]

@martint could you maybe take a look?

[losipiuk]

Major changes:

• Rebased on top of Support for killing individual task if cluster is out of memory #11129.
• Dropped mechanism of per-task memory limit. We are not enforcing limits right now. We are estimating tasks size, and bin-packing based on that, but we allow it to go above. If memory pool is exhaust memory kill will intervene and kill one of the tasks
• I dropped integration test. With memory-killer approach I was not able to make it work (deterministically enforce that one of the tasks would require full not allocation)

@arhimondr PTAL once again

NVM - comment was meant for different PR

[losipiuk]

CI: #10631 (Table '...' not found)