Fix scheduling for splits with locality requirement in Tardigrade

[arhimondr]

Description

Fixes several problems related to scheduling of non remotely accessible splits

Is this change a fix, improvement, new feature, refactoring, or other?

Fix

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

Core engine (Tardigrade)

How would you describe this change to a non-technical end user or system administrator?

Fixes scheduling for non remotely accessible splits in certain corner cases. Prior this fix in some queries scanning over non remotely accessible splits might've been failing.

Related issues, pull requests, and links

Documentation

(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

() No release notes entries required.
(x) Release notes entries required with the following suggested text:

# Tardigrade
* Fix scheduling for non remotely accessible splits 

[losipiuk]

would that make sense to have more than one partition on single node?

[arhimondr]

Hmm, interesting question. For example to make tasks smaller for more granular retries?

[losipiuk]

Yeah for example. For partitions, we should probably opt for those to be similarly sized. Building partitions on top of the enforced bucket<->node mapping does not necessarily imply that. Not something that we need to address here. Just a random thought.

[linzebing]

Seems existingRequirement will have at most one element. Then we don't really need a set?

[arhimondr]

A split has a list of hosts in it's requirement. The set is needed to support split specific requirements.

[linzebing]

I think it' beneficial to add a comment explaining the logic, e.g. "make sure all buckets mapped to the same node map to the same partition, such that locality requirements are respected in scheduling".