Passing k8s node-affinity clauses to user and system workloads of kurtosis

[0x416e746f6e]

Background & motivation

We want to be able to completely isolate kurtosis workloads from the rest in our k8s cluster(s). We could use taints/tolerations for that, but it would be very cumbersome to add artificial taints to everything to just keep kurtosis away from the nodes where it shouldn't be.

Desired behaviour

It will be far more logical and easy to maintain if we could pass node-affinity clauses to kurtosis workloads (both, user ones and system ones like kurtosis-engine-xxx).

How important is this to you?

Painful; the lack of this feature makes using Kurtosis frictionful.

What area of the product does this pertain to?

CLI: the Command Line Interface

[tedim52]

There isn't a way to specify this for an engine or APIC so this makes sense. - We do have the node_selector label on the ServiceConfig but it seems like for your k8s setup, you need the expressivity of node affinities. Do you have environment with varying node setups / types in your cluster with nodes maybe dying / being provisioned frequently?

[tedim52]

Would you like to be able to configure node affinities on a per service basis or is passing one node affinity clause that applies to all services managed by Kurtosis sufficient? (not referring to engine and enclave containers, only user services)