kubernetes: bottlerocket doesn't understand the "Exists" taint

[diranged]

Platform I'm building on:
Bottlerocket 1.0.5 on EKS 1.18

What I expected to happen:
Kubernetes has an Exists taint that has no value key in it. So the CLI argument looks like this:

kubelet ... --register-with-taints=MyKey:NoSchedule

and a matching Toleration would look like this:

tolerations:
  - operator: Exists
    key: MyKey
    effect: NoSchedule

However, there is no way to put that into the settings.kubernetes.node-taints configuration because the pattern is hard-coded to expect an Equals taint:

bottlerocket/packages/kubernetes-1.18/kubelet-env

Line 3 in b5fd55f

NODE_TAINTS={{join_map "=" "," "no-fail-if-missing" settings.kubernetes.node-taints}}

What actually happened:
I could not boot up nodes - they failed to parse the key and threw an error.

[etungsten]

Hi @diranged,

The --register-with-taints argument to kubelet takes a list of taints of the form <key1>=<value1>:<effect1>,<key2>=<value2>:<effect2>. The equal sign is a necessary part of the separators. (See documentation for this argument here and here).

The problem is that the validation we have for KubernetesTaintValue currently doesn’t allow for an empty taint value.

bottlerocket/sources/models/src/modeled_types/kubernetes.rs

Lines 206 to 236 in fdb0353

	// =^..^= =^..^= =^..^= =^..^= =^..^= =^..^= =^..^= =^..^= =^..^=
	/// KubernetesTaintValue represents a string that contains a valid Kubernetes taint value, which is
	/// like a label value, plus a colon, plus an "effect". It stores the original string and makes it
	/// accessible through standard traits.
	///
	/// Note: Kubelet won't launch if you specify an effect it doesn't know about, but we don't want to
	/// gatekeep all possible values, so be careful.
	// Note: couldn't find an exact spec for this. Cobbling things together, and guessing a bit as to
	// the syntax of the effect.
	// https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
	// https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
	#[derive(Debug, Clone, Eq, PartialEq, Hash)]
	pub struct KubernetesTaintValue {
	inner: String,
	}
	lazy_static! {
	pub(crate) static ref KUBERNETES_TAINT_VALUE: Regex = Regex::new(
	r"(?x)^
	[[:alnum:]] # at least one alphanumeric
	(
	([[:alnum:]._-]{0,61})? # more characters allowed in middle
	[[:alnum:]] # have to end with alphanumeric
	)?
	: # separate the label value from the effect
	[[:alnum:]]{1,253} # effect
	$"
	)
	.unwrap();
	}

This is being fixed in #1406. Once that’s merged, you should be able to specify taints without values and use Exists tolerations. For example, in userdata:

[settings.kubernetes.node-taints]
MyKey = ":NoSchedule"
SomeOtherKey = "SomeValue:NoSchedule"