Add flag to automatically attempt to distribute parallel stages across nodes #5002
Labels
area/tekton
area/tekton indicates that the issue was reported against the Servereless Jenkins X Pipelines
kind/enhancement
An enhancement of an existing feature
priority/important-soon
Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
A problem we hit in Jenkins X's own production builds for the
jenkins-x-builders
repo is that Kaniko, particularly with very large images and layers, can cause problems with the Kubernetes node's container GC, particularly if there are multiple Kaniko pods running on the node at the same time. This is annoying. =)Something that might help us with this - and that will hopefully be useful in other contexts - would be to put an affinity policy on the
PipelineRun
saying "if possible, put the pods for stages in this run on different nodes from each other". It wouldn't be perfect, but it would definitely help.Now, this is not saying I'm going to add the ability to configure full affinity policies explicitly on the pipeline - first, the only way I can make this work requires using the
PipelineRun
name (to use in an anti-affinity policy that says "avoid nodes with a pod already on them with the labeltekton.dev/pipelineRun=THIS_PIPELINERUN_NAME
, and since the name is unique per run, you can't exactly hardcode the name into thejenkins-x.yml
. Second, I'm not sure there's enough value in allowing full affinity policy configuration to be worth the added complexity. So I'm just going to add an option (say,distributeParallelAcrossNodes: true
) that can be set on apipeline
'soptions
(but not a stage, since we only have control over the affinity policy on thePipelineRun
).The text was updated successfully, but these errors were encountered: