Skip to content

Pod Created twice when cluster under pressure #14833

@shuangkun

Description

@shuangkun

Pre-requisites

  • I have double-checked my configuration
  • I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
  • I have searched existing issues and could not find a match for this bug
  • I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

The log like this, the same pod created twice:

time="2025-09-09T16:35:08.922Z" level=info msg="Created pod: workflow-template-dag-wsklb.B.D (workflow-template-dag-wsklb-echo-19727502)" namespace=default workflow=workflow-template-dag-wsklb
time="2025-09-09T16:29:34.050Z" level=info msg="cleaning up pod" action=deletePod key=default/workflow-template-dag-wsklb-echo-19727502/deletePod
time="2025-09-09T16:20:22.287Z" level=info msg="Created pod: workflow-template-dag-wsklb.B.D (workflow-template-dag-wsklb-echo-19727502)" namespace=default workflow=workflow-template-dag-wsklb

Version(s)

v3.5.13

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

1w+ workflows in cluster.

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions