fix: cluster workflow template store is not initialized in namespace mode. Fixes #14763

[eduardodbr]

Fixes #14763

Motivation

Submitting a workflow via UI, either from workflow template or via cron workflow, in namespace mode is failing because the informer is not initialized.

Modifications

The template store is always initialized to NullClusterWorkflowTemplate when RBAC is not enabled.

Verification

Run with namespace-install.yaml.

Submited via:

kubectl create
Argo submit
Cron submit via UI
Workflow template submit via UI
argoKubeClient

Documentation

[mraerino]

even with this patch, i get a nil-pointer error when trying to use the api client on a cluster where i have no access to ClusterWorkflowTemplates

Stack Trace

2025/08/18 19:01:05 http: panic serving [::1]:53376: runtime error: invalid memory address or nil pointer dereference
goroutine 14 [running]:
net/http.(*conn).serve.func1()
        /Users/uqzb/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.darwin-arm64/src/net/http/server.go:1943 +0xb4
panic({0x104a2dae0?, 0x1067493c0?})
        /Users/uqzb/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.0.darwin-arm64/src/runtime/panic.go:783 +0x120
github.com/argoproj/argo-workflows/v3/server/workflow.(*workflowServer).SubmitWorkflow(0x14000830580, {0x104ee8598, 0x1400099c550}, 0x14000518ae0)
        /Users/uqzb/go/pkg/mod/github.com/eduardodbr/argo-workflows/v3@v3.0.0-20250815172450-f2599c34015f/server/workflow/workflow_server.go:804 +0x694
github.com/argoproj/argo-workflows/v3/pkg/apiclient.(*argoKubeWorkflowServiceClient).SubmitWorkflow(0x23?, {0x104ee8598?, 0x1400099c550?}, 0x1040e51ac?, {0x14000893240?, 0xb?, 0x14000069788?})
        /Users/uqzb/go/pkg/mod/github.com/eduardodbr/argo-workflows/v3@v3.0.0-20250815172450-f2599c34015f/pkg/apiclient/argo-kube-workflow-service-client.go:122 +0x2c
github.com/argoproj/argo-workflows/v3/pkg/apiclient.(*errorTranslatingWorkflowServiceClient).SubmitWorkflow(0x104eb2378?, {0x104ee8598?, 0x1400099c550?}, 0x1046e35f8?, {0x140000698f8?, 0x2?, 0x2?})
        /Users/uqzb/go/pkg/mod/github.com/eduardodbr/argo-workflows/v3@v3.0.0-20250815172450-f2599c34015f/pkg/apiclient/error-translating-workflow-service-client.go:100 +0x38
main.(*Handler).processAlert(0x1400082ce70, {0x104ee8598, 0x1400099c550}, 0x14000830800, {{0x140008931c0, 0x6}, 0x14000b89aa0, 0x14000b89ad0, {0x2c10513c, 0xed2f600ba, ...}, ...})

it seems that the argoKubeClient does not initialize the cwftmplStore in all cases, but argoWorkflowServer always tries to call Getter on it (on an interface, so this patch does not fix it)

[mraerino]

@eduardodbr i might provide a PR on top of your PR if that's cool?

[eduardodbr]

Sure go ahead

[mraerino]

here is my added fix: eduardodbr#1

If it doesn't make sense to integrate it into this PR, i'm happy to do it against the main repo

[eduardodbr]

@mraerino can you provide a reproducible example to test? thanks

[mraerino]

Here is some code to reproduce:

main.go

package main

import (
	"os"

	argoclient "github.com/argoproj/argo-workflows/v3/pkg/apiclient"
	wfclient "github.com/argoproj/argo-workflows/v3/pkg/apiclient/workflow"
	"github.com/argoproj/argo-workflows/v3/pkg/apis/workflow"
	argoworkflowsv1alpha1 "github.com/argoproj/argo-workflows/v3/pkg/apis/workflow/v1alpha1"
	"k8s.io/client-go/tools/clientcmd"
)

func main() {
	loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
	clientcfg := clientcmd.NewInteractiveDeferredLoadingClientConfig(loadingRules, nil, os.Stdin)

	argoCtx, argoClient, err := argoclient.NewClientFromOpts(argoclient.Opts{
		ArgoKubeOpts: argoclient.ArgoKubeOpts{
			CacheWorkflowTemplates: true,
		},
		ClientConfigSupplier: func() clientcmd.ClientConfig {
			return clientcfg
		},
	})
	if err != nil {
		panic(err)
	}

	c := argoClient.NewWorkflowServiceClient()
	_, err = c.SubmitWorkflow(argoCtx, &wfclient.WorkflowSubmitRequest{
		Namespace:    "dc-automation",
		ResourceKind: workflow.WorkflowTemplateKind,
		ResourceName: "netbox-info",
		SubmitOptions: &argoworkflowsv1alpha1.SubmitOpts{
			ServerDryRun: true,
		},
	})
	if err != nil {
		panic(err)
	}
}

Backtrace

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x18 pc=0x103d4c0f4]

goroutine 1 [running]:
github.com/argoproj/argo-workflows/v3/server/workflow.(*workflowServer).SubmitWorkflow(0x140008f2300, {0x104a887f8, 0x140006d59b0}, 0x14000a56600)
        /Users/uqzb/go/pkg/mod/github.com/argoproj/argo-workflows/v3@v3.7.1/server/workflow/workflow_server.go:797 +0x694
github.com/argoproj/argo-workflows/v3/pkg/apiclient.(*argoKubeWorkflowServiceClient).SubmitWorkflow(0x104a53ac0?, {0x104a887f8?, 0x140006d59b0?}, 0x103d65400?, {0xa0?, 0x1049a0ce0?, 0x140008f5401?})
        /Users/uqzb/go/pkg/mod/github.com/argoproj/argo-workflows/v3@v3.7.1/pkg/apiclient/argo-kube-workflow-service-client.go:122 +0x2c
github.com/argoproj/argo-workflows/v3/pkg/apiclient.(*errorTranslatingWorkflowServiceClient).SubmitWorkflow(0x140006db680?, {0x104a887f8?, 0x140006d59b0?}, 0x0?, {0x0?, 0x0?, 0x0?})
        /Users/uqzb/go/pkg/mod/github.com/argoproj/argo-workflows/v3@v3.7.1/pkg/apiclient/error-translating-workflow-service-client.go:100 +0x38
main.main()
        /Users/<redacted>/main.go:30 +0x204
exit status 2

This is with a kubernetes role that can access WorkflowTemplate but not ClusterWorkflowTemplate

[eduardodbr]

@mraerino thank you for your contribution. I have made a small change to also make sure argo server always starts clusterworkflowtemplate store

[piby180]

Hi,
Thanks for the fix. When can we expect a new patch version for Argo Workflows with this change? We updated to 3.7.1 and submit functionality is currently broker on Argo UI. We have to downgrade soon if we don't fix this issue in a few days.

[terrytangyuan]

LGTM. Thanks for the fix!