Skip to content

controller-runtime: starting/stopping manager.Manager leaks goroutines #1280

@andy-kimball

Description

@andy-kimball

What happened:
Each time the controller manager is started and then stopped, it leaks goroutines (see the Go code below for a simple repro).

What you expected to happen:
When the controller manager is stopped, all resources should be cleaned up, and the leaktest package should not find any leaked goroutines.

How to reproduce it (as minimally and precisely as possible):
Run go test on the following code:

package temp

import (
	"github.com/fortytw2/leaktest"
	ctrl "sigs.k8s.io/controller-runtime"
	"sigs.k8s.io/controller-runtime/pkg/manager"
	"testing"
)

func Test(t *testing.T) {
	defer leaktest.Check(t)()

	for i := 0; i < 10; i++ {
		mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), manager.Options{Port: 9443})
		if err != nil {
			panic(err)
		}

		stopChan := make(chan struct{})
		close(stopChan)
		err = mgr.Start(stopChan)
		if err != nil {
			panic(err)
		}
	}
}

Anything else we need to know?:
Here are some of the goroutines being leaked (duplicated each time manager is started and stopped):

        k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc000317580)
        	/Users/andyk/go/pkg/mod/k8s.io/[email protected]/pkg/watch/mux.go:207 +0x66
        created by k8s.io/apimachinery/pkg/watch.NewBroadcaster
        	/Users/andyk/go/pkg/mod/k8s.io/[email protected]/pkg/watch/mux.go:75 +0xce
    leaktest.go:150: leaktest: leaked goroutine: goroutine 343 [chan receive]:
        k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1(0x23cc0e0, 0xc000994540, 0xc000091b10)
        	/Users/andyk/go/pkg/mod/k8s.io/[email protected]/tools/record/event.go:288 +0xaa
        created by k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher
        	/Users/andyk/go/pkg/mod/k8s.io/[email protected]/tools/record/event.go:286 +0x6e

Environment:
Kubernetes version (use kubectl version): 1.19
Cloud provider or hardware configuration: N/A
OS (e.g: cat /etc/os-release): Mac
Kernel (e.g. uname -a): Darwin Andrews-MacBook-Pro.local 19.6.0 Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT 2020; root:xnu-6153.141.2~1/RELEASE_X86_64 x86_64
Install tools:
Network plugin and version (if this is a network-related bug):
Others:

Metadata

Metadata

Assignees

No one assigned

    Labels

    lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions