test(controller): race condition in handleSigterm()

[vflaux]

What does it do ?

Fix a race condition between the signal handler setup and the signal send in the test for handleSigterm().

In the test, in rare case the SIGTERM signal could be send before the handleSigterm() calls signal.Notify().

In this case, the cancel function is never called.

The test always pass because it silently ignore this with the select case sig := <-sigChan.

I think we need to test that the signal received is a SIGTERM and that the cancel func has been called. Not one of the two conditions.

Motivation

This primarily concerns testing and ensuring correctness. There is most likely no issue when the controller run.

I could reproduce this with go test -timeout 30s -count 1000 -run ^TestHandleSigterm$ sigs.k8s.io/external-dns/controller after removing the sig := <-sigChan select case in the current code.
(edit: test was removed in #5816)

Details

func TestHandleSigterm(t *testing.T) {
	cancelCalled := make(chan bool, 1)
	cancel := func() {
		cancelCalled <- true
	}

	var logOutput bytes.Buffer
	log.SetOutput(&logOutput)
	defer log.SetOutput(os.Stderr)

	go handleSigterm(cancel)

	// Simulate sending a SIGTERM signal
	sigChan := make(chan os.Signal, 1)
	signal.Notify(sigChan, syscall.SIGTERM)
	err := syscall.Kill(syscall.Getpid(), syscall.SIGTERM)
	assert.NoError(t, err)

	// Wait for the cancel function to be called
	select {
	case <-cancelCalled:
		assert.Contains(t, logOutput.String(), "Received SIGTERM. Terminating...")
	// case sig := <-sigChan:
	// 	assert.Equal(t, syscall.SIGTERM, sig)
	case <-time.After(1 * time.Second):
		t.Fatal("cancel function was not called")
	}
}

$ go test -timeout 30s -count 1000 -run ^TestHandleSigterm$ sigs.k8s.io/external-dns/controller
--- FAIL: TestHandleSigterm (1.00s)
    execute_test.go:350: cancel function was not called
--- FAIL: TestHandleSigterm (1.00s)
    execute_test.go:350: cancel function was not called
--- FAIL: TestHandleSigterm (1.00s)
    execute_test.go:350: cancel function was not called
--- FAIL: TestHandleSigterm (1.00s)
    execute_test.go:350: cancel function was not called
time="2026-03-28T04:55:43+01:00" level=info msg="Received SIGTERM. Terminating..."
--- FAIL: TestHandleSigterm (1.00s)
    execute_test.go:350: cancel function was not called
--- FAIL: TestHandleSigterm (1.00s)
    execute_test.go:350: cancel function was not called
time="2026-03-28T04:55:45+01:00" level=info msg="Received SIGTERM. Terminating..."
FAIL
FAIL    sigs.k8s.io/external-dns/controller     6.352s
FAIL

After this patch:

$ go test -timeout 30s -count 1000 -run ^TestHandleSigterm$ sigs.k8s.io/external-dns/controller
ok      sigs.k8s.io/external-dns/controller     0.032s [no tests to run]

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

[k8s-ci-robot]

Hi @vflaux. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coveralls]

Pull Request Test Coverage Report for Build 23676867886

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 10 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.007%) to 78.215%

---

Files with Coverage Reduction	New Missed Lines	%
execute.go	10	76.64%

Totals
Change from base Build 23660098847:	0.007%
Covered Lines:	16408
Relevant Lines:	20978

---

💛 - Coveralls

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

• Mark this PR as fresh with /remove-lifecycle stale
• Close this PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ivankatliarchuk]

Need to better understand the benefits.

/ok-to-test

[ivankatliarchuk]

Worth to execute binary and terminate it. To validate behaviour. This sigterms, I'm not sure

[ivankatliarchuk]

Suggested change

	t.Fatal("signal was not recieved")
	t.Fatal("signal was not received")

[ivankatliarchuk]

case sig := <-sigChan:
      assert.Equal(t, syscall.SIGTERM, sig)  // Always true if sigChan fires

[ivankatliarchuk]

Goroutine leak in tests: The goroutine spawned by setupSigtermHandler is blocked on <-signals. After defer signal.Reset(syscall.SIGTERM) runs, nothing will ever send on that channel, so the goroutine leaks for the duration of the test binary

[ivankatliarchuk]

maybe similar, hard to say

func TestSetupSigtermHandler(t *testing.T) {
      cancelCalled := make(chan bool, 1)
      cancel := func() { cancelCalled <- true }

      hook := logtest.LogsUnderTestWithLogLevel(log.InfoLevel, t)

      defer signal.Reset(syscall.SIGTERM)
      setupSigtermHandler(cancel)

      err := syscall.Kill(syscall.Getpid(), syscall.SIGTERM)
      assert.NoError(t, err)

      select {
      case <-cancelCalled:
          logtest.TestHelperLogContainsWithLogLevel("Received SIGTERM. Terminating...", log.InfoLevel, hook, t)
      case <-time.After(5 * time.Second):
          t.Fatal("cancel function was not called")
      }
}

[ivankatliarchuk]

The goroutine leaks a registration with the signal package. After receiving, we should deregister:

  go func() {
      <-signals
      signal.Stop(signals)
      log.Info("Received SIGTERM. Terminating...")
      cancel()
  }()

[ivankatliarchuk]

Can we not simply replace the whole logic with signal.NotifyContext?

in Execute()

// Before
  ctx, cancel := context.WithCancel(context.Background())
  setupSigtermHandler(cancel)

// After
 ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
 defer stop()

[ivankatliarchuk]

+

context.AfterFunc(ctx, func() {
      log.Info("Received SIGTERM. Terminating...")
  })

context.AfterFunc runs the callback in its own goroutine once the context is cancelled (for any reason — signal or otherwise), so the log line is preserved without a custom handler.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from ivankatliarchuk. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment