chore(tests): Add YAML-driven integration test framework for sources

[ivankatliarchuk]

What does it do ?

I'm going to add more tests in follow-up, in this PR focus is on adding the testing framework.

Main challanges this should help to solve

• we have multiple layers like

  • combination of multiple sources (this is currently not tested at all)
  • wrappers and post processors - on top of multiple sources
  • inconsistent values passed to providers

• wrappers are going to play key role in future, especially when we remove most of the providers and keep only webhook. Most of the logic is going to be happening in service postprocessors -> and this combination service + service + wrapper + wrapper needs a testing framework

  • Scenario-based test framework using YAML configuration
  • Demonstrating pattern
    • Multieple sources with target filter
    • Multiple sources with deduplications
  • Auto-generation of dependent resources (Pods, EndpointSlices) from service definitions
    • Initial headless service test case demonstrating

The folder tests/integration/toolkit require revisit, theres are some duplicates, in follow-up I'll try to review designs to reduce duplicaiton

Motivation

At the moment we are testing in isolation: sources, wrapper. This approach misses the behavioral tests, when multiple sources with wrappers(processors,post-processors, deduplicates are enabled)

Unit tests for sources are scattered and require significant boilerplate to set up complex scenarios like headless services with pods and endpoint slices or multiple sources behaviour + wrappers to modify behaviour. A declarative approach simplifies adding new test cases.

Follow-up

• add inmemory provider to have it trully end-2-end
• Address // TODO: copied from controller/execute.go#buildSources. if the controller's buildSources logic changes, this copy diverges silently.
• Duplicated helpers -> move them to shared location
• more scenarios with current known limitations/bugs

More

• Yes, this PR title follows Conventional Commits
• Yes, I added unit tests
• Yes, I updated end user documentation accordingly

Use cases (in theory should help to prevent regressions)

• feat(endpoint): reject alias property on unsupported record types #6188
• feat(source)!: introduce optional force-default-targets #5316
• fix(flags): restore --min-ttl option #6014
• --service-type-filter=ClusterIP + headless service with external-dns.alpha.kubernetes.io/endpoints-type=NodeExternalIP does not produce records #5867 (comment)
• and more, as we have multiple layers and changes in one layer may change some others

[coveralls]

Pull Request Test Coverage Report for Build 23136525461

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage decreased (-1.0%) to 78.192%

---

Files with Coverage Reduction	New Missed Lines	%
openshift_route.go	1	83.59%

Totals
Change from base Build 23136104975:	-1.0%
Covered Lines:	16576
Relevant Lines:	21199

---

💛 - Coveralls

[ivankatliarchuk]

/test pull-external-dns-unit-test

[vflaux]

Only description is required?

[ivankatliarchuk]

added few more fields. I was thinking about adding github.com/go-playground/validator/v10. and

The struct tag approach would look like:
  type Scenario struct {
      Name        string `json:"name"        validate:"required"`
      Description string `json:"description" validate:"required"`
      Config      ScenarioConfig             `json:"config"`
      ...
  }

  And in LoadScenarios:
  validate := validator.New()
  for i, s := range scenarios.Scenarios {
      if err := validate.Struct(s); err != nil {
          return nil, fmt.Errorf("scenario %d (%q): %w", i, s.Name, err)
      }
  }

But for now probably overkill. Will see

[vflaux]

I would prefer to used file embedding, because editing the tests.yaml file does not invalidate the test cache, which then requires to run the test with go test -count=1.
Embedding it invalidates the cache when the file is modified.

Something like this:

package scenarios

import _ "embed"

//go:embed tests.yaml
var Tests []byte

    var testScenarios TestScenarios
	if err := yaml.Unmarshal(scenarios.Tests, &testScenarios); err != nil {
		return nil, err
	}

[ivankatliarchuk]

Make sense. Implemented

[vflaux]

Why not a not implemented error as in RESTConfig()? Or an explicit panic?
It should panic at m.Called() since there is no .On().Return for this method.

[ivankatliarchuk]

At the moment is return nil, fmt.Errorf("RESTConfig: not implemented"). Basically, in real scenario is required, at the moment, we are using moscks directly. I simply have no use case at the moment.

[vflaux]

/lgtm

[ivankatliarchuk]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ivankatliarchuk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [ivankatliarchuk]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment