[Fleet] Deprecate is_default and is_default_fleet_server flags

[barkbay]

This PR:

• Returns a warning in the webhook response if spec.PolicyID is not set:

{
	"kind": "AdmissionReview",
	"apiVersion": "admission.k8s.io/v1beta1",
	"response": {
		"uid": "705ab4f5-6393-11e8-b7cc-42010a800002",
		"allowed": true,
		"status": {
			"metadata": {},
			"code": 200
		},
		"warnings": [
			"spec.PolicyID is empty, spec.PolicyID will become mandatory in a future release"
		]
	}
}

• Removes is_default and is_default_fleet_server from our examples and our documentation, adds spec.PolicyID instead.

TODO:

• Create an event/log if is_default or is_default_fleet_server is set.
• There are still some references in:
  • The Helm Charts
  • E2E tests: I think I will not update E2EFleetPolicies as long as the flags are supported. spec.PolicyID will be tested as part of the updates in config/recipes/elastic-agent, for example with TestFleetKubernetesIntegrationRecipe. TODO: create an issue to track this effort once these 2 flags are actually removed
• For the next major release, should we already skip the default policy mechanism, and make a spec.PolicyID mandatory field for any stack version > 8.x.x ?

Fix #6678

[barkbay]

This is what the warning looks like from a CLI perspective:

[pebrc]

LGTM want to run a few tests still. I like the way you integrated warnings into the webhook tooling 👍

[pebrc]

Do we know it is going away in 9.0.0 or is this only a placeholder? If so let's add a comment.

[barkbay]

Hi @juliaElastic 👋 , could you confirm that is_default and is_default_fleet_server flags should be removed in 9.0.0?

[juliaElastic]

Yes, that is the plan.

[pebrc]

Do we need the timeout given that this is always used in a unit test context and there is no expectation of concurrent use?

	t.Helper()
	if minEventCount == 0 {
		return nil
	}
	gotEvents := make([]string, 0, minEventCount)
	gotEventCount := 0
	for i := 0; i < minEventCount; i++ {
		select {
		case e := <-recorder.Events:
			gotEvents = append(gotEvents, e)
			gotEventCount++
		default:
			t.Errorf("expected at least %d events, got %d", minEventCount, gotEventCount)
		}
	}
	return gotEvents

also the name is could be then something along the lines of ReadNEvents

[barkbay]

Do we need the timeout given that this is always used in a unit test context and there is no expectation of concurrent use?

I think we need a timeout so we eventually exit the function if there are less events than minEventCount. Otherwise, the function may wait forever on case e := <-recorder.Events.

[pebrc]

That's why I added the default clause in the select so that it immediately errors out or maybe I am missing your point.

[barkbay]

Sorry I'm missed your original point. I was assuming that we didn't want to depend on whether or not events were published synchronously in the recorder. But I agree that it is fair to say that it will be the case here.

[pebrc]

LGTM I tried the webhook very nice!