Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_kubernetes_cluster: Support for upgrade path from Azure CNI to Azure CNI Overlay #26260

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

azurerm_kubernetes_cluster: Support for upgrade path from Azure CNI to Azure CNI Overlay #26260

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
a81a846
`azurerm_kubernetes_cluster`: Support for upgrade path from Azure CNI…
jkroepke Jun 6, 2024
92d0b43
Merge branch 'hashicorp:main' into pod-cidr
jkroepke Jun 17, 2024
d43f7cf
Merge branch 'hashicorp:main' into pod-cidr
jkroepke Jun 22, 2024
1fc34b8
Merge branch 'hashicorp:main' into pod-cidr
jkroepke Jul 5, 2024
4513485
Merge branch 'main' into pod-cidr
jkroepke Aug 14, 2024
8185135
Merge branch 'main' into pod-cidr
jkroepke Aug 19, 2024
23036f3
Merge branch 'main' into pod-cidr
jkroepke Aug 24, 2024
58789aa
Merge branch 'main' into pod-cidr
jkroepke Sep 4, 2024
066f275
Merge branch 'main' into pod-cidr
jkroepke Sep 9, 2024
67d4db8
Merge branch 'main' into pod-cidr
jkroepke Sep 15, 2024
904cd30
Merge branch 'main' into pod-cidr
jkroepke Sep 26, 2024
c033c7e
Merge branch 'main' into pod-cidr
jkroepke Sep 30, 2024
5299adf
Merge branch 'main' into pod-cidr
jkroepke Nov 8, 2024
a7011a3
Merge branch 'main' into pod-cidr
jkroepke Dec 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions internal/services/containers/kubernetes_cluster_resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,12 @@ func resourceKubernetesCluster() *pluginsdk.Resource {
pluginsdk.ForceNewIfChange("network_profile.0.network_policy", func(ctx context.Context, old, new, meta interface{}) bool {
return old.(string) != "" || new.(string) != string(managedclusters.NetworkPolicyCilium)
}),
pluginsdk.ForceNewIfChange("network_profile.0.pod_cidr", func(ctx context.Context, old, new, meta interface{}) bool {
return old.(string) != ""
}),
pluginsdk.ForceNewIfChange("network_profile.0.pod_cidrs", func(ctx context.Context, old, new, meta interface{}) bool {
return len(old.([]interface{})) > 0
}),
jkroepke marked this conversation as resolved.
Show resolved Hide resolved
Comment on lines +115 to +120
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this will work for upgrading from Azure CNI to Azure CNI Overlay, it will cause issues for users that are using other networking configurations which we still need to support.

pod_cidr and pod_cidrs are both Optional and Computed because there are instances where the API will assign values to these if they're omitted e.g. when using kubenet, this compounds with the fact that these value change checks performed in the CustomizeDiff compare the old state value and the new state value and not whether the value in the users configuration has changed.

Ultimately this will end up triggering a ForceNew for users that are still using kubenet but haven't specified pod_cidr or pod_cidrs (and potentially other scenarios!).

I would appreciate if you could add some test cases (or extend existing test cases) to ensure that we're not triggering a ForceNew in situations where we shouldn't be, those should also help you to refine the condition needed here to enable the behaviour you're after without introducing unwanted side effects.

pluginsdk.ForceNewIfChange("custom_ca_trust_certificates_base64", func(ctx context.Context, old, new, meta interface{}) bool {
return len(old.([]interface{})) > 0 && len(new.([]interface{})) == 0
}),
Expand Down Expand Up @@ -1031,15 +1037,13 @@ func resourceKubernetesCluster() *pluginsdk.Resource {
Type: pluginsdk.TypeString,
Optional: true,
Computed: true,
ForceNew: true,
ValidateFunc: validate.CIDR,
},

"pod_cidrs": {
Type: pluginsdk.TypeList,
Optional: true,
Computed: true,
ForceNew: true,
Elem: &pluginsdk.Schema{
Type: pluginsdk.TypeString,
ValidateFunc: validation.StringIsNotEmpty,
Expand Down Expand Up @@ -2340,6 +2344,21 @@ func resourceKubernetesClusterUpdate(d *pluginsdk.ResourceData, meta interface{}
existing.Model.Properties.NetworkProfile.NetworkDataplane = pointer.To(managedclusters.NetworkDataplane(ebpfDataPlane))
}

if key := "network_profile.0.network_plugin_mode"; d.HasChange(key) {
networkPluginMode := d.Get(key).(string)
existing.Model.Properties.NetworkProfile.NetworkPluginMode = pointer.To(managedclusters.NetworkPluginMode(networkPluginMode))
}

if key := "network_profile.0.pod_cidr"; d.HasChange(key) {
podCidr := d.Get(key).(string)
existing.Model.Properties.NetworkProfile.PodCidr = pointer.To(podCidr)
}

if key := "network_profile.0.pod_cidrs"; d.HasChange(key) {
podCidrs := d.Get(key).([]interface{})
existing.Model.Properties.NetworkProfile.PodCidrs = utils.ExpandStringSlice(podCidrs)
}

if key := "network_profile.0.outbound_type"; d.HasChange(key) {
outboundType := managedclusters.OutboundType(d.Get(key).(string))
existing.Model.Properties.NetworkProfile.OutboundType = pointer.To(outboundType)
Expand Down
4 changes: 2 additions & 2 deletions website/docs/r/kubernetes_cluster.html.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -699,9 +699,9 @@ A `network_profile` block supports the following:

* `outbound_type` - (Optional) The outbound (egress) routing method which should be used for this Kubernetes Cluster. Possible values are `loadBalancer`, `userDefinedRouting`, `managedNATGateway` and `userAssignedNATGateway`. Defaults to `loadBalancer`. More information on supported migration paths for `outbound_type` can be found in [this documentation](https://learn.microsoft.com/azure/aks/egress-outboundtype#updating-outboundtype-after-cluster-creation).

* `pod_cidr` - (Optional) The CIDR to use for pod IP addresses. This field can only be set when `network_plugin` is set to `kubenet` or `network_plugin_mode` is set to `overlay`. Changing this forces a new resource to be created.
* `pod_cidr` - (Optional) The CIDR to use for pod IP addresses. This field can only be set when `network_plugin` is set to `kubenet` or `network_plugin_mode` is set to `overlay`. Changing an non-empty value forces a new resource to be created.

* `pod_cidrs` - (Optional) A list of CIDRs to use for pod IP addresses. For single-stack networking a single IPv4 CIDR is expected. For dual-stack networking an IPv4 and IPv6 CIDR are expected. Changing this forces a new resource to be created.
* `pod_cidrs` - (Optional) A list of CIDRs to use for pod IP addresses. For single-stack networking a single IPv4 CIDR is expected. For dual-stack networking an IPv4 and IPv6 CIDR are expected. Changing an non-empty value forces a new resource to be created.

* `service_cidr` - (Optional) The Network Range used by the Kubernetes service. Changing this forces a new resource to be created.

Expand Down
Loading