diff --git a/Documentation/dev/node-bootstrap-flow.md b/Documentation/dev/node-bootstrap-flow.md
index aa364f7a2a..4500ddeaa4 100644
--- a/Documentation/dev/node-bootstrap-flow.md
+++ b/Documentation/dev/node-bootstrap-flow.md
@@ -7,32 +7,26 @@ This is a development document which describes the bootstrapping flow for Contai
When a cluster node is being bootstrapped from scratch, it goes through several phases in the following order:
1. first-boot OS configuration, via ignition (systemd units, node configuration, etc)
-1. provisioning of additional assets (k8s manifests, TLS material), via either of:
+2. provisioning of additional assets (k8s manifests, TLS material), via either of:
* pushing from terraform file/remote-exec (SSH)
* pulling from private cloud stores (S3 buckets)
-1. system-wide updates via `k8s-node-bootstrap.service`, which includes:
- * determining current kubernetes cluster version (when joining an existing cluster)
- * triggering a ContainerLinux update, via update-engine (optional)
- * downloading and deploying proper docker addon version, via tectonic-torcx
- * writing the `kubelet.env` file
-1. if needed, a node reboot is triggered to apply systemd-wide changes and to clean container runtime datadir
-1. `kubelet.service` picks up the `kubelet.env` file and actually starts the kubelet as a rkt-fly service.
+3. if needed, a node reboot is triggered to apply systemd-wide changes and to clean container runtime datadir
+4. `kubelet.service` picks up the `kubelet.env` file and actually starts the kubelet as a rkt-fly service.
Additionally, only on one of the master nodes the following kubernetes bootstrapping happens:
1. `bootkube.service` is started after `kubelet.service` start
-1. a static bootstrapping control-plane is deployed
-1. a fully self-hosted control-plane starts and takes over the previous one
-1. `bootkube.service` is completed with success
-1. `tectonic.service` is started
-1. a self-hosted tectonic control-plane is deployed
-1. `tectonic.service` is completed with success
+2. a static bootstrapping control-plane is deployed
+3. a fully self-hosted control-plane starts and takes over the previous one
+4. `bootkube.service` is completed with success
+5. `tectonic.service` is started
+6. a self-hosted tectonic control-plane is deployed
+7. `tectonic.service` is completed with success
## Systemd units
-The following systemd units are deployed to a node by tectonic-installer and take part in the bootstrapping process:
+The following systemd unit is deployed to a node by tectonic-installer and take part in the bootstrapping process:
-* `k8s-node-bootstrap.service` ensures node and assets freshness. It is automatically started on boot, can crash-loop, and it runs only during bootstrap
* `kubelet.service` is the main kubelet daemon. It is automatically started on boot, it is crash-looping until `kubelet.env` is populated, and it runs on each boot
Additionally, only on one of the master nodes the following kubernetes bootstrapping happens:
@@ -42,35 +36,12 @@ Additionally, only on one of the master nodes the following kubernetes bootstrap
* `tectonic.service` deploys tectonic control-plane. It is started only after `bootkube.service` _has completed_. It is a oneshot unit and cannot crash, and it runs only during bootstrap
* `bootkube.path` waits for tectonic assets/scripts to exist on disk and triggers `tectonic.service`
-`k8s-node-bootstrap` runs [tectonic-torcx][tectonic-torcx] as a containerized service, thus relying on a container runtime being already on the node.
-It currently assumes that Docker is available and working. In case of version changes, a cleanup of the Docker datadir `/var/lib/docker` is scheduled before rebooting.
-
[tectonic-torcx]: https://github.com/coreos/tectonic-torcx
## Service ordering
Service ordering is enforced via systemd dependencies. This is the rationale for the settings, with relevant snippets:
-### `k8s-node-bootstrap.service`
-
-```
-ConditionPathExists=!/etc/kubernetes/kubelet.env
-Before=kubelet.service
-Restart=on-failure
-ExecStartPre=[...]
-ExecStart=/usr/bin/echo "node components bootstrapped"
-WantedBy=multi-user.target kubelet.service
-```
-
-This service is enabled by default and can crash-loop until success.
-Main logic happens in `Pre`, before the unit is marked as started, to block further services (a synchronous reboot can happen here).
-
-In particular, this blocks kubelet from starting by:
- * a `WantedBy=` and `Before=`
- * writing the actual `kubelet.env` file on success.
-
-It is skipped on further boots, as the condition-path exists.
-
### `kubelet.service`
```
@@ -81,7 +52,6 @@ WantedBy=multi-user.target
```
This service is enabled by default and can crash-loop until success.
-On first boot, it is initially blocked by `k8s-node-bootstrap.service`.
It crash-loop until the `kubelet.env` file exists.
It is started on every boot.
@@ -135,7 +105,6 @@ This is a visual simplified representation of the overall bootstrapping flow.
Legend:
* TF -> terraform provisioner
* IGN -> ignition
- * knb.s -> k8s-node-bootstrap.service
* k.s -> kubelet.service
* b.p -> bootkube.path
* b.s -> bootkube.service
@@ -144,28 +113,28 @@ Legend:
* rm.p -> rm-assets.path
* rm.s -> rm-assets.service
-.---------------------------------------------------------------------------------------------------------------------------------------+
-| |
-| Provision cloud/userdata +----------+ Provision files |
-| ,----------------------------------------------o| TF |o-----------------.------------------------.-----------------+ |
-| | +----------+ | | | |
-| | v v v |
-| | +----------+ +-----+ +-------+ +------+ |
-| | .--->| (reboot) |----. | b.p | | t.p | | rm.p | |
-| | | +----------+ | +-----+ +-------+ +------+ |
-| V | | o o o |
-| +-------+ | v Before +------------+ Before | Trigger Trigger | Trigger | |
-| | IGN | | *---------->| k.s |o--------. | | | |
-| +-------+ o ^ +------------+ | v v v |
-| | +----------+ | ^ | | +-----+ Before +-------+ Before +-----+ |
-| '------>| knb.s |o--------------' | v '--->| b.s |o--------------->| t.s |--------> |rm.s | |
-| Enable +----------+ '------' +-----+ +-------+ +-----+ |
-| ^ | |
-| | v |
-| '----' o o |
-| | | |
-| * First boot | * Each boot | * First boot |
-| * All nodes | * All nodes | * Bootkube master |
-| | | |
-'----------------------------------------------o----------------------------o-----------------------------------------------------------+
+.--------------------------------------------------------------------------------------------------------------------------------+
+| |
+| Provision cloud/userdata +----------+ Provision files |
+| ,---------------------------------------o| TF |o-----------------.------------------------.-----------------+ |
+| | +----------+ | | | |
+| | v v v |
+| | +-----+ +-------+ +------+ |
+| | | b.p | | t.p | | rm.p | |
+| | +-----+ +-------+ +------+ |
+| V o o o |
+| +-------+ Before +------------+ Before | Trigger Trigger | Trigger | |
+| | IGN | .--------------->| k.s |o--------. | | | |
+| +-------+ | +------------+ | v v v |
+| | | ^ | | +-----+ Before +-------+ Before +-----+ |
+| '----------------------' | v '--->| b.s |o--------------->| t.s |--------> |rm.s | |
+| Enable '------' +-----+ +-------+ +-----+ |
+| |
+| |
+| o o |
+| | | |
+| | * Each boot | * First boot |
+| | * All nodes | * Bootkube master |
+| | | |
+'---------------------------------------o----------------------------o-----------------------------------------------------------+
```
diff --git a/Documentation/variables/config.md b/Documentation/variables/config.md
index 96a8223931..e96d3bdfdf 100644
--- a/Documentation/variables/config.md
+++ b/Documentation/variables/config.md
@@ -9,7 +9,6 @@ This document gives an overview of variables used in all platforms of the Tecton
| tectonic_admin_email | (internal) The e-mail address used to: 1. login as the admin user to the Tectonic Console. 2. generate DNS zones for some providers.
Note: This field MUST be in all lower-case e-mail address format and set manually prior to creating the cluster. | string | - |
| tectonic_admin_password | (internal) The admin user password to login to the Tectonic Console.
Note: This field MUST be set manually prior to creating the cluster. Backslashes and double quotes must also be escaped. | string | - |
| tectonic_base_domain | The base DNS domain of the cluster. It must NOT contain a trailing period. Some DNS providers will automatically add this if necessary.
Example: `openstack.dev.coreos.systems`.
Note: This field MUST be set manually prior to creating the cluster. This applies only to cloud platforms.
[Azure-specific NOTE] To use Azure-provided DNS, `tectonic_base_domain` should be set to `""` If using DNS records, ensure that `tectonic_base_domain` is set to a properly configured external DNS zone. Instructions for configuring delegated domains for Azure DNS can be found here: https://docs.microsoft.com/en-us/azure/dns/dns-delegate-domain-azure-dns | string | - |
-| tectonic_bootstrap_upgrade_cl | (internal) Whether to trigger a ContainerLinux upgrade on node bootstrap. | string | `true` |
| tectonic_ca_cert | (optional) The content of the PEM-encoded CA certificate, used to generate Tectonic Console's server certificate. If left blank, a CA certificate will be automatically generated. | string | `` |
| tectonic_ca_key | (optional) The content of the PEM-encoded CA key, used to generate Tectonic Console's server certificate. This field is mandatory if `tectonic_ca_cert` is set. | string | `` |
| tectonic_ca_key_alg | (optional) The algorithm used to generate tectonic_ca_key. The default value is currently recommended. This field is mandatory if `tectonic_ca_cert` is set. | string | `RSA` |
diff --git a/config.tf b/config.tf
index 872b38ee0f..96587038c7 100644
--- a/config.tf
+++ b/config.tf
@@ -323,12 +323,6 @@ variable "tectonic_networking" {
EOF
}
-variable "tectonic_bootstrap_upgrade_cl" {
- type = "string"
- default = "true"
- description = "(internal) Whether to trigger a ContainerLinux upgrade on node bootstrap."
-}
-
variable "tectonic_kubelet_debug_config" {
type = "string"
default = ""
diff --git a/modules/ignition/assets.tf b/modules/ignition/assets.tf
index 008d4eba66..71cea0a88f 100644
--- a/modules/ignition/assets.tf
+++ b/modules/ignition/assets.tf
@@ -49,23 +49,6 @@ data "ignition_systemd_unit" "kubelet" {
content = "${data.template_file.kubelet.rendered}"
}
-data "template_file" "k8s_node_bootstrap" {
- template = "${file("${path.module}/resources/services/k8s-node-bootstrap.service")}"
-
- vars {
- bootstrap_upgrade_cl = "${var.bootstrap_upgrade_cl}"
- tectonic_torcx_image = "${var.container_images["tectonic_torcx"]}"
- torcx_skip_setup = "false"
- torcx_store_url = "${var.torcx_store_url}"
- }
-}
-
-data "ignition_systemd_unit" "k8s_node_bootstrap" {
- name = "k8s-node-bootstrap.service"
- enabled = true
- content = "${data.template_file.k8s_node_bootstrap.rendered}"
-}
-
data "ignition_systemd_unit" "rm_assets" {
name = "rm-assets.service"
enabled = true
diff --git a/modules/ignition/outputs.import b/modules/ignition/outputs.import
index 7bd99a6f17..c50348b840 100644
--- a/modules/ignition/outputs.import
+++ b/modules/ignition/outputs.import
@@ -24,10 +24,6 @@ variable "ign_installer_kubelet_env_id" {
type = "string"
}
-variable "ign_k8s_node_bootstrap_service_id" {
- type = "string"
-}
-
variable "ign_update_ca_certificates_dropin_id" {
type = "string"
}
diff --git a/modules/ignition/outputs.tf b/modules/ignition/outputs.tf
index a74a032442..3a48a5a611 100644
--- a/modules/ignition/outputs.tf
+++ b/modules/ignition/outputs.tf
@@ -32,7 +32,6 @@ output "ignition_systemd_id_list" {
"${data.ignition_systemd_unit.docker_dropin.id}",
"${data.ignition_systemd_unit.kubelet.id}",
"${data.ignition_systemd_unit.locksmithd.id}",
- "${data.ignition_systemd_unit.k8s_node_bootstrap.id}",
"${data.ignition_systemd_unit.update_ca_certificates_dropin.id}",
"${data.ignition_systemd_unit.rm_assets.id}",
]
diff --git a/modules/ignition/resources/services/k8s-node-bootstrap.service b/modules/ignition/resources/services/k8s-node-bootstrap.service
deleted file mode 100644
index 84c13e757b..0000000000
--- a/modules/ignition/resources/services/k8s-node-bootstrap.service
+++ /dev/null
@@ -1,43 +0,0 @@
-[Unit]
-Description=Bootstrap Kubernetes Node Components
-ConditionPathExists=!/etc/kubernetes/kubelet.env
-Wants=update-ca-certificates.service
-After=update-ca-certificates.service
-Before=kubelet.service
-
-[Service]
-Type=simple
-RemainAfterExit=true
-Restart=on-failure
-RestartSec=10
-TimeoutStartSec=1h
-ExecStartPre=/usr/bin/mkdir -p /etc/kubernetes
-ExecStartPre=/usr/bin/docker run --rm \
- --env http_proxy \
- --env HTTP_PROXY \
- --env https_proxy \
- --env HTTPS_PROXY \
- --env no_proxy \
- --env NO_PROXY \
- --tmpfs /tmp \
- -v /usr/share:/usr/share:ro \
- -v /usr/lib/os-release:/usr/lib/os-release:ro \
- -v /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro \
- -v /var/lib/torcx:/var/lib/torcx \
- -v /var/run/dbus:/var/run/dbus \
- -v /run/metadata:/run/metadata:ro \
- -v /run/torcx:/run/torcx:ro \
- -v /run/systemd:/run/systemd \
- -v /etc/coreos:/etc/coreos:ro \
- -v /etc/torcx:/etc/torcx \
- -v /etc/kubernetes:/etc/kubernetes \
- ${tectonic_torcx_image} \
- /tectonic-torcx-bootstrap \
- --upgrade-os=${bootstrap_upgrade_cl} \
- --torcx-manifest-url="${torcx_store_url}" \
- --torcx-skip-setup=${torcx_skip_setup} \
- --verbose=debug
-ExecStart=/usr/bin/echo "node components bootstrapped"
-
-[Install]
-WantedBy=multi-user.target kubelet.service
diff --git a/modules/ignition/variables.tf b/modules/ignition/variables.tf
index 174d1254c3..ef29ce0134 100644
--- a/modules/ignition/variables.tf
+++ b/modules/ignition/variables.tf
@@ -45,12 +45,6 @@ variable "cloud_provider_config" {
default = ""
}
-variable "bootstrap_upgrade_cl" {
- type = "string"
- description = "(optional) Whether to trigger a ContainerLinux OS upgrade during the bootstrap process."
- default = "true"
-}
-
variable "torcx_store_url" {
type = "string"
description = "(optional) URL template for torcx store. Leave empty to use the default CoreOS endpoint."
diff --git a/steps/assets/aws/main.tf b/steps/assets/aws/main.tf
index b9104b6625..5e64dde133 100644
--- a/steps/assets/aws/main.tf
+++ b/steps/assets/aws/main.tf
@@ -21,7 +21,6 @@ module assets_base {
ingress_kind = "haproxy-router"
tectonic_base_domain = "${var.tectonic_base_domain}"
- tectonic_bootstrap_upgrade_cl = "${var.tectonic_bootstrap_upgrade_cl}"
tectonic_cluster_name = "${var.tectonic_cluster_name}"
tectonic_container_images = "${var.tectonic_container_images}"
tectonic_image_re = "${var.tectonic_image_re}"
diff --git a/steps/assets/base/ignition-bootstrap.tf b/steps/assets/base/ignition-bootstrap.tf
index 276f2bb4df..68676b7612 100644
--- a/steps/assets/base/ignition-bootstrap.tf
+++ b/steps/assets/base/ignition-bootstrap.tf
@@ -1,7 +1,6 @@
module "ignition_bootstrap" {
source = "../../../modules/ignition"
- bootstrap_upgrade_cl = "${var.tectonic_bootstrap_upgrade_cl}"
cloud_provider = "${var.cloud_provider}"
container_images = "${var.tectonic_container_images}"
etcd_ca_cert_pem = "${local.etcd_ca_cert_pem}"
diff --git a/steps/assets/libvirt/main.tf b/steps/assets/libvirt/main.tf
index 4447b62f95..c267f23974 100644
--- a/steps/assets/libvirt/main.tf
+++ b/steps/assets/libvirt/main.tf
@@ -9,7 +9,6 @@ module assets_base {
ingress_kind = "haproxy-router"
tectonic_base_domain = "${var.tectonic_base_domain}"
- tectonic_bootstrap_upgrade_cl = "${var.tectonic_bootstrap_upgrade_cl}"
tectonic_cluster_name = "${var.tectonic_cluster_name}"
tectonic_container_images = "${var.tectonic_container_images}"
tectonic_image_re = "${var.tectonic_image_re}"