Add an entry to `/etc/hosts` for the current hostname #1713

zmrow · 2021-08-18T22:12:28Z

Issue number:
Fixes #1700

Description of changes:

    schnauzer: Add a new template helper: `add_unresolvable_hostname`
    
    This change adds a new template helper `add_unresolvable_hostname` that
    will attempt to resolve the hostname given to it in the template (from
    `settings.network.hostname`).  If the hostname is unresolvable or
    resolves to localhost (127.0.0.1 or ::1), the helper adds an alias to
    the hostname for both the IPv4 and IPv6 addresses

    Add a hostname entry to `/etc/hosts` if hostname is unresolvable
    
    Previously, `/etc/hosts` was a static file (in the `release` package)
    that we copied into the image when building Bottlerocket.  In order to
    have an entry in the file corresponding to hostname, a new "hosts"
    service was added to the affected services for
    `settings.network.hostname`.  This new "hosts" service has a templated
    configuration file that uses `settings.network.hostname` to populate the
    entry in `/etc/hosts` if the current hostname is unresolvable in DNS.
    The templated configuration file uses the previously added
    `add_unresolvable_hostname` template renderer helper.

    migrations: Add migrations for `/etc/hosts` "service"
    
    This change includes the 2 migrations necessary to make `/etc/hosts` an
    "affected service" of `settings.network.hostname`: one to add the
    service and configuration file for the hosts service, and another to
    add the hosts service to `settings.network.hostname`.

Testing done:
Boot an aws-k8s-1.19 host and observe proper behavior, run a pod, etc. /etc/hosts looks like:

bash-5.0# cat /etc/hosts 
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

Upgrade the same host to a version including these changes, observe proper boot and behavior. /etc/hosts looks the same because the hostname resolves in EC2:

bash-5.0# cat /etc/hosts 
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4l
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

I followed the same procedure with a VMware variant. /etc/hosts before the upgrade:

bash-5.0# cat /etc/hosts 
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

/etc/hosts after the upgrade (hosts-test is the hostname of my VM):

bash-5.0# cat /etc/hosts 
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 hosts-test
::1 hosts-test

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

bcressey · 2021-08-19T00:49:29Z

I'd prefer that we scope this more narrowly so that we only add the hosts entry if our configured hostname isn't resolvable through DNS.

That's really the only case where we know it's needed and helpful.

zmrow · 2021-08-23T21:51:16Z

^ Addresses @bcressey 's concern and adds a new template renderer helper that will only add the entry to /etc/hosts if the hostname isn't resolvable or resolves to localhost.

zmrow · 2021-08-23T21:52:01Z

^ Splits the template helper into its own commit

zmrow · 2021-08-23T21:55:25Z

sources/api/schnauzer/src/helpers.rs

@@ -935,8 +1022,9 @@ fn pause_registry<S: AsRef<str>>(region: S) -> String {
 }

 /// Calculates and returns the amount of CPU to reserve
-fn kube_cpu_helper(num_cores: usize) -> Result<String, TemplateHelperError>{
-    let num_cores = u16::try_from(num_cores).context(error::ConvertUsizeToU16 { number: num_cores })?;
+fn kube_cpu_helper(num_cores: usize) -> Result<String, TemplateHelperError> {


This was a rustfmt change

zmrow · 2021-08-23T21:55:41Z

sources/api/schnauzer/src/helpers.rs

@@ -949,7 +1037,11 @@ fn kube_cpu_helper(num_cores: usize) -> Result<String, TemplateHelperError>{
            KUBE_RESERVE_4_CORES + ((num_cores - 4.0) * KUBE_RESERVE_ADDITIONAL)
        }
    };
-    Ok(format!("{}{}", cpu_to_reserve.floor().to_string(), millicores_unit))
+    Ok(format!(


Same here - thanks rustfmt

zmrow · 2021-08-23T22:25:34Z

^ rebased on develop

webern

Don't we need to add something to README for the new settings. Possibly a warning in k8s docs about kubelet not liking a hostname change?

sources/Cargo.toml

webern · 2021-08-25T15:42:37Z

sources/api/schnauzer/src/helpers.rs

+    fn resolves_to_localhost_renders_entries() {
+        let result = setup_and_render_template(
+            "{{add_unresolvable_hostname name}}",
+            &json!({"name": "localhost"}),


I guess this test should always work on every system?

I believe it should? I'm willing to be told otherwise though.

webern · 2021-08-25T15:43:48Z

sources/api/schnauzer/src/helpers.rs

+    fn resolvable_hostname_renders_nothing() {
+        let result = setup_and_render_template(
+            "{{add_unresolvable_hostname name}}",
+            &json!({"name": "amazon.com"}),


Why is this unresolvable by DNS? Will it always be unresolvable? Would something completely invalid be better? not*a*valid*hostname?

The helper only renders if the name is unresolvable or resolves to 127.0.0.1 or ::1. I realize that amazon.com isn't a realistic hostname, but it should resolve mostly anywhere and that's the behavior I was after.

sources/api/schnauzer/src/lib.rs

zmrow · 2021-08-25T18:11:55Z

^ Added the comments @webern asked for

sources/api/schnauzer/src/lib.rs

zmrow · 2021-08-30T17:01:24Z

^ Fixed my misspelling. :)

packages/release/release.spec

sources/api/schnauzer/src/helpers.rs

zmrow · 2021-09-01T18:26:42Z

^ Addressed @bcressey 's comments

bcressey · 2021-09-02T02:17:37Z

sources/api/schnauzer/src/helpers.rs

+        #[snafu(display("Invalid IP address '{}': {}", ip, source))]
+        IpFromString {
+            ip: String,
+            source: std::net::AddrParseError,
+        },
+


I don't think this error is used any more.

This change adds a new template helper `add_unresolvable_hostname` that will attempt to resolve the hostname given to it in the template (from `settings.network.hostname`). If the hostname is unresolvable or resolves to localhost (127.0.0.1 or ::1), the helper adds an alias to the hostname for both the IPv4 and IPv6 addresses.

Previously, `/etc/hosts` was a static file (in the `release` package) that we copied into the image when building Bottlerocket. In order to have an entry in the file corresponding to hostname, a new "hosts" service was added to the affected services for `settings.network.hostname`. This new "hosts" service has a templated configuration file that uses `settings.network.hostname` to populate the entry in `/etc/hosts` if the current hostname is unresolvable in DNS. The templated configuration file uses the previously added `add_unresolvable_hostname` template renderer helper.

This change includes the 2 migrations necessary to make `/etc/hosts` an "affected service" of `settings.network.hostname`: one to add the service and configuration file for the hosts "service", and another to add the hosts service to `settings.network.hostname`.

zmrow · 2021-09-02T15:08:42Z

^ removed the unused error variant

The /etc/hosts file was removed and replaced with a template in a prior change (bottlerocket-os#1713). This removes the entry from `release-tmpfiles.conf`.

zmrow requested review from bcressey, tjkirch and cbgbt August 18, 2021 22:13

zmrow force-pushed the etc-hosts-template branch from 0782870 to 4b7a0e3 Compare August 23, 2021 21:32

zmrow force-pushed the etc-hosts-template branch from 4b7a0e3 to e1c7834 Compare August 23, 2021 21:51

zmrow commented Aug 23, 2021

View reviewed changes

zmrow force-pushed the etc-hosts-template branch from e1c7834 to 7a69c22 Compare August 23, 2021 22:24

webern approved these changes Aug 25, 2021

View reviewed changes

zmrow force-pushed the etc-hosts-template branch from 7a69c22 to 6e14a5d Compare August 25, 2021 18:11

jpculp approved these changes Aug 30, 2021

View reviewed changes

webern approved these changes Aug 30, 2021

View reviewed changes

sources/api/schnauzer/src/lib.rs Outdated Show resolved Hide resolved

zmrow force-pushed the etc-hosts-template branch from 6e14a5d to bcd0591 Compare August 30, 2021 17:00

bcressey reviewed Sep 1, 2021

View reviewed changes

packages/release/release.spec Outdated Show resolved Hide resolved

sources/api/schnauzer/src/helpers.rs Outdated Show resolved Hide resolved

sources/api/schnauzer/src/helpers.rs Show resolved Hide resolved

zmrow force-pushed the etc-hosts-template branch from bcd0591 to e7194b4 Compare September 1, 2021 18:26

bcressey reviewed Sep 2, 2021

View reviewed changes

zmrow added 3 commits September 2, 2021 15:06

zmrow force-pushed the etc-hosts-template branch from e7194b4 to 1c25ade Compare September 2, 2021 15:08

bcressey approved these changes Sep 2, 2021

View reviewed changes

zmrow merged commit 084fc3b into bottlerocket-os:develop Sep 9, 2021

zmrow deleted the etc-hosts-template branch September 9, 2021 15:45

tjkirch mentioned this pull request Sep 13, 2021

Update release-tmpfiles for new /etc/hosts source #1744

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add an entry to `/etc/hosts` for the current hostname #1713

Add an entry to `/etc/hosts` for the current hostname #1713

zmrow commented Aug 18, 2021 •

edited

Loading

bcressey commented Aug 19, 2021

zmrow commented Aug 23, 2021

zmrow commented Aug 23, 2021

zmrow Aug 23, 2021

zmrow Aug 23, 2021

zmrow commented Aug 23, 2021

webern left a comment

webern Aug 25, 2021

zmrow Aug 25, 2021

webern Aug 25, 2021

zmrow Aug 25, 2021 •

edited

Loading

zmrow commented Aug 25, 2021

zmrow commented Aug 30, 2021

zmrow commented Sep 1, 2021

bcressey Sep 2, 2021

zmrow commented Sep 2, 2021

Add an entry to /etc/hosts for the current hostname #1713

Add an entry to /etc/hosts for the current hostname #1713

Conversation

zmrow commented Aug 18, 2021 • edited Loading

bcressey commented Aug 19, 2021

zmrow commented Aug 23, 2021

zmrow commented Aug 23, 2021

zmrow Aug 23, 2021

Choose a reason for hiding this comment

zmrow Aug 23, 2021

Choose a reason for hiding this comment

zmrow commented Aug 23, 2021

webern left a comment

Choose a reason for hiding this comment

webern Aug 25, 2021

Choose a reason for hiding this comment

zmrow Aug 25, 2021

Choose a reason for hiding this comment

webern Aug 25, 2021

Choose a reason for hiding this comment

zmrow Aug 25, 2021 • edited Loading

Choose a reason for hiding this comment

zmrow commented Aug 25, 2021

zmrow commented Aug 30, 2021

zmrow commented Sep 1, 2021

bcressey Sep 2, 2021

Choose a reason for hiding this comment

zmrow commented Sep 2, 2021

Add an entry to `/etc/hosts` for the current hostname #1713

Add an entry to `/etc/hosts` for the current hostname #1713

zmrow commented Aug 18, 2021 •

edited

Loading

zmrow Aug 25, 2021 •

edited

Loading