[openstack] Custom dns prefixes and domains

[bogdando]

Support separate public/private DNS records, yet limited to an
external DNS managed elsewhere. Users may benefit from this
feature by sending nsupdates to private and/or public servers
as they want it.

Additional changes:

• Name Heat stack by openshift_openstack_full_public_dns_domain
• Add openshift_openstack_public_dns_domain to the role defaults
  to not rely on the group vars example only.
• Move openshift_master_cluster_public_hostname
  and openshift_master_default_subdomain from the group vars
  example to the role defaults, where the values these take are
  now evaluated.
• Evaluate openshift_master_cluster_hostname based on a private
  domain name as it may be different to the public name now.

Describe configuration scenarios for external public and private
DNS servers managing public and custom private domain records for the
openshift cluster nodes.

Specify manual commands required to deploy DNS service with automation
playbooks for osp 10 on ocp 3.6 ref arch coming from openshift-ansible-contrib.

This improves UX with DNS stack not managed in openshift-ansible.

How to test:

• With defaults, e2e should pass
• Follow the included guide to verify an external public and private DNS server.

[openshift-ci-robot]

Hi @bogdando. Thanks for your PR.

I'm waiting for a openshift member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bogdando]

@tomassedovic PTAL

[rh-atomic-bot]

Can one of the admins verify this patch?
I understand the following commands:

• bot, add author to whitelist
• bot, test pull request
• bot, test pull request once

[bogdando]

e2e with defaults works for me, @tomassedovic
I'll test with a custom external DNS hosting private records as well.

[bogdando]

@tomassedovic @e-minguez PTAL . The provided documentation for external DNS services deployment works for me locally. I was using these custom vars for osp10-dns-service's vars.yaml and openshift cluster's external_dns.yaml.

[tomassedovic]

Thanks! I still see the same issue as in the contrib patch: this should be doing hostname suffixes not zone prefixes.

When I have a DNS server configured with the openshift.example.com zone, then setting:

openshift_openstack_private_hostname_suffix: private

I expect it to create entries with the master-0.private or even master-0private hostname and openhsift.example.com zone.

Instead, it keeps hostname as: master-0 and changes the zone to private.openshift.example.com, resulting in the following failure:

failed: [localhost] (item=({u'key_algorithm': u'hmac-sha256', u'zone': u'private.openshift.example.com', u'key_name': u'private-openshift.example.com', u'server': u'10.40.128.138', u'key_secret': u'l[redacted]', u'view': u'private'}, {'ip': u'192.168.99.13', 'hostname': u'master-0', 'type': u'A'})) => {"attempts": 10, "changed": false, "dns_rc": 9, "dns_rc_str": "NOTAUTH", "failed": true, "item": [{"key_algorithm": "hmac-sha256", "key_name": "private-openshift.example.com", "key_secret": "l[redacted]", "server": "10.40.128.138", "view": "private", "zone": "private.openshift.example.com"}, {"hostname": "master-0", "ip": "192.168.99.13", "type": "A"}], "msg": "Failed to create DNS record (rc: 9)"}

Please update the code to leave the zones alone (or possibly as another option), but update the hostname entries.

[bogdando]

@tomassedovic

openshift_openstack_private_hostname_suffix: private
I expect it to create entries with the master-0.private or even master-0private hostname and openhsift.example.com zone.

Just to clarify, it should put 'master-0.private' entries (the shortname 'master-0' kept intact) living in a subdomain 'private' of the main (unchanged) openshift.example.com. So, FQDN would be master-0.private.openshift.example.com and nodes can resolve master-0.private?

[tomassedovic]

I'm sorry @bogdando, I should have specified this more clearly. I'll try to do better next time.

This should not even treat it as a subdomain. Just a hostname/record suffix.

With openshift_openstack_clusterid: "openshift" and openshift_openstack_public_dns_domain: "example.com":

If we set openshift_openstack_private_hostname_suffix: "-private", the record we set would be master-0-private, the FQDN master-0-private.openshift.example.com and the zone: openshift.example.com.

Here's the nsupdate entry this should generate:

nsupdate:
  record: "master-0-private"
  zone: "openshift.example.com"
...

The nodes resolve the FQDN (just like they do today), so: master-0-private.openshift.example.com.

If we set openshift_openstack_private_hostname_suffix: ".private", the record would be master-0.private (you can have dots in a record without changing the zone), the FQDN master-0.private.openshift.example.com and the zone would still be openshift.example.com.

The resulting nsupdate would look like this:

nsupdate:
  record: "master-0.private"
  zone: "openshift.example.com"
...

But no matter what we put in openshift_openstack_private_hostname_suffix or openshift_openstack_public_hostname_suffix, the zone must not change.

[bogdando]

This should not even treat it as a subdomain. Just a hostname/record suffix.

I'd like to not modify hostnames, yet using subdomains. Updated for that case. See the docs update to test please.

zone must not change

Is it a strict requirement? I'll have to rework once again then...

[tomassedovic]

Yes, the zone must not change. The feature we need is very simple: if you specify a suffix, that string gets concatenated in the record portion of the nsupdate module. That is all. No zone changes, no additional domain management.

And yeah, this shouldn't modify the hostnames. Just the nsupdate records.

[bogdando]

@tomassedovic this illustrates your case, yet using subdomains. I found this simpler to implement and bringing less changes into the code than manipulating the hostnames (in the populate-dns task). I'd prefer to not touch hostnames at all. Let's please keep shortnames intact, if possible. Just thinking of the populate-dns taks complexity, it's overcomplicated allready, let's not make it even more complex :)

On the other hand, we'll end up having a two separate DNS services (if not using multi views) anyway. And to me it seems not very important, which zone will be hosting the 2nd one, used for private names only. One can put whatever he wants for his private domains/subdomains/zones. Those can't be seen from outside, right? I think we should keep this simple and just using this implementation (w/o touching the populate-dns thing)...

[bogdando]

@tomassedovic note, we already have role names suffixes that alter nova servers' names, we need no another altering suffix for this case IMO

[bogdando]

@tomassedovic I think subdomains give more flexibility than just hostname suffixes. You cannot alter private domain fully with the latter it seems.

[bogdando]

Anyway, to make it with hostnames suffixes, let's think of integrating this into openshift_openstack_master_hostname et al as well ... @tomassedovic any ideas?

[tomassedovic]

This is what we need:

https://github.com/tomassedovic/openshift-ansible/commit/c053061525a7e823ebca2a8bdc0ea66e330a4aa9

Plus making sure this propagates to the dynamic inventory. That is all.

We can set the domain already by changing the cluster id and default subdomain. It is possible we might want to add support for two completely separate configurable domains at some point, but that should be a separate change.

[bogdando]

We can set the domain already by changing the cluster id and default subdomain. It is possible we might want to add support for two completely separate configurable domains at some point, but that should be a separate change.

I see. I'll keep this PR for that future case then.

So I got it, it seems all we want is a single public DNS service capable to resolve
foo.openshift.example.com into a public/floating IP, while foo-private.openshift.example.com into a private IP, given a private hostname prefix '-private'. I'll take a look into that, thanks!

[bogdando]

@tomassedovic shall those suffixes be added to openstack_<role name>_hostname, or only the public suffix, if defined?

FWIW, that's my current understanding I'm gonna put into the new PR docs' pages:

If you want to use a two sets of hostnames for public and private/prefixed DNS
records for your externally managed public DNS server, you can specify
`openshift_openstack_public_hostname_suffix` and/or
`openshift_openstack_private_hostname_suffix`. The public suffix will be added
to all of the `openstack_<role name>_hostname`. And the private suffix will go
for the default `<role name>`. Those are empty by default.

**Note** the servers' hostnames will not be updated. The deployment may be done on
arbitrary named hosts with the hostnames managed by cloud-init. Only inventory
hostnames will be altered to match the public FQDNs with added suffixes.

please confirm if I got that right! :)

The most concerning part is inventory hostnames. Is it enough for the change, or other vars like openshift_(public)_hostname and openshift_master_cluster_(public)_hostname shall carry on those suffixes as well?..

Sigh. That becomes too complicated

[bogdando]

So we decided to stick with the nsupdates only suffixes #6381 .
Abandoning this for now.