platform/gcp: improve tests support #2178

enxebre · 2017-10-18T15:41:25Z

*.tectonic.gcp.dev.coreos.system needs to be set to point to gcp NS

coreosbot · 2017-10-18T15:41:26Z

Can one of the admins verify this patch?

coreosbot · 2017-10-18T15:41:26Z

Can one of the admins verify this patch?

cpanato · 2017-10-19T15:17:23Z

tests/rspec/lib/container_linux.rb

 require 'ssh'

-SSH_CMD_CONTAINER_LINUX_VERSION = 'sudo cat /var/lib/update_engine/prefs/aleph-version'
+SSH_CMD_CONTAINER_LINUX_VERSION = 'sudo cat /etc/os-release | grep VERSION_ID= | cut -d = -f 2'


This change here works for other platforms?

From my understanding Container Linux should be shipping with this identification file, which is expected by systemd so should be fine in any platform. This makes the test deterministic as otherwise the test might be run before update-engine generates the aleph-version file

This is not correct. This change would revert a fix that was merged in #2093. /etc/os-release contains the current version when we are actually interested in the originally installed version of CL. This change would not make the test deterministic: what if this SSH command was executed on the node after it rebooted? This is actually what was consistently happening and causing the tests to fail. In any case, a possible change would be to:

if [ -f /var/lib/update_engine/prefs/aleph-version ] ; then sudo cat /var/lib/update_engine/prefs/aleph-version else source /usr/share/coreos/release && echo "$COREOS_RELEASE_VERSION" fi

"/etc/os-release contains the current version when we are actually interested in the originally installed version of CL" thanks for spotting that @squat

will rebase with #2193

2193 is merged please rebase and lets get this merged as well

cpanato · 2017-10-19T15:18:27Z

adding smoke tests to run in azure and aws because saw some changes that might affect other platforms

cpanato · 2017-10-19T15:18:32Z

ok to test

cpanato · 2017-10-19T15:42:26Z

@enxebre got this error:

Using build directory [/home/ubuntu/workspace/tectonic-installer_PR-2178-DVVSYYJUWI2MHJUVIOQLQ7UWE3GSBNYMLXZMI5DXRB3XO73EAL4Q/build/demo]
terraform-docs --no-required markdown  config.tf    >>  Documentation/variables/config.md
terraform-docs --no-required markdown  platforms/aws/variables.tf    >>  Documentation/variables/aws.md
terraform-docs --no-required markdown  platforms/azure/variables.tf    >>  Documentation/variables/azure.md
terraform-docs --no-required markdown  platforms/openstack/neutron/variables.tf    >>  Documentation/variables/openstack-neutron.md
terraform-docs --no-required markdown  platforms/metal/variables.tf    >>  Documentation/variables/metal.md
terraform-docs --no-required markdown  platforms/vmware/variables.tf    >>  Documentation/variables/vmware.md
terraform-docs --no-required markdown  platforms/gcp/variables.tf    >>  Documentation/variables/gcp.md
make[1]: Leaving directory '/home/ubuntu/workspace/tectonic-installer_PR-2178-DVVSYYJUWI2MHJUVIOQLQ7UWE3GSBNYMLXZMI5DXRB3XO73EAL4Q'
diff --git a/Documentation/variables/gcp.md b/Documentation/variables/gcp.md
index 357ca05d..9148d056 100644
--- a/Documentation/variables/gcp.md
+++ b/Documentation/variables/gcp.md
@@ -15,6 +15,7 @@ This document gives an overview of variables used in the Google Cloud platform o
 | tectonic_gcp_master_disktype | The type of disk (pd-standard or pd-ssd) for the master nodes. | string | `pd-standard` |
 | tectonic_gcp_master_gce_type | Instance size for the master node(s). Example: `n1-standard-2`. | string | `n1-standard-2` |
 | tectonic_gcp_region | The GCP region to use. Some regions only have 2 zones. | string | - |
+| tectonic_gcp_ssh_key | (required) Path to an SSH public key file to be provisioned as the SSH key for the 'core' user. | string | - |
 | tectonic_gcp_worker_disk_size | The size of the disk in gigabytes for the root block device of worker nodes. | string | `30` |
 | tectonic_gcp_worker_disktype | The type of disk (pd-standard or pd-ssd) for the worker nodes. | string | `pd-standard` |
 | tectonic_gcp_worker_gce_type | Instance size for the worker node(s). Example: `n1-standard-2`. | string | `n1-standard-2` |
outdated docs (run 'make docs' to fix)
Makefile:139: recipe for target 'structure-check' failed
make: *** [structure-check] Error 1

enxebre · 2017-10-19T15:47:29Z

@cpanato thanks for having a look, variables issue should be fixed now.

squat · 2017-10-20T10:22:34Z

Makefile

 	-e GOOGLE_CLOUD_KEYFILE_JSON \
 	-e GCLOUD_KEYFILE_JSON \
+	-e GOOGLE_PROJECT \
+	-e tectonic_gcp_ssh_key \


This needs to be TF_VAR_tectonic_gcp_ssh_key

squat · 2017-10-20T10:25:24Z

modules/gcp/network/loadbalancer.tf


+resource "google_compute_forwarding_rule" "tectonic-api-external-ssh-fwd-rule" {
+  load_balancing_scheme = "EXTERNAL"
+  name                  = "tectonic-api-external-ssh-fwd-rule"


please remove tectonic- from the name for consistency with other platforms, e.g. https://github.com/coreos/tectonic-installer/blob/master/modules/aws/master-asg/master.tf#L24

Will rebase an change this after #2167 gets in

squat · 2017-10-20T10:25:35Z

modules/gcp/network/loadbalancer.tf

@@ -1,5 +1,6 @@
 resource "google_compute_target_pool" "tectonic-master-targetpool" {
-  name = "tectonic-master-targetpool"
+  name             = "tectonic-master-targetpool"


same as below

squat · 2017-10-20T10:27:57Z

tests/rspec/lib/container_linux.rb

 require 'ssh'

-SSH_CMD_CONTAINER_LINUX_VERSION = 'sudo cat /var/lib/update_engine/prefs/aleph-version'
+SSH_CMD_CONTAINER_LINUX_VERSION = 'sudo cat /etc/os-release | grep VERSION_ID= | cut -d = -f 2'


This is not correct. This change would revert a fix that was merged in #2093. /etc/os-release contains the current version when we are actually interested in the originally installed version of CL. This change would not make the test deterministic: what if this SSH command was executed on the node after it rebooted? This is actually what was consistently happening and causing the tests to fail. In any case, a possible change would be to:

if [ -f /var/lib/update_engine/prefs/aleph-version ] ; then sudo cat /var/lib/update_engine/prefs/aleph-version else source /usr/share/coreos/release && echo "$COREOS_RELEASE_VERSION" fi

enxebre · 2017-10-20T17:22:58Z

Rebased, should be good now

squat · 2017-10-20T18:12:53Z

ok to test

squat · 2017-10-20T18:15:28Z

@cpanato, I saw this on the AWS VPC test:


Destroy complete! Resources: 24 destroyed.

An error occurred in an `after(:context)` hook.
Failure/Error: FileUtils.cp '/etc/resolv.conf.bak', '/etc/resolv.conf'

Errno::ENOENT:
  No such file or directory @ rb_sysopen - /etc/resolv.conf.bak
# ./lib/aws_vpc.rb:115:in `recover_etc_resolv'
# ./lib/aws_vpc.rb:85:in `ensure in destroy'
# ./lib/aws_vpc.rb:85:in `destroy'
# ./spec/aws_vpc_internal_spec.rb:25:in `block (2 levels) in <top (required)>'
# ------------------
# --- Caused by: ---
# NoMethodError:
#   undefined method `stop' for nil:NilClass
#   ./lib/aws_vpc.rb:80:in `destroy'

squat · 2017-10-20T18:49:42Z

@enxebre looking good but we need to rebase now that the resource rename pr merged.

enxebre · 2017-10-20T19:40:19Z

@squat thanks, just rebased it

enxebre · 2017-10-20T20:13:55Z

With the latest rebase I'm seeing this happening occasionally when running the tests on gcp:

     Failure/Error: driver = Selenium::WebDriver.for :chrome, options: options
     
     Selenium::WebDriver::Error::WebDriverError:
       unable to connect to chromedriver 127.0.0.1:9515

haven't dug into it yet

enxebre · 2017-10-24T18:13:18Z

Test are consistently working on gcp now with this PR, the two failing seems unrelated? From my perspective this one should be good to go cc @squat @cpanato

squat · 2017-10-25T06:54:21Z

Yes does seem unrelated. I’d like to see if recent changes to master fix these flakes

squat · 2017-10-25T06:55:58Z

ok to test

cpanato · 2017-10-25T07:49:43Z

the issues in Azure is because the latest CL image is not there :/

cpanato · 2017-10-25T13:14:35Z

@squat @enxebre tests are green

cpanato · 2017-10-25T15:22:39Z

I will wait to this get merged then I will together with @enxebre to merge #2184

enxebre force-pushed the gcp-tests branch from 2e2f70b to d2ce318 Compare October 19, 2017 13:39

enxebre mentioned this pull request Oct 19, 2017

platform/gcp: add jenkins config for tests #2184

Closed

cpanato reviewed Oct 19, 2017

View reviewed changes

cpanato added area/testing platform/aws platform/azure run-smoke-tests labels Oct 19, 2017

enxebre force-pushed the gcp-tests branch from d2ce318 to 31cf1d1 Compare October 19, 2017 15:24

enxebre force-pushed the gcp-tests branch from 31cf1d1 to 68ddfbd Compare October 19, 2017 15:46

enxebre force-pushed the gcp-tests branch 2 times, most recently from 555e75f to 103e576 Compare October 19, 2017 17:07

squat suggested changes Oct 20, 2017

View reviewed changes

enxebre force-pushed the gcp-tests branch from 103e576 to 82bb740 Compare October 20, 2017 13:26

enxebre mentioned this pull request Oct 20, 2017

tests/rspec/lib: ensure CL version is available #2193

Merged

squat previously approved these changes Oct 20, 2017

View reviewed changes

enxebre dismissed squat’s stale review via 27e1230 October 20, 2017 17:18

enxebre force-pushed the gcp-tests branch from 82bb740 to 27e1230 Compare October 20, 2017 17:18

enxebre force-pushed the gcp-tests branch 2 times, most recently from 8ea2670 to c1214a7 Compare October 20, 2017 19:08

enxebre force-pushed the gcp-tests branch from c1214a7 to 381e769 Compare October 24, 2017 16:53

squat added the platform/gcp label Oct 25, 2017

enxebre added 2 commits October 25, 2017 14:17

platform/gcp: improve tests support

efacdfe

platform/gcp: add TF_VAR_tectonic_gcp_ssh_key

ed73252

enxebre force-pushed the gcp-tests branch from 381e769 to ed73252 Compare October 25, 2017 12:17

enxebre mentioned this pull request Oct 26, 2017

Bootstrap #2230

Merged

cpanato approved these changes Oct 26, 2017

View reviewed changes

squat merged commit d6f8644 into coreos:master Oct 26, 2017

platform/gcp: improve tests support #2178

platform/gcp: improve tests support #2178

Uh oh!

Conversation

enxebre commented Oct 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coreosbot commented Oct 18, 2017

Uh oh!

coreosbot commented Oct 18, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

enxebre Oct 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cpanato commented Oct 19, 2017

Uh oh!

cpanato commented Oct 19, 2017

Uh oh!

cpanato commented Oct 19, 2017

Uh oh!

enxebre commented Oct 19, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

enxebre commented Oct 20, 2017

Uh oh!

squat commented Oct 20, 2017

Uh oh!

squat commented Oct 20, 2017

Uh oh!

squat commented Oct 20, 2017

Uh oh!

enxebre commented Oct 20, 2017

Uh oh!

enxebre commented Oct 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

enxebre commented Oct 24, 2017

Uh oh!

squat commented Oct 25, 2017

Uh oh!

squat commented Oct 25, 2017

Uh oh!

cpanato commented Oct 25, 2017

Uh oh!

cpanato commented Oct 25, 2017

Uh oh!

cpanato commented Oct 25, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

enxebre commented Oct 18, 2017 •

edited

Loading

enxebre Oct 20, 2017 •

edited

Loading

enxebre commented Oct 20, 2017 •

edited

Loading