Skip to content

Conversation

@wking
Copy link
Member

@wking wking commented May 23, 2019

Some regions have per-zone support differences for available instance classes. For example, us-west-2 has m4 instances in zones a through c, but not in zone d. It supports m5 instances in all four zones.

Unfortunately, we can't switch to just using the reserved-instance offerings, because Paris claims those for m4.large in all three of its zones:

$ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
eu-west-3a
eu-west-3b
eu-west-3c

but still does not price on-demand compute instances for that region:

$ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
AWS GovCloud (US)
Asia Pacific (Mumbai)
Asia Pacific (Osaka-Local)
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
Canada (Central)
EU (Frankfurt)
EU (Ireland)
EU (London)
South America (Sao Paulo)
US East (N. Virginia)
US East (Ohio)
US West (N. California)
US West (Oregon)

So the new test here only assumes a match when both APIs claim support for the instance class in a given region to avoid re-breaking Paris (#1127).

After rerolling the test to account for that sort of thing, I made the default change to address:

$ AWS_PROFILE=ci go test -count 1 .
--- FAIL: TestGetDefaultInstanceClass (57.36s)
    --- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
      default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
      default_instance_class_test.go:177:
          Error Trace:  default_instance_class_test.go:177
          Error:        Not equal:
                        expected: "m4"
                        actual  : "m5"

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1 +1 @@
                        -m4
                        +m5
          Test:         TestGetDefaultInstanceClass/US_West_(Oregon)
FAIL
FAIL  github.com/openshift/installer/platformtests/aws  57.374s

The zone-error fallback covers regions that I don't have access too, where requesting zones returns errors like (us-gov-east-1):

AuthFailure: AWS was not able to validate the provided access credentials

and (ap-northeast-3):

OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.

@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels May 23, 2019
@wking
Copy link
Member Author

wking commented May 23, 2019

/cherrypick release-4.1

@openshift-cherrypick-robot

@wking: once the present PR merges, I will cherry-pick it on top of release-4.1 in a new PR and assign it to you.

Details

In response to this:

/cherrypick release-4.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@wking wking force-pushed the per-zone-aws-instance-support branch from bfa422f to bd10ac0 Compare May 23, 2019 19:26
Some regions have per-zone support differences for available instance
classes.  For example, us-west-2 has m4 instances in zones a through
c, but not in zone d.  It supports m5 instances in all four zones [1].

Unfortunately, we can't switch to just using the reserved-instance
offerings, because Paris claims those for m4.large in all three of its
zones:

  $ AWS_PROFILE=ci aws --region eu-west-3 ec2 describe-reserved-instances-offerings --instance-tenancy default --instance-type m4.large --product-description 'Linux/UNIX' --filters Name=scope,Values='Availability Zone' | jq -r '[.ReservedInstancesOfferings[].AvailabilityZone] | sort | unique[]'
  eu-west-3a
  eu-west-3b
  eu-west-3c

but still does not price on-demand compute instances for that region:

  $ AWS_PROFILE=ci aws --region us-east-1 pricing get-products --service-code AmazonEC2 --filters Field=tenancy,Type=TERM_MATCH,Value=Shared Field=productFamily,Type=TERM_MATCH,Value='Compute Instance' Field=operatingSystem,Type=TERM_MATCH,Value=Linux Field=instanceFamily,Type=TERM_MATCH,Value='General purpose' | jq -r '[.PriceList[] | fromjson | .product.attributes | select(.instanceType == "m4.large").location] | sort | unique[]'
  AWS GovCloud (US)
  Asia Pacific (Mumbai)
  Asia Pacific (Osaka-Local)
  Asia Pacific (Seoul)
  Asia Pacific (Singapore)
  Asia Pacific (Sydney)
  Asia Pacific (Tokyo)
  Canada (Central)
  EU (Frankfurt)
  EU (Ireland)
  EU (London)
  South America (Sao Paulo)
  US East (N. Virginia)
  US East (Ohio)
  US West (N. California)
  US West (Oregon)

So the new test here only assumes a match when both APIs claim support
for the instance class in a given region to avoid re-breaking Paris
[2].

After rerolling the test to account for that sort of thing, I made the
default change to address:

  $ AWS_PROFILE=ci go test -count 1 .
  --- FAIL: TestGetDefaultInstanceClass (57.36s)
      --- FAIL: TestGetDefaultInstanceClass/US_West_(Oregon) (1.22s)
        default_instance_class_test.go:176: map[m4:map[us-west-2c:{} us-west-2a:{} us-west-2b:{}] m5:map[us-west-2b:{} us-west-2c:{} us-west-2d:{} us-west-2a:{}]]
        default_instance_class_test.go:177:
            Error Trace:  default_instance_class_test.go:177
            Error:        Not equal:
                          expected: "m4"
                          actual  : "m5"

                          Diff:
                          --- Expected
                          +++ Actual
                          @@ -1 +1 @@
                          -m4
                          +m5
            Test:         TestGetDefaultInstanceClass/US_West_(Oregon)
  FAIL
  FAIL  github.com/openshift/installer/platformtests/aws  57.374s

The zone-error fallback covers regions that I don't have access too,
where requesting zones returns errors like (us-gov-east-1):

  AuthFailure: AWS was not able to validate the provided access credentials

and (ap-northeast-3):

  OptInRequired: You are not subscribed to this service. Please go to http://aws.amazon.com to subscribe.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1713157#c1
[2]: openshift#1127
@wking wking force-pushed the per-zone-aws-instance-support branch from bd10ac0 to 3514543 Compare May 23, 2019 19:35
@wking
Copy link
Member Author

wking commented May 23, 2019

CC @kalexand-rh

@kalexand-rh
Copy link
Contributor

@abhinavdahiya
Copy link
Contributor

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label May 23, 2019
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhinavdahiya, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [abhinavdahiya,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-cherrypick-robot

@wking: new pull request created: #1787

Details

In response to this:

/cherrypick release-4.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Contributor

@wking: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/prow/e2e-openstack 3514543 link /test e2e-openstack
ci/prow/e2e-aws-scaleup-rhel7 3514543 link /test e2e-aws-scaleup-rhel7

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@wking wking deleted the per-zone-aws-instance-support branch May 24, 2019 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants