Adding openstack bakery (#3) #632

edwinavalos · 2017-07-31T16:36:14Z

Adding Openstack Bakery commands to halyard

h/t @shazy792 @dylanbernhardt @MatthewKohner @msilpala

@shazy792

Adding Openstack Bakery commands to halyard h/t @shazy792 @dylanbernhardt @MatthewKohner @msilpala

lwander

Nice work guys, just a few comments

lwander · 2017-07-31T17:20:57Z

...pinnaker/halyard/cli/command/v1/config/providers/openstack/OpenstackAddBaseImageCommand.java

+
+@Parameters(separators = "=")
+public class OpenstackAddBaseImageCommand extends AbstractAddBaseImageCommand{
+    protected String getProviderName() {return "openstack"; }


Formatting seems off here

lwander · 2017-07-31T17:25:20Z

...spinnaker/halyard/cli/command/v1/config/providers/openstack/OpenstackEditAccountCommand.java

+      names = "--remove-consul-config",
+      description = "Removes currently configured consul config file."
+  )
+  private boolean removeConsulConfig;


These "-remove" style flags aren't needed. You can always clear with --consul-config=.

lwander · 2017-07-31T17:27:40Z

...ava/com/netflix/spinnaker/halyard/config/model/v1/providers/openstack/OpenstackProvider.java

+    result.setAuthUrl("http://default.com");
+    result.setFloatingIpPool("default_pool");
+    result.setInsecure(false);
+    result.setNetworkId("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx");


These defaults will be set any time someone initializes the bakery options. The way these read, I'm guessing they'd have to be cleared/overwritten each time for this to be useful. If that's the case, just leave the fields blank.

lwander · 2017-07-31T17:28:35Z

...flix/spinnaker/halyard/config/validate/v1/providers/openstack/OpenstackAccountValidator.java

    OpenstackAccount.OpenstackLbaasOptions lbaas = account.getLbaas();
-    String regions = account.getRegions();
+    // TODO: consul config does not yet exist for OpenStack


Where does it not exist? Could you link to an open bug in the component where it needs to be added in the TODO() braces?

lwander · 2017-07-31T17:29:30Z

...flix/spinnaker/halyard/config/validate/v1/providers/openstack/OpenstackAccountValidator.java

    if (!StringUtils.endsWith(authUrl, "/v3")) {
      psBuilder.addProblem(Problem.Severity.WARNING, "You must use Keystone v3. The default auth url will be of the format IP:5000/v3.");
    }

+    if (domainName == null || domainName.isEmpty()) {


All of these string checks can be replaced with StringUtils.isEmpty(..) from org.apache.commons.lang3.

lwander · 2017-07-31T17:30:01Z

...innaker/halyard/config/validate/v1/providers/openstack/OpenstackBakeryDefaultsValidator.java

+            return;
+        }
+
+


Extra newline

lwander · 2017-07-31T17:30:17Z

...innaker/halyard/config/validate/v1/providers/openstack/OpenstackBakeryDefaultsValidator.java

+        boolean insecure = n.isInsecure();
+        List<OpenstackBaseImage> baseImages = n.getBaseImages();
+
+        if (StringUtils.isEmpty(authUrl) &&


Why is this acceptable?

We kept this line so the bakery defaults can be null when an account is enabled and added, but bakery parameters aren't set yet.

Ah, and there are no defaults that can be set which work out of the box?

Ok makes sense, thanks

lwander · 2017-07-31T17:30:31Z

...ix/spinnaker/halyard/config/validate/v1/providers/openstack/OpenstackBaseImageValidator.java

+            p.addProblem(Problem.Severity.ERROR, "No ssh username supplied for openstack base image.");
+        }
+
+        // TODO(shazy792) Add check to see if image actually exists on openstack instance(thum


Looks like the comment got cut off here

lwander · 2017-07-31T17:30:45Z

...lix/spinnaker/halyard/config/validate/v1/providers/openstack/OpenstackProviderValidator.java

+        n.getAccounts().forEach(openstackAccount -> openstackAccountValidator.validate(p, openstackAccount));
+
+        new OpenstackBakeryDefaultsValidator(credentialsList, halyardVersion).validate(p, n.getBakeryDefaults());
+


Extra newline

markus-silpala-tgt · 2017-07-31T21:37:51Z

...x/spinnaker/halyard/cli/command/v1/config/providers/openstack/OpenstackBaseImageCommand.java

+
+import com.beust.jcommander.Parameters;
+import com.netflix.spinnaker.halyard.cli.command.v1.config.providers.bakery.AbstractBaseImageCommand;
+


Most of our commands seem to have a little class-level comment describing what they do in very generic terms. Do we want that here? @edwinavalos

edwinavalos · 2017-07-31T21:38:52Z

...spinnaker/halyard/cli/command/v1/config/providers/openstack/OpenstackEditAccountCommand.java


  @Parameter(
-      names = "--remove-user-data-file",


Yep, Lars explained a difference between these removes and others in halyard,only removes that edit a list should be implemented the halyard way is to set the variable to nothing

markus-silpala-tgt · 2017-07-31T22:06:26Z

.../spinnaker/halyard/cli/command/v1/config/providers/openstack/OpenstackCommandProperties.java

+
+  static final String SSH_USER_NAME_DESCRIPTION = "The ssh username for the baking configuration.";
+
+  static final String REGION_DESCRIPTION = "The region for the baking configuration.";


@edwinavalos Does this argument support multiple regions separated by commas? If so it's likely worth calling that out here in the description. If not, thank you for educating me.

No, for Base Image virtualization settings it only supports one region.

markus-silpala-tgt · 2017-07-31T22:08:22Z

...java/com/netflix/spinnaker/halyard/config/model/v1/providers/openstack/OpenstackAccount.java

 import com.netflix.spinnaker.halyard.config.model.v1.node.Account;
 import com.netflix.spinnaker.halyard.config.model.v1.node.LocalFile;
 import com.netflix.spinnaker.halyard.config.model.v1.node.Validator;
 import com.netflix.spinnaker.halyard.config.problem.v1.ConfigProblemSetBuilder;
 import lombok.Data;
 import lombok.EqualsAndHashCode;

+import java.util.ArrayList;


Unused import.

markus-silpala-tgt · 2017-07-31T22:08:42Z

...java/com/netflix/spinnaker/halyard/config/model/v1/providers/openstack/OpenstackAccount.java

  private String authUrl;
  private String username;
  private String password;
  private String projectName;
  private String domainName;
  private Boolean insecure = false;
+  private String heatTemplateLocation;
+  private String consulConfig;
+//  private  lbaasConfig;


Any reason not to just delete this line?

markus-silpala-tgt · 2017-07-31T22:11:22Z

...ava/com/netflix/spinnaker/halyard/config/model/v1/providers/openstack/OpenstackProvider.java

@@ -1,11 +1,15 @@
 package com.netflix.spinnaker.halyard.config.model.v1.providers.openstack;

+import com.netflix.spinnaker.halyard.config.model.v1.node.HasImageProvider;
 import com.netflix.spinnaker.halyard.config.model.v1.node.Provider;


Unused import.

markus-silpala-tgt · 2017-07-31T22:13:13Z

...ard-config/src/main/java/com/netflix/spinnaker/halyard/config/services/v1/BakeryService.java

@@ -77,7 +77,8 @@ private BaseImage getBaseImage(NodeFilter filter, String baseImageName) {
    }
  }

-  public BakeryDefaults getBakeryDefaults(String deploymentName, String providerName) {
+  public BakeryDefaults
+  getBakeryDefaults(String deploymentName, String providerName) {


Pretty sure this line-break was accidental.

markus-silpala-tgt · 2017-07-31T22:42:56Z

...flix/spinnaker/halyard/config/validate/v1/providers/openstack/OpenstackAccountValidator.java

          .lbaasConfig(lbaasConfig)
          .userDataFile(userDataFile)
          .build();
+      if (openstackCredentials == null) {
+        return;


I'm fairly certain that it's impossible for this value to be null at this point: Java constructors cannot return null, and the build() method only calls a constructor and returns its result.

Did this check come about in response to an error? Or does it simply conform to convention elsewhere within the spinnaker/halyard code base? I'm new here so this might be a silly question. @edwinavalos

I agree that this code path can't be hit - what's the intent here? Unless .build() can return null?

markus-silpala-tgt · 2017-07-31T22:50:04Z

...innaker/halyard/config/validate/v1/providers/openstack/OpenstackBakeryDefaultsValidator.java

+import com.netflix.spinnaker.halyard.config.model.v1.providers.openstack.OpenstackBaseImage;
+import com.netflix.spinnaker.halyard.config.problem.v1.ConfigProblemSetBuilder;
+import com.netflix.spinnaker.halyard.config.validate.v1.providers.google.GoogleBaseImageValidator;
+import com.netflix.spinnaker.halyard.core.problem.v1.Problem;


Three unused imports:

GoogleNamedAccountCredentials

OpenstackCredentials

GoogleBaseImageValidator

markus-silpala-tgt · 2017-07-31T23:03:14Z

.../java/com/netflix/spinnaker/halyard/deploy/spinnaker/v1/profile/deck/DeckProfileFactory.java

@@ -124,7 +124,7 @@ protected void setProfile(Profile profile, DeploymentConfiguration deploymentCon
    if (openstackProvider.getPrimaryAccount() != null) {
      OpenstackAccount openstackAccount = (OpenstackAccount) accountService.getProviderAccount(deploymentConfiguration.getName(), "openstack", openstackProvider.getPrimaryAccount());
      //Regions in openstack are a comma separated list. Use the first as primary.


This comment is out of date, isn't it? Remove or update. Regions are now a list of strings from command all the way to domain objects.

markus-silpala-tgt · 2017-07-31T23:39:28Z

@edwinavalos I felt bad for putting in so many nit-sized remarks about unused imports, so I made a secondary PR you and use to just take all those changes en masse: https://github.com/edwinavalos/halyard/pull/6

* fix formatting * fix virtualization settings for openstack base image

edwinavalos · 2017-08-02T16:58:56Z

@lwander this one is ready for a once over again

lwander

Nice, just a few more comments

lwander · 2017-08-02T17:02:53Z

...er/halyard/cli/command/v1/config/providers/openstack/OpenstackEditBakeryDefaultsCommand.java

+ */
+@Parameters(separators = "=")
+public class OpenstackEditBakeryDefaultsCommand extends AbstractEditBakeryDefaultsCommand<OpenstackBakeryDefaults> {
+    protected String getProviderName() { return "openstack"; }


I'd prefer to have this on a new line to be consistent with other formatting

lwander · 2017-08-02T17:03:44Z

...er/halyard/cli/command/v1/config/providers/openstack/OpenstackEditBakeryDefaultsCommand.java

+        names = "--insecure",
+        description = "Set the default security setting (true or false) to connect to the openstack account."
+    )
+    private boolean insecure = false;


This should be nullable Boolean to decide if it's been set or not (otherwise will always default to false when a user makes an unrelated edit).

lwander · 2017-08-02T17:04:25Z

...innaker/halyard/cli/command/v1/config/providers/openstack/OpenstackEditBaseImageCommand.java

+        virtualizationSettings.setRegion(isSet(region) ? region : virtualizationSettings.getRegion());
+        virtualizationSettings.setInstanceType(isSet(instanceType) ? instanceType : virtualizationSettings.getInstanceType());
+        virtualizationSettings.setSshUserName(isSet(sshUserName) ? sshUserName : virtualizationSettings.getSshUserName());
+//        baseImage.setVirtualizationSettings(virtualizationSettings);


Please delete or uncomment

lwander · 2017-08-02T17:05:14Z

...ava/com/netflix/spinnaker/halyard/config/model/v1/providers/openstack/OpenstackProvider.java

+    result.setFloatingIpPool(null);
+    result.setInsecure(false);
+    result.setNetworkId(null);
+    result.setPassword(null);


Setting null isn't needed, you can just return new OpenstackBakeryDefaults()

lwander · 2017-08-02T17:06:20Z

...flix/spinnaker/halyard/config/validate/v1/providers/openstack/OpenstackAccountValidator.java

          .lbaasConfig(lbaasConfig)
          .userDataFile(userDataFile)
          .build();
+      if (openstackCredentials == null) {
+        return;


I agree that this code path can't be hit - what's the intent here? Unless .build() can return null?

* remove extraneous if * fix intellij formatting * Make bakery insecure parameter nullable * remove commented out line * remove OpenstackBakeryDefaults null setting and make insecure required setting so vaildation works

lwander · 2017-08-02T20:11:44Z

I'm ready to merge this, any objections?

MatthewKohner · 2017-08-02T20:13:10Z

no objections, merge away!

Adding openstack bakery (#3)

2bac8b3

Adding Openstack Bakery commands to halyard h/t @shazy792 @dylanbernhardt @MatthewKohner @msilpala

lwander reviewed Jul 31, 2017

View reviewed changes

dylanbernhardt added 2 commits July 31, 2017 15:15

fix openstack bakery commands (#4)

c0e0c66

fix formatting (#5)

e8c5e95

markus-silpala-tgt reviewed Jul 31, 2017

View reviewed changes

edwinavalos commented Jul 31, 2017

View reviewed changes

markus-silpala-tgt reviewed Jul 31, 2017

View reviewed changes

markus-silpala-tgt and others added 2 commits July 31, 2017 19:06

Remove unused imports and some extra spaces. (#6)

986cce7

fix virtualization settings for openstack base image (#7)

52ff81b

* fix formatting * fix virtualization settings for openstack base image

lwander reviewed Aug 2, 2017

View reviewed changes

Bakery Options finishing touches (#8)

3038818

* remove extraneous if * fix intellij formatting * Make bakery insecure parameter nullable * remove commented out line * remove OpenstackBakeryDefaults null setting and make insecure required setting so vaildation works

lwander approved these changes Aug 2, 2017

View reviewed changes

lwander merged commit 94bac35 into spinnaker:master Aug 2, 2017

		n.getAccounts().forEach(openstackAccount -> openstackAccountValidator.validate(p, openstackAccount));

		new OpenstackBakeryDefaultsValidator(credentialsList, halyardVersion).validate(p, n.getBakeryDefaults());


		import com.beust.jcommander.Parameters;
		import com.netflix.spinnaker.halyard.cli.command.v1.config.providers.bakery.AbstractBaseImageCommand;


		static final String SSH_USER_NAME_DESCRIPTION = "The ssh username for the baking configuration.";

		static final String REGION_DESCRIPTION = "The region for the baking configuration.";

Adding openstack bakery (#3) #632

Adding openstack bakery (#3) #632

Conversation

edwinavalos commented Jul 31, 2017

lwander left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markus-silpala-tgt commented Jul 31, 2017

edwinavalos commented Aug 2, 2017

lwander left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lwander commented Aug 2, 2017

MatthewKohner commented Aug 2, 2017