How to use SSH protocol mode in cookbook?

[isuftin]

Jenkins, by default, starts with SSH mode disabled. It's unclear what the workflow should be in order to set ssh as the default protocol in node['executor']['protocol']. If the cookbook initially attempts to converge using the ssh protocol, any CLI command attempted will fail because Jenkins is not accepting SSH connections. A user would manually need to go into the running Jenkins server and turn SSHd on. This breaks a fully automated CM workflow.

With current Jenkins releases, the remoting protocol is deprecated and should not be used.

This leaves the user with only http as the protocol that may be used.

I can't find any documentation for Jenkins that describes how Jenkins may be started with SSHd running.

Cookbook version

5.0.1

Chef-client version

13.0.118

Platform Details

CentOS 6.8 VM in VirtualBox. Running on MacOS 10.10.x

Scenario:

Set protocol to SSH for initial creation of the Jenkins server, attempt to perform any CLI action via downstream actions like adding a user or installing a plugin once server is running via jenkins::master

Steps to Reproduce:

Set execute protocol to ssh, run jenkins::master and then attempt to install a plugin via the plugin installation jenkins_plugin action.

Expected Result:

CLI command runs as expected

Actual Result:

           ================================================================================
           Error executing action `install` on resource 'jenkins_plugin[display-url-api]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '255'
           ---- Begin output of "java" -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://localhost:8080 -"ssh" -user "jenkins" install-plugin /tmp/kitchen/cache/display-url-api-2.0.plugin -name display-url-api  ----
           STDOUT:
           STDERR: May 12, 2017 1:02:49 PM hudson.cli.SSHCLI sshConnection
           WARNING: No header 'X-SSH-Endpoint' returned by Jenkins
           ---- End output of "java" -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://localhost:8080 -"ssh" -user "jenkins" install-plugin /tmp/kitchen/cache/display-url-api-2.0.plugin -name display-url-api  ----
           Ran "java" -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://localhost:8080 -"ssh" -user "jenkins" install-plugin /tmp/kitchen/cache/display-url-api-2.0.plugin -name display-url-api  returned 255

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/jenkins/libraries/_executor.rb:87:in `execute!'
           /tmp/kitchen/cache/cookbooks/jenkins/libraries/plugin.rb:317:in `install_plugin_from_url'
           /tmp/kitchen/cache/cookbooks/jenkins/libraries/plugin.rb:289:in `install_plugin_from_update_center'
           /tmp/kitchen/cache/cookbooks/jenkins/libraries/plugin.rb:124:in `block (2 levels) in <class:JenkinsPlugin>'
           /tmp/kitchen/cache/cookbooks/jenkins/libraries/plugin.rb:154:in `block in <class:JenkinsPlugin>'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/owi_jenkins/recipes/plugins.rb

            15:   jenkins_plugin plugin do
            16:     version version if version
            17:     install_deps true
            18:     notifies :restart, 'service[jenkins]', :delayed
            19:   end
            20: end

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/owi_jenkins/recipes/plugins.rb:15:in `block in from_file'

           jenkins_plugin("display-url-api") do
             action [:install]
             default_guard_interpreter :default
             declared_type :jenkins_plugin
             cookbook_name "owi_jenkins"
             recipe_name "plugins"
             version "2.0"
             install_deps true
           end

           System Info:
           ------------
           chef_version=13.0.118
           platform=centos
           platform_version=6.8
           ruby=ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
           program_name=chef-client worker: ppid=7682;start=13:00:49;
           executable=/opt/chef/bin/chef-client

       [2017-05-12T13:02:49+00:00] ERROR: jenkins_plugin[display-url-api] (owi_jenkins::plugins line 15) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '255'
       ---- Begin output of "java" -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://localhost:8080 -"ssh" -user "jenkins" install-plugin /tmp/kitchen/cache/display-url-api-2.0.plugin -name display-url-api  ----
       STDOUT:
       STDERR: May 12, 2017 1:02:49 PM hudson.cli.SSHCLI sshConnection
       WARNING: No header 'X-SSH-Endpoint' returned by Jenkins
       ---- End output of "java" -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://localhost:8080 -"ssh" -user "jenkins" install-plugin /tmp/kitchen/cache/display-url-api-2.0.plugin -name display-url-api  ----
       Ran "java" -jar "/tmp/kitchen/cache/jenkins-cli.jar" -s http://localhost:8080 -"ssh" -user "jenkins" install-plugin /tmp/kitchen/cache/display-url-api-2.0.plugin -name display-url-api  returned 255

[isuftin]

Here's my solution for flipping SSHd on by defualt in my owi_jenkins cookbook. Got the idea from https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Hook+Script and the Jenkins Docker container Dockerfile and how they're doing automated setup. This still does not fix plugin installation, btw.

owi_jenkins::master :

include_recipe 'jenkins::master'

jenkins_home = node['jenkins']['master']['home']

directory File.join(jenkins_home, 'init.groovy.d') do
	user node['jenkins']['master']['user']
	group node['jenkins']['master']['group']
end

template File.join(jenkins_home, 'init.groovy.d', 'sshd_configure.groovy') do
	source 'sshd_configure.groovy.erb'
	user node['jenkins']['master']['user']
	group node['jenkins']['master']['group']
	variables(
		'port' => node['owi_jenkins']['ssh']['port']
	)
       notifies :restart, 'service[jenkins]', :immediate
end

sshd_configure.groovy.erb :

import jenkins.model.*

def instance = Jenkins.getInstance()
def sshd = instance.getDescriptor("org.jenkinsci.main.modules.sshd.SSHD")
def currentPort = sshd.getActualPort()
def expectedPort = <%= @port %>

if (currentPort != expectedPort) {
  sshd.setPort(expectedPort)
}

default.rb attributes:

# Enable/disable SSHd.
# If the port is 0, Jenkins will serve SSHd on a random port
# If the port is > 0, Jenkins will serve SSHd on that port specifically
# SIf the port is is -1 turns off SSHd.
default['owi_jenkins']['ssh']['port'] = 0

[kevlogan90]

There is also a file created that you could edit and restart Jenkins to take effect it's located at /var/lib/jenkins/org.jenkinsci.main.modules.sshd.SSHD.xml and you just need to enter a port for it to listen on. Then Jenkins will allow ssh protocol.

[jj2007]

Need help :
I am using ssh as protocol to connect to jenkins via cli .
The same command is working when run manually , but fails via my cookbook .

server # java -jar /opt/tomcat/webapps/ROOT/WEB-INF/jenkins-cli.jar -s http://localhost:8181/ -ssh -user update -i /home/update/.ssh/id_rsa who-am-i
Feb 16, 2018 11:18:02 AM hudson.cli.SSHCLI$1 verifyServerKey
WARNING: Unknown host key for server:32855
Authenticated as: update
Authorities:
authenticated

Failure via chef cookbook recipe : I am running it as a bash command resource in my recipe .

[2018-02-16T11:27:39+01:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: bash[install jenkins plugins] (sap-bs-jenkins::bs_jenkins line 104) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '255'
---- Begin output of "bash" "/tmp/chef-script20180216-23269-9xs4ow" ----
STDOUT:
STDERR: Feb 16, 2018 11:27:39 AM hudson.cli.SSHCLI sshConnection
WARNING: No header 'X-SSH-Endpoint' returned by Jenkins
---- End output of "bash" "/tmp/chef-script20180216-23269-9xs4ow" ----

Jenkins ssh port is set to random , and X-SSH-Endpoint is indeed set

server # curl -I http://localhost:8181
HTTP/1.1 200 OK
........
X-SSH-Endpoint: server:41228

[espoelstra]

@jj2007 you may need to run ssh-keyscan -t rsa -p $sshPort $YourJenkinsHost >> ~/.ssh/known_hosts as the user that Chef is running as. I'm not sure if the jenkins-cli uses the default SSH paths, otherwise you may need to pass an option to either ignore or trust the host key depending on how secure your environment needs to be.

[espoelstra]

@isuftin I think if you use the snippet(s) from this answer in your authentication.rb or wherever you tell Chef what authentication to use it will fix your plugin installation issues as well.

[isuftin]

@espoelstra Thanks I will give it a shot!

[espoelstra]

So according to this post and what I've gathered from the README here, you should just be able to add the snippets for the node.run_state[:jenkins_private_key] from my prior comment and the cookbook is supposed to figure out whether to use anonymous or fall back to SSH. I ended up having to reorder our recipes to put the SSH enable and the run_state bit BEFORE our LDAP configuration, otherwise it seemed to have issues, but with newer versions of Jenkins where more bits of the core have been split out, things like the Mailer plugin not being installed can break the run unless the cookbook handles it (there was a PR that addressed jenkins_user and Mailer specifically).