Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update and add onto the Pipeline Handbook chapter #465

Merged
merged 22 commits into from
Dec 2, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
2bd70fc
Move the chapter sub-sections up above the chapter overview page's ToC
Nov 22, 2016
a77a969
Add a glossary section as a future home for terms
Nov 22, 2016
e34e22b
Update the Pipeline chapter index page to more succinctly state what …
Nov 22, 2016
d120a23
Add some clarification on how casing for Pipeline vs. pipeline should…
Nov 22, 2016
3593537
Move the definition of Pipeline terms to the chapter index page
Nov 22, 2016
4da7a3c
Flesh out a more suitable "Why Pipeline" section for the Pipeline cha…
Nov 22, 2016
5ea2afc
"Overview" => "Getting Started" which more accurately captures that s…
Nov 22, 2016
e5b82b1
Add a place-holder subsection for covering Pipeline Shared Libraries
Nov 22, 2016
1ff5d6b
Gutting and reworking the entire Getting Started sub-section
Nov 30, 2016
a80610b
Add the Advanced section for the Pipeline chapter to house some conte…
Nov 30, 2016
2ee31a9
Move some multibranch content out into its own section
Nov 30, 2016
4301473
Finish up the first-finalish-draft of the Pipeline > Getting Started …
Nov 30, 2016
1c55423
Add redirect from doc/book/pipeline/overview for URL compatibility
Dec 1, 2016
f7a1e2c
Finish the first-draft of "The Jenkinsfile" section
Dec 1, 2016
327a0bf
Tidy up a concise Multibranch Pipeline section
Dec 1, 2016
8421597
Add the initial import of Shared Libraries documentation
Dec 1, 2016
6da74f5
On second thought, not sure any of this content is especially useful/…
Dec 1, 2016
f4da684
Copy-edits courtesy of a review from @omehegan
Dec 1, 2016
e643d90
"in the Jenkins environment" as a means for referring to the collecti…
Dec 2, 2016
c184c25
Minor copy edits courtesy of a review by @alvin-huang
Dec 2, 2016
8966306
Add notes for where declarative pipeline examples are not yet impleme…
Dec 2, 2016
ce79b63
Re-order and restructure a bit of the Shared Libraries document
Dec 2, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions STYLEGUIDE.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,25 @@ CAUTION: This document is still a work in progress.
* Master
* Node
* Pipeline
* Executor

Mixing of these terms is incorrect

NOTE: When referring to a Jenkins Pipeline via short-hand ("Pipeline"), it
should always be title-cased. When referring to a conceptual pipeline
("continuous delivery pipeline"), it it should always be lower-cased.

Copy link
Contributor

@bitwiseman bitwiseman Dec 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a guideline based on your comment comment to me:

a style thing that should probably not need to be said, but with sections that built into each other, I think there's a need for an "intro"/preamble at the beginning of one chapter section, and an "outtro" at the end

It shouldn't need to be said, but it still needs to be said.


When needing to refer to the collection of executors and/or nodes it is best to
refer to them collectively as "the Jenkins environment", for example:

____
[..] they will now execute in parallel assuming the requisite capacity exists
in the Jenkins environment.
____

As opposed to "the Jenkins cluster", "agent pool" or any other phrase to
describe the collective set of resources provided by Jenkins.

== Syntax/Formatting

Expand Down Expand Up @@ -110,6 +126,15 @@ By default, all of these use the class name as label, but that can be customized
javadoc:hudson.scm.SCM#all()[a list of all known SCM implementations]
----

== Handbook Style Guide

* For consecutive sections that are related to or build on each other, there
should be a reasonable "intro" or preamble at the beginning of one section
and a reasonable "outtro" at the end, to provide continuity between the
documents



== Assorted comments

* Prefer "for example" over "e.g." which can be more clear to non-native english
Expand Down
24 changes: 15 additions & 9 deletions content/_layouts/chapter.html.haml
Original file line number Diff line number Diff line change
Expand Up @@ -34,20 +34,26 @@ layout: default
%a.next.page-link{:href => File.join('..', next_chapter.key)}
#{next_chapter.title} ⇒



.section
%h1
= page.title

= content
- if chapter.sections && chapter.sections.size > 0
%h2
Sub-sections

%ul
- chapter.sections.each do |section|
%li
%a{:href => section.key}
= section.title
#subsections
.toc
#toc-title
Chapter Sub-Sections
%ul
- chapter.sections.each do |section|
%li
%a{:href => section.key}
= section.title


= content


.section_nav.pagination-links
- if previous_chapter
Expand Down
2 changes: 2 additions & 0 deletions content/css/jenkins.css
Original file line number Diff line number Diff line change
Expand Up @@ -1345,8 +1345,10 @@ blockquote {
background-color: #f9f9f9;
border-left: 1px solid #c9c9c9;
float: right;
width: 10rem;
margin-left: 15px;
padding: 10px;
clear: right;
}

.toc li li {
Expand Down
1 change: 1 addition & 0 deletions content/doc/book/book.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ chapters:
- operating
- scaling
- appendix
- glossary
2 changes: 2 additions & 0 deletions content/doc/book/glossary/chapter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
sections:
19 changes: 19 additions & 0 deletions content/doc/book/glossary/index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
layout: chapter
---
:description:
:author:
:email: [email protected]
:sectanchors:
:toc: left
:notitle:

[glossary]
= Glossary


[glossary]

Freestyle Job::
A Freestyle Job
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tautology for the sake of place holder?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I need to spend more time thinking about a crystal-clear definition for some terms in the glossary 👿

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rtyler If you're trying to define something and having trouble, try thinking about it as it appears in the wild--who is using it? What do they use it for? What does it do that is unique within the product or feature set? What's the business case for it, ie why do your users care about it?


4 changes: 3 additions & 1 deletion content/doc/book/pipeline/chapter.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
---
sections:
- overview
- getting-started
- jenkinsfile
- multibranch
- shared-libraries
231 changes: 231 additions & 0 deletions content/doc/book/pipeline/getting-started.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
---
layout: section
---
:notitle:
:description:
:author:
:email: [email protected]
:sectanchors:
:toc:
:imagesdir: /doc/book/resources/pipeline
:hide-uri-scheme:

= Getting Started

Jenkins Pipeline is a suite of plugins which supports implementing and
integrating continuous delivery pipelines into Jenkins. Pipeline provides an
extensible set of tools for modeling simple-to-complex delivery pipelines "as
code" via the Pipeline DSL.
footnoteref:[dsl,link:https://en.wikipedia.org/wiki/Domain-specific_language[Domain-Specific Language]]

This section will introduce some of the key concepts to Jenkins Pipeline and
Copy link

@scribblingfox scribblingfox Dec 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nit, but "This section will introduce" could just be "This section introduces" Present tense is stronger, and the reader is literally in the section already--no need to talk about what will happen when they're reading right now.

help introduce the basics of defining and working with Pipelines inside of a
running Jenkins instance.

== Prerequisites

To use Jenkins Pipeline, you will need:

* Jenkins 2.x or later (older versions back to 1.642.3 may work but are not
recommended)
* Pipeline plugin
footnoteref:[pipeline, link:https://plugins.jenkins.io/workflow-aggregator[Pipeline plugin]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To use Jenkins Pipeline, you will need:

  • Jenkins 2.x or later (older versions back to 1.642.3 may work but are not
    recommended)
  • Pipeline plugin
    footnoteref:[pipeline, link:https://plugins.jenkins.io/workflow-aggregator[Pipeline plugin]]


To learn how to install and manage plugins, consult <<../managing/plugins#, Managing Plugins>>.

== Defining a Pipeline

Pipeline Script is written in
link:http://groovy-lang.org/[Groovy].
The relevant bits of
link:http://groovy-lang.org/semantics.html[Groovy syntax]
will be introduced as necessary in this document, so while an understanding of
Groovy is helpful, it is not required to use Pipeline Script.

A basic Pipeline can be created in either of the following ways:

* By entering a script directly in the Jenkins web UI.
* By creating a `Jenkinsfile` which can be checked into a project's source
control repository.

The syntax for defining a Pipeline with either approach is the same, but while
Jenkins supports entering Pipeline Script directly into the web UI, it's
generally considered best practice to define the Pipeline in a `Jenkinsfile`
which Jenkins will then load directly from source control.
footnoteref:[scm, https://en.wikipedia.org/wiki/Source_control_management]


=== Defining a Pipeline in the Web UI

To create a basic Pipeline in the Jenkins web UI, follow these steps:

* Click *New Item* on Jenkins home page.

image::new-item-selection.png["Click *New Item* on the Jenkins home page", role=center]

* Enter a name for your Pipeline, select *Pipeline* and click *OK*.

[CAUTION]
====
Jenkins uses the name of the Pipeline to create directories on disk. Pipeline
names which include spaces may uncover bugs in scripts which do not expect
paths to contain spaces.
====

image::new-item-creation.png["Enter a name, select *Pipeline*, and click *OK*", role=center]



* In the *Script* text-area, enter a Pipeline script and click *Save*.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

text-area probably does not need to be hypenated. "Text area" or "Text field" would both suffice.


image::hello-world-script.png["In the *Script* text-area, enter a Pipeline script and click Save", role=center]

* Click *Build Now* to run the Pipeline.

image::build-now.png["Click *Build Now* to run the Pipeline", role=center]


* Click *#1* under "Build History" and then click *Console Output* to see the
full output from the Pipeline.

image::hello-world-console-output.png["Click *Console Output* for the Pipeline", role=center]

The example above shows a successful run of a basic Pipeline created in the Jenkins
web UI, using two valuable steps.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lose "valuable" here, it's fluff. You don't need to sell the value of a product or service in its documentation; the product should speak for itself.

Node and echo are also two familiar terms for most readers of this doc, so reminding everyone how valuable they are is probably overkill.


[pipeline]
----
// Script //
node { // <1>
echo 'Hello World' // <2>
}
// Declarative not yet implemented //
----
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No // Declarative produces a warning.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I think we need to play with the warnings a bit more. There is some documentation which refers explicitly to Pipeline Script.

I'm also not including any Declarative Pipeline in this pass since I expect to merge this before Declarative is made a dependency of the workflow-aggregator plugin

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what should go here is something like // Declarartive - TODO or in the cases where it doesn't apply // Declarartive - Not applicable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the current extension, those will result in Decalarive blocks being printed on the page. IMO this is outside the scope of this pull request and should be discussed elsewhere.

Copy link
Contributor

@bitwiseman bitwiseman Dec 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'm good with that.

<1> `node` allocates an executor and workspace in the Jenkins environment.
<2> `echo` writes simple string in the Console Output.


// Despite :sectanchors:, explicitly defining an anchor because it will be
// referenced from other documents
[[defining-a-pipeline-in-scm]]
=== Defining a Pipeline in SCM

Complex pipelines would be cumbersome to write and maintain if you could only do
that in the text area provided by the Jenkins job configuration page.

Accordingly, you also have the option of writing Pipeline scripts
(Jenkinsfiles) with a text editor and then loading those scripts into Jenkins
using the *Pipeline Script from SCM* option.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposed rephrase, just to shorten and clarify:

"Complex pipelines are hard to write and maintain within the text area of the Jenkins job configuration page. If necessary, you can write Pipeline scripts (Jenkinsfiles) in a text editor and access them in Jenkins with the Pipeline Script from SCM option."

Loading pipeline scripts using the `checkout scm` step leverages the
idea of "Pipeline as code" and allows easy maintenance of Pipelines with source
control and text-editors.

To do this, select *Pipeline script from SCM* when defining the pipeline.

With the *Pipeline script from SCM* option selected, you do not enter any Groovy
code in the Jenkins UI; you just indicate by specifying a path where in source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you just indicate by specifying a path where in source code you want to retrieve the pipeline from. -> you just specify a path to a file in source control.

code you want to retrieve the pipeline from. When you update the designated
repository, a new build is triggered, as long as your job is configured with an
SCM polling trigger.

[TIP]
====
The first line of a `Jenkinsfile` should be `#!groovy`
footnoteref:[shebang, https://en.wikipedia.org/wiki/Shebang_(Unix)]
which text editors, IDEs, GitHub, etc will use to syntax highlight the
`Jenkinsfile` properly as Groovy code.
====

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this get moved up to the beginning of this section? This tip seems more like instructions for how to prevent a fatal error--so, not optional, and probably worth mentioning as a prerequisite for writing Jenkinsfiles successfully.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not a prerequisite for the Pipeline to execute properly and only serves to signal an IDE/editor for syntax highlighting.



== Built-in Documentation

Pipeline ships with built-in documentation features to make it
easier to create Pipelines of varying complexities. This built-in documentation
is automatically generated and updated based on the plugins installed in the
Jenkins instance.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to give some real-world examples here of what the Snippet Generator might be used to do. It's kinda hard to understand the use case from this section of the docs.

The built-in documentation can be found globally at:
link:http://localhost:8080[localhost:8080/pipeline-syntax/],
assuming you have a Jenkins instance running on localhost port 8080. The same
documentation is also linked as *Pipeline Syntax* in the side-bar for any
configured Pipeline project.

image::pipeline-syntax-sidebar.png[Pipeline Syntax in the side-bar, role=center]

[[snippet-generator]]
=== Snippet Generator

The built-in "Snippet Generator" utility is helpful for creating bits of
code for individual steps, discovering new steps provided by plugins, or
experimenting with different parameters for a particular step.

The Snippet Generator is dynamically populated with a list of the steps
available to the Jenkins instance. The number of steps available is dependent
on the plugins installed which explicitly expose steps for use in Pipeline.

To generate a step snippet with the Snippet Generator:

. Navigate to the Snippet Generator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one navigate to the snippet generator? First step is vague, unless there's something dead-simple obvious in the image that I can't see. :)

. Select the desired step in the *Sample Step* dropdown menu
. Use the dynamically populated area below the *Sample Step* dropdown to configure the selected step.
. Click *Generate Pipeline Script* to create a snippet of Pipeline which can be
copied and pasted into a Pipeline.

image::snippet-generator.png[Snippet Generator, role=center]

To access additional information and/or documentation about the step selected,
click on the help icon (blue question mark).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A screenshot of the location of the blue question mark probably wouldn't go amiss here.

=== Global Variable Reference
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we put a TODO here to move the details about specific global variables below to a different section? This feels a bit odd.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's part of "built-in documentation" though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The info about the globals is part of the built in docs. That first paragraph makes sense here.

But the details about each of the globals belongs in subsection of the general pipeline section.


In addition to the Snippet Generator, which only surfaces steps, Pipeline also
provides a built-in "*Global Variable Reference*." Like the Snippet Generator,
it is also dynamically populated by plugins. Unlike the Snippet Generator
however, the Global Variable Reference only contains documentation for
*variables*, provided by Pipeline or plugins, which are available for

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unnecessary comma here:

s/"...documentation forvariables, provided by Pipeline..."/"...documentation forvariables provided by Pipeline..."

Pipelines.

The variables provided by default in Pipeline are:

env::

Environment variables accessible from Pipeline Script, for example:
`env.PATH` or `env.BUILD_ID`. Consult the built-in
link:http://localhost:8080/pipeline-syntax/globals#env[Global Variable Reference]
for a complete, and up to date, list of environment variables
available in Pipeline.

params::

Exposes all parameters defined for the Pipeline as a read-only
link:http://groovy-lang.org/syntax.html#_maps[Map],
for example: `params.MY_PARAM_NAME`.

currentBuild::

May be used to discover information about the currently executing Pipeline,
with properties such as `currentBuild.result`, `currentBuild.displayName`,
etc. Consult the built-in
link:http://localhost:8080/pipeline-syntax/globals#currentBuild[Global Variable Reference]
for a complete, and up to date, list of properties available on `currentBuild`.


== Further Reading

This section merely scratches the surface of what can be done with Jenkins
Pipeline, but should provide enough of a foundation for you to start
experimenting with a test Jenkins instance.

In the next section, <<jenkinsfile#, The Jenkinsfile>>, more Pipeline steps
will be discussed along with patterns for implementing successful, real-world,
Jenkins Pipelines.


=== Additional Resources

* link:https://jenkins.io/doc/pipeline/steps[Pipeline Steps Reference],
encompassing all steps provided by plugins distributed in the Jenkins Update
Center.
* link:https://jenkins.io/doc/pipeline/examples[Pipeline Examples], a
community-curated collection of copyable Pipeline examples.
Loading