Building continuous delivery pipelines and similarly complex tasks in Jenkins using freestyle projects and traditional plugins can be awkward. You need to mix Parameterized Trigger, Copy Artifact, Promoted Builds, Conditional Build Step, and more just to express what should be a simple script. The Workflow plugin suite attempts to make it possible to directly write that script, what people often call a workflow (sometimes abbreviated flow), while integrating with Jenkins features like slaves and publishers.
Your whole workflow is a single Groovy script using an embedded DSL, possibly quite short and legible; there is no need to jump between multiple job configuration screens to see what is going on. Conditions, loops, variables, parallel tasks, and so on are defined using regular language constructs. At any point you can insert a shell/batch script to do “real work” (compilation, etc.).
Standard DSL functions (“steps”) let you run external processes, grab slave nodes and workspaces, perform SCM checkouts, build other projects (workflow or freestyle), wait for external conditions, and so on. Plugins can add further steps.
If Jenkins is restarted (intentionally, or because of a crash) while your workflow is running, when it comes back up, execution is resumed where it left off. This applies to external processes (shell scripts) so long as the slave can be reattached, and losing the slave connection temporarily is not fatal either.
Flows can pause in the middle and wait for a human to approve something, or enter some information. Executors need not be consumed while the flow is waiting.
Workflows can be divided into sequential stages, not only for labeling but to throttle concurrency.
Read the tutorial to get started writing workflows.
Releases are available on the Jenkins update center. You need to be running a sufficiently recent Jenkins release: an LTS in the 1.580.x line or newer (currently 1.596.x for the latest updates), or a weekly release. See the changelog for news.
For OSS Jenkins users, install Workflow: Aggregator (its dependencies will be pulled in automatically). You will need to restart Jenkins to complete installation.
Jenkins Enterprise by CloudBees users get Workflow automatically as of the 14.11 (1.580.1.1) release. Otherwise install CloudBees Workflow: Aggregator from the update center. Again dependencies will be pulled in automatically, including all the OSS plugins.
- Changelog
- jenkins-workflow tag on StackOverflow
- JIRA (file issues in the
workflow-plugincomponent, or other components with theworkflowlabel) - User list discussions (mention
workflowin the subject) - #JenkinsWorkflow on Twitter
See the demo overview using Docker if you want to try a complete setup quickly. In short:
docker run -p 8080:8080 -p 8081:8081 -p 8022:22 -ti jenkinsci/workflow-demo
and browse localhost:8081.
Workflow Meetup London (Mar 2015): slides
Webinar Orchestrating the Continuous Delivery Process in Jenkins with Workflow (Dec 2014): video
JUC San Francisco (Oct 2014): video
Reusing build steps from freestyle projects
Using version control from a workflow
- Ongoing plugin compatibility list
- Source repository
- Open pull requests
- CI job
- Video tutorial on implementing a Step API
- Video walkthrough of code
If you want to try running recent development changes, rather than released binaries, you have two options. You can run directly from the source tree; from the root of the repository:
mvn -DskipTests clean install && mvn -f aggregator hpi:run
Then visit http://localhost:8080/jenkins/ to play with the plugins.
(If your IDE supports compile-on-save mode this is especially convenient since each hpi:run will pick up compiled changes from member plugins without needing to run to package phase.)
You can also run the Docker demo with snapshot binaries:
make -C demo run-snapshot
The snapshot Docker demo is mainly useful for verifying the effect of ongoing changes on future demo binary releases. You get the cd sample job set up, but your environment is thrown away if you kill the Docker container (for example with Ctrl-C). When using hpi:run the same aggregator/work/ home directory is reused so long as you do not explicitly delete it.
While the implementation is divided into a number of plugins, for ease of prototyping they are all kept in one repository using snapshot dependencies.
step-apidefines a generic build step interface (not specific to flows) that many plugins could in the future depend on.basic-stepsadd some generic step implementations. There is more documentation there.apidefines the essential aspects of flows and their executions. In particular, the engine running a flow is extensible and so could in the future support visual orchestration languages.supportadds general implementations of some internals needed by flows, such as storing state.jobprovides the actual job type and top-level UI for defining and running flows.durable-task-stepuses thedurable-taskplugin to define a shell script step that can survive restarts.scm-stepadds SCM-related steps. There is more documentation there.cpsis the flow engine implementation based on the Groovy language, and supporting long-running flows using a continuation passing style transformation of the script.cps-global-libadds a Git-backed repository for Groovy libraries available to scripts.stmis a simple engine implementation using a state transition machine, less intended for end users than as a reference for how engines can work. Currently only partly implemented.aggregatoris a placeholder plugin allowing you tomvn hpi:runand see everything working together, as well as holding integration tests.