diff --git a/dev_guide/builds.adoc b/dev_guide/builds.adoc index 875f86b356eb..0bd9cab228d8 100644 --- a/dev_guide/builds.adoc +++ b/dev_guide/builds.adoc @@ -10,8 +10,9 @@ toc::[] == Overview -A link:../architecture/core_objects/builds.html[build] is a process of creating -runnable images to be used on OpenShift. There are three build strategies: +A link:../architecture/core_objects/builds.html[build] is the end result of the +process of creating runnable images to be used on OpenShift. There are three +types of build strategies: - link:../architecture/core_objects/builds.html#source-build[Source-To-Image] - link:../architecture/core_objects/builds.html#docker-build[Docker] @@ -19,16 +20,15 @@ runnable images to be used on OpenShift. There are three build strategies: == Defining a buildConfig -A `*buildConfig*` describes a single build definition and a -set of link:#triggers[triggers] for when a new build should be created. +A `*buildConfig*` describes a single build definition and a set of +link:#triggers[triggers] for when a new build is to be created. A buildConfig is a REST object, which can be used in a POST to the API server to -create a new instance. The following configuration results in a new build -every time a Docker image tag or the source code changes: +create a new instance. The following is an example buildConfig, which results +in a new build each time a Docker image tag or source code changes: +[source.json] ==== - -[source,json] ---- { "kind": "BuildConfig", @@ -80,22 +80,29 @@ every time a Docker image tag or the source code changes: ---- <1> This specification will create a new buildConfig named ruby-sample-build. -<2> You can specify a list of triggers, which causes a new build to be created. -The `*ImageChange*` trigger creates a new build each time the Docker image specified in the `*strategy.from*` field changes. The `*github*` and `*generic*` triggers starts the build when a callback is made from the GitHub or other source code repository hosting service, indicating change in the source code specified in `*source*` section. -<3> The `*source*` section defines the source code repository location. You can provide additional options, such as `*sourceSecret*` or `*contextDir*` here. -<4> The `*strategy*` section describes the build strategy used to execute the build. You can specify `*Source*`, `*Docker*` and `*Custom*` strategies here. This specification uses the `*ruby-20-centos7*` Docker image that Source-To-Image will use for the application build. -<5> After the Docker image is successfully built, it will be pushed into the repository described in the `*output*` section. +<2> You can specify a list of link:#build-triggers[triggers], which causes a new build to be created. +The `*ImageChange*` trigger creates a new build each time the Docker image in +the `*strategy.from*` field changes. The `*github*` and `*generic*` triggers +start the build when a callback is made from the GitHub or other source code +repository hosting service, indicating change in the source code specified in +`*source*` section. +<3> The `*source*` section defines the source code repository location. You can +provide additional options, such as `*sourceSecret*` or `*contextDir*`, here. +<4> The `*strategy*` section describes the build strategy used to execute the +build. You can specify `*Source*`, `*Docker*`, and `*Custom*` strategies here. +The above example uses the `*ruby-20-centos7*` Docker image that +Source-To-Image will use for the application build. +<5> After the Docker image is successfully built, it will be pushed into the +repository described in the `*output*` section. ==== == Defining an incremental buildConfig -Source-To-Image (S2I) builds provide an option to perform incremental build, which means they will -reuse artifacts from previously built image. To create an incremental build one -needs to create a BuildConfig as previously explained, but with a slight modification -in strategy definition as presented below: - -==== +Source-To-Image (S2I) builds provide an option to perform incremental builds, +meaning they will reuse artifacts from previously built images. To create an +incremental build create a BuildConfig much like the above example, but +with a modification to the strategy parameter: -[source,json] +[incremental.json] ---- { "strategy": { @@ -111,51 +118,57 @@ in strategy definition as presented below: } ---- -<1> You need to specify here an image that supports incremental builds. Note: the S2I images provided by OpenShift do not implement artifact reuse, so setting incremental to true will have no effect on BuildConfigs using those builder images. -<2> This flag controls whether an incremental build is attempted. If the builder image does not support incremental builds, the build will still succeed but you will get a log message that the incremental build was not successful because of a missing `save-artifacts` script. -==== +<1> Specify the image that supports incremental builds. The S2I images provided +by OpenShift do not implement artifact reuse, so setting the `*incremental*` +value to true will have no effect on a buildConfig using those builder images. +<2> This flag controls whether incremental builds are attempted. If the builder +image does not support incremental builds, the build will still succeed, but you +will receive a log message saying the incremental build was not successful +because of a missing `save-artifacts` script. -NOTE: For information on how to create a builder image which supports incremental build see link:../creating_images/sti.html[creating images guide]. +[NOTE] +For information on how to create a builder image that supports incremental build +see the link:../creating_images/sti.html[creating images guide]. == Starting a Build You can manually invoke a build using the following command: -**** -`$ oc start-build __` -**** +---- +$ oc start-build +---- A build can be re-run using the `--from-build` flag: -**** -`$ oc start-build --from-build=__` -**** +---- +$ oc start-build --from-build= +---- Specify the `--follow` flag to stream the build's logs in stdout: -**** -`$ oc start-build __ --follow` -**** +---- +$ oc start-build --follow +---- == Canceling a Build -You can manually cancel a build using the following command: +Manually cancel a build using the following command: -**** -`$ oc cancel-build __` -**** +---- +$ oc cancel-build +---- == Accessing Build Logs To allow access to build logs, use the following command: -**** -`$ oc build-logs __` -**** +---- +$ oc build-logs +---- *Source Build Logs* -link:../architecture/core_objects/builds.html#source-build[Source builds] by default -show full output of the *_assemble_* script and all subsequent errors. -To enable more verbose output, you can pass the `*BUILD_LOGLEVEL*` -environment variable as part of the `*sourceStrategy*` in BuildConfig: +By default, link:../architecture/core_objects/builds.html#source-build[Source +builds] show the full output of the *_assemble_* script and all subsequent +errors. To enable a more verbose output, pass the `*BUILD_LOGLEVEL*` environment +variable as part of the `*sourceStrategy*` in the buildConfig: ==== @@ -176,32 +189,35 @@ environment variable as part of the `*sourceStrategy*` in BuildConfig: <1> Adjust this value to the desired log level. ==== -NOTE: A platform administrator can set verbosity for the entire OpenShift -instance by passing the `--loglevel` option to the `openshift start` command. -If both `--loglevel` and `BUILD_LOGLEVEL` are specified, `BUILD_LOGLEVEL` takes precedence. +[NOTE] +A platform administrator can set verbosity for the entire OpenShift instance by +passing the `--loglevel` option with the `openshift start` command. If both +`--loglevel` and `BUILD_LOGLEVEL` are specified, `BUILD_LOGLEVEL` takes +precedence. Available log levels for Source builds are as follows: [horizontal] -Level 0:: Produces output from containers running the *_assemble_* script and all encountered errors. This is the default. +Level 0:: Produces output from containers running the *_assemble_* script and +all encountered errors. This is the default. Level 1:: Produces basic information about the executed process. Level 2:: Produces very detailed information about the executed process. -Level 3:: Produces very detailed information about the executed process, and a listing of the archive contents. +Level 3:: Produces very detailed information about the executed process, and a +listing of the archive contents. == Source Code The source code location is one of the required parameters for the -`*BuildConfig*`. The build uses this location and fetches the source code that -is later built. The source code location definition is part of the -`*parameters*` section in the `*BuildConfig*`: +`*buildConfig*`. The build uses this location and fetches the source code to +build. The source code location definition is part of the `*parameters*` section +in the `*buildConfig*`: ==== - ---- { "source" : { "type" : "Git", <1> - "git" : { <2> - "uri": "git://github.com/openshift/ruby-hello-world.git" + "git" : { + "uri": "git://github.com/openshift/ruby-hello-world.git" <2> }, "contextDir": "app/dir", <3> }, @@ -209,12 +225,11 @@ is later built. The source code location definition is part of the ---- <1> The `*type*` field describes what SCM is used to fetch your source code. -<2> In this example, the `*git*` field contains the URI to the remote Git -repository where your source code lives. It might optionally specify the `*ref*` -field if you want to check out a specific Git reference. A valid `*ref*` can be -a SHA1 tag or a branch name. +<2> The `*git*` field contains the URI to the remote Git repository of the +source code. Optionally, you can specify the `*ref*` field to checkout a +specific Git reference. A valid `*ref*` can be a SHA1 tag or a branch name. <3> The `*contextDir*` field allows you to override the default location inside -the source code repository, where the build looks for the application source +the source code repository where the build looks for the application source code. If your application exists inside a sub-directory, you can override the default location (the root folder) using this field. ==== @@ -222,60 +237,59 @@ default location (the root folder) using this field. [[configuring-the-source-environment]] == Source Environment -There are two ways to make environment variables available to the link:../architecture/core_objects/builds.html#source-build[Source] -build process and resulting image, Environment files and BuildConfig environment values. +There are two ways to make environment variables available to the +link:../architecture/core_objects/builds.html#source-build[Source] build process +and resulting image: Environment files and BuildConfig environment values. === Environment files -Source enables you to set environment values -in your application by specifying them in a *_.sti/environment_* file in the -source repository. The environment variables are then present during the build -process and in the final docker image. The complete list of supported -environment variables are available in the documentation for each image. +Source enables you to set environment values in your application by specifying +them in a *_.sti/environment_* file in the source repository. The environment +variables are then present during the build process and in the final docker +image. The complete list of supported environment variables are available in the +link:using_images/overview.html[documentation for each image]. If you provide a *_.sti/environment_* file in your source repository, S2I reads this file during the build. This allows customization of the build behavior as the *_assemble_* script may use these variables. -For example, if you want to disable assets compilation for your Rails -application, you can add `*DISABLE_ASSET_COMPILATION=true*` in the -*_.sti/environment_* file to cause assets compilation to be skipped during the -build. +For example, if you want to disable assets compilation for a Rails application, +add `*DISABLE_ASSET_COMPILATION=true*` to the *_.sti/environment_* file to cause +assets compilation to be skipped during the build. In addition to builds, the specified environment variables are also available in the running application itself. For example, you can add -`*RAILS_ENV=development*` in the *_.sti/environment_* file to cause the Rails -application to be started in `development` mode instead of `production`. +`*RAILS_ENV=development*` to the *_.sti/environment_* file to cause the Rails +application to start in `development` mode instead of `production`. === BuildConfig Environment -You can add environment variables to the SourceStrategy definition of the BuildConfig. -Environment variables defined here will be visible during the *_assemble_* script -execution and will be defined in the output image, making them also available to -the *_run_* script and application code. +You can add environment variables to the SourceStrategy definition of the +buildConfig. Defined Environment variables will be visible during the +*_assemble_* script execution and will be defined in the output image, making +them also available to the *_run_* script and application code. == Build Triggers -When defining a `*BuildConfig*`, you can define triggers to control the -circumstances in which a build should be run for the `*BuildConfig*`. There are two -types of triggers available: +When constructing a `*buildConfig*`, you can define triggers to control the +circumstances in which the `*buildConfig*` should be run. There are two types of +triggers available: * Webhook * Image change === Webhook Triggers Webhook triggers allow you to trigger a new build by sending a request to the -OpenShift API endpoint. You can define these triggers using -https://developer.github.com/webhooks/[GitHub webhooks] or generic webhooks. +OpenShift API endpoint. Define these triggers using +https://developer.github.com/webhooks/[GitHub] or generic webhooks. *GitHub Webhooks* -https://developer.github.com/webhooks/creating/[GitHub webhooks] can handle the -call made by GitHub when a repository is updated. When defining the trigger, you -must specify a *secret* as part of the URL you supply to GitHub when -configuring the webhook. The *secret* ensures that only you and your -repository can trigger the build. The following example is a trigger definition -JSON within the `*BuildConfig*`: +https://developer.github.com/webhooks/creating/[GitHub webhooks] handle the call +made by GitHub when a repository is updated. When defining the trigger, you must +specify a link:architecture/core_objects/kubernetes_model.html#secret[*secret*] +as part of the URL you supply to GitHub when configuring the webhook. The +*secret* ensures that only you and your repository can trigger the build. The +following example is a trigger definition JSON within the `*BuildConfig*`: ==== - ---- { "type": "github", @@ -289,19 +303,18 @@ JSON within the `*BuildConfig*`: The payload URL is returned as the GitHub Webhook URL by the `describe` command (see link:#describe-buildconfig[below]), and is structured as follows: -**** -`http://__/osapi/v1/namespaces/__/buildconfigs/__/webhooks/__/github` -**** +---- +http:///osapi/v1/namespaces//buildconfigs//webhooks//github +---- *Generic Webhooks* Generic webhooks can be invoked from any system capable of making a web request. As with a GitHub webhook, you must specify a *secret* when defining the trigger, and the caller must provide this *secret* to trigger the build. The -following is an example trigger definition JSON within the `*BuildConfig*`: +following is an example trigger definition JSON within the `*buildConfig*`: ==== - ---- { "type": "generic", @@ -315,14 +328,13 @@ following is an example trigger definition JSON within the `*BuildConfig*`: To set up the caller, supply the calling system with the URL of the generic webhook endpoint for your build: -**** -`http://__/osapi/v1/namespaces/__/buildconfigs/__/webhooks/__/generic` -**** +---- +http:///osapi/v1/namespaces//buildconfigs//webhooks//generic +---- The endpoint can accept an optional payload with the following format: ==== - ---- { type: 'git', @@ -350,9 +362,9 @@ The endpoint can accept an optional payload with the following format: Use the following command to display the webhook URLs associated with a build configuration: -**** -`oc describe buildConfig __` -**** +---- +$ oc describe buildConfig +---- If the above command does not display any webhook URLs, then no webhook trigger is defined for that build configuration. @@ -366,32 +378,29 @@ latest RHEL base image. Configuring an image change trigger requires the following actions: -1. Define an `*ImageStream*` that points to the upstream image you want to +. Define an `*ImageStream*` that points to the upstream image you want to trigger on: + ==== - ---- { + "metadata":{ + "name": "ruby-20-centos7", + }, "kind": "ImageStream", - "apiVersion": "v1", - "metadata": { - "name": "ruby-20-centos7" - } + "apiVersion": "v1beta1", } ---- ==== + This defines the image stream that is tied to a Docker image repository -located at `__/__/ruby-20-centos7`. The -`__` is defined as a service with the name `docker-registry` +located at `//ruby-20-centos7`. The +`` is defined as a service with the name `docker-registry` running in OpenShift. -2. Define a build with a strategy that consumes the image stream; for -example: +. Define a build with a strategy that consumes the image stream. For example: + ==== - ---- { "strategy": { @@ -410,7 +419,7 @@ example: In this case, the Source strategy definition is consuming the `latest` tag of the ImageStream named `ruby-20-centos7` located within this namespace. -3. Define an image change trigger: +. Define an image change trigger: + ==== @@ -422,11 +431,11 @@ ImageStream named `ruby-20-centos7` located within this namespace. ---- ==== + -This defines an image change trigger which monitors `*ImageStream*` and `*Tag*` defined -by the Strategy's From field. When a change occurs, a new build is triggered -and is supplied with an immutable Docker tag that points to the new image that -was just created. This new image will be used by the Strategy when it executes -for the build. For example, the resulting build will be: +This defines an image change trigger that monitors `*ImageStream*` and `*Tag*` +defined by the Strategy's `*From*` field. When a change occurs, a new build is +triggered and is supplied with an immutable Docker tag that points to the new +image that was just created. This new image will be used by the Strategy when +it executes for the build. For example, the resulting build will be: ==== @@ -446,29 +455,28 @@ for the build. For example, the resulting build will be: ==== This ensures that the triggered build uses the new image that was just pushed to -the repository, and the build can be re-run anytime with exactly the same -inputs. +the repository, and the build can be re-run anytime with the same inputs. -In addition to setting the image field for all `*Strategy*` types, for custom builds, -the `OPENSHIFT_CUSTOM_BUILD_BASE_IMAGE` environment variable is checked. If it does -not exist, then it is created with the immutable image reference. If it does exist -then it is updated with the immutable image reference. +In addition to setting the image field for all `*Strategy*` types, for custom +builds, the `OPENSHIFT_CUSTOM_BUILD_BASE_IMAGE` environment variable is checked. +If it does not exist, then it is created with the immutable image reference. If +it does exist then it is updated with the immutable image reference. If a build is triggered due to a webhook trigger or manual request, the build that is created uses the `*immutableid*` resolved from the -`*ImageStream*` referenced by the `*Strategy*`. This ensures that builds +`*ImageStream*` referenced by the `*Strategy*`. This ensures that builds are performed using consistent image tags for ease of reproduction. -[#using-docker-credentials-for-pushing-and-pulling-images] -== Using Docker Credentials for Pushing and Pulling Images - -Supply the `.dockercfg` file with valid Docker Registry credentials in order to push the output image into a private Docker Registry or pull the -builder image from the private Docker Registry that requires authentication. -For the OpenShift Docker Registry, you don't have to do this because the Secrets -are generated automatically for you by OpenShift. +[#using-docker-credentials-to-push-and-pull-images] +== Using Docker Credentials to Push and Pull Images +Supply the `.dockercfg` file with valid Docker Registry credentials in order to +push the output image into a private Docker Registry or pull the builder image +from the private Docker Registry that requires authentication. For the OpenShift +Docker Registry, you don't have to do this because the Secrets are generated +automatically for you by OpenShift. -The *_.dockercfg_* JSON file exists in your home directory by default and has -following format: +The *_.dockercfg_* JSON file is found in your home directory by default and has +the following format: ==== @@ -488,38 +496,35 @@ following format: You can define multiple Docker registry entries in this file. Alternatively, you can also add authentication entries to this file by running the `docker login` -command. The file will be created if it does not exist. - -Kubernetes provides the https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secret] -resource, which is used to store your configuration and passwords. - -To create the `*Secret*` resource from your local `.dockercfg` file, you can run -following command: +command. The file will be created if it does not exist. Kubernetes provides the +ability to create +https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secrets], +which are used to store your configuration and passwords. +. Create a `*Secret*` resource from your local `.dockercfg` file: ++ ==== ---- -$ oc secrets new dockerhub ~/.dockercfg +$ openshift ex bundle-secret dockerhub ~/.dockercfg | oc create -f - ---- ==== - ++ This command generates JSON specification of the Secret resource named 'dockerhub' and creates the object. -Once you have created your Secret, it must be added to the builder service account -in order to be used. - +. Once you have created your Secret, it must be added to the builder service +account in order to be used. ++ ==== ---- $ oc secrets add serviceaccount/builder secrets/dockerhub ---- ==== -Once you have the `*Secret*` created, you can add a `PushSecret` field into the -`Output` section of the `BuildConfig` and set it to the name of the `*Secret*` -that you created, which in the above example is `*dockerhub*`: - +. Add a `PushSecret` field into the `Output` section of the `BuildConfig` and +set it to the name of the `*Secret*` that you created, which in the above example is `*dockerhub*`: ++ ==== - ---- { "parameters": { @@ -536,11 +541,10 @@ that you created, which in the above example is `*dockerhub*`: ---- ==== -Pull the builder Docker image from a private Docker registry by specifying the -`PullSecret` field, which is part of the build strategy definition: - +. Pull the builder Docker image from a private Docker registry by specifying the +`pullSecret` field, which is part of the build strategy definition: ++ ==== - ---- { "strategy": { @@ -560,7 +564,7 @@ Pull the builder Docker image from a private Docker registry by specifying the ==== [#using-private-repositories-for-builds] -== Using Private Repositories for Builds +== Accessing Private Repositories for Builds using SSH Supply valid credentials to build an application from a private repository. Currently, only SSH key based authentication is supported. The repository keys @@ -569,67 +573,71 @@ are located in the `$HOME/.ssh/` directory, and are named `id_dsa.pub`, credentials with the following command: ==== - ---- $ ssh-keygen -t rsa -C "your_email@example.com" ---- ==== -Two files will be created: the public key (as explained above) and a -corresponding private key (one of `id_dsa`, `id_ecdsa`, `id_ed25519` or -`id_rsa`). With both of these in place you should consult your source control -management (SCM) system's manual on how to upload the public key. The private -key will be used to access your private repository. +Two files will be created: the public key and a corresponding private key (one +of `id_dsa`, `id_ecdsa`, `id_ed25519` or `id_rsa`). With both of these in place +consult your source control management (SCM) system's manual on how to upload +the public key. The private key will be used to access your private repository. The https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/design/secrets.md[Secret] -resource is used to store your keys. Create the `*Secret*` first before using -the SSH key to access the private repository. The `*data*` field for the -`*Secret*` object must contain your private key with the value set to the -base64-encoded content of that file: +resource is used to store your keys. +. Create the `*Secret*` first before using the SSH key to access the private +repository: ++ +==== +---- +$ osc create -f secret.json +---- ==== +. The `*data*` field for the +`*Secret*` object must contain your private key with the value set to the +base64-encoded content of that file: ++ +==== ---- $ base64 -w 0 $HOME/.ssh/id_rsa 6yJodHRwc1ovL2zuZGV4LmRvY21lci5aby92MS8iOnsiYXV0aCI6ImJXWnZhblJwYXpwdVoybGxkR2d4TUE9PSIsImVtYWlsIj8ibWlAbWlmby5zayJ9fQ== ---- ==== -Copy the value returned from the above command and place it into the +. Copy the value returned from the above command and place it into the `ssh-privatekey` field in `*_secret.json_*` file: - ++ ==== - ---- { + "apiVersion": "v1beta3", "kind": "Secret", - "apiVersion": "v1", "metadata": { "name": "scmsecret" }, "data": { "ssh-privatekey": "6yJodHRwc1ovL2zuZGV4LmRvY21lci5aby92MS8iOnsiYXV0aCI6ImJXWnZhblJwYXpwdVoybGxkR2d4TUE9PSIsImVtYWlsIj8ibWlAbWlmby5zayJ9fQ==" - }, - "type": "Opaque" + } } + ---- ==== -Then, create the `*Secret*` from the *_secret.json_* file using the following -command: - +. Create a second `*Secret*` for Docker from the *_secret.json_* file: ++ ==== - ---- $ oc create -f secret.json ---- ==== -Add a `SourceSecret` field into the `Source` section inside the `BuildConfig` +. Add a `SourceSecret` field into the `Source` section inside the `BuildConfig` and set it to the name of the `*Secret*` that you created, in this case `*scmsecret*`: - ++ ==== ---- @@ -647,7 +655,7 @@ and set it to the name of the `*Secret*` that you created, in this case }, "source": { "git": { - "uri": "git@repository.com:user/app.git" + "uri": "git@repository.com:user/app.git" <1> }, "sourceSecret": { "name": "scmsecret" @@ -665,9 +673,6 @@ and set it to the name of the `*Secret*` that you created, in this case } } ---- -==== - -[NOTE] -==== -The URL of private repository is usually in the form `git@example.com:username/repository` +<1> The URL of private repository is usually in the form +`git@example.com:username/repository` ====