Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions _build_cfg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,8 @@ Topics:
File: sti
- Name: Testing S2I Images
File: sti_testing
- Name: Custom Builder
File: custom

---
Name: Using Images
Expand Down
35 changes: 20 additions & 15 deletions architecture/core_objects/builds.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,33 +10,38 @@
toc::[]

== Overview
A build is the end result of transforming input parameters, typically source code, into a resulting object, typically a runnable image.
A build is the process of transforming input parameters, typically source code,
into a resulting object, typically a runnable image.

For a list of build commands, see the
link:../../dev_guide/builds.html[Developer's Guide].

== Build Strategies

There are three available link:openshift_model.html#build-strategies[build strategies]:
There are three link:openshift_model.html#build-strategies[build strategies] available:

* link:#docker-build[Docker build]
* link:#source-build[Source-to-Image build]
* link:#source-build[Source-to-Image (S2I) build]
* link:#custom-build[Custom build]

The resulting object depends on the builder used to create the image.
The resulting object depends on the builder used to create it, for Docker and S2I builds
those will be runnable images, for Custom build - whatever the builder image author
wants.

[#docker-build]
=== Docker Build
Docker builds invoke the plain https://docs.docker.com/reference/commandline/cli/#build[docker build command], and therefore expect a repository with a *_Dockerfile_* and all required directories.
Docker build invokes the plain https://docs.docker.com/reference/commandline/cli/#build[docker build]
command, and therefore expects a repository with a *_Dockerfile_* and all required
artifacts in it to produce a runnable image.

[#source-build]
=== Source-to-Image Build
link:../../creating_images/sti.html[Source-to-Image (S2I)] is a tool for
building reproducible Docker images. It produces ready-to-run images by
injecting a user source into a docker image and assembling a new docker image.
The new image incorporates the base image and built source, and is ready to use
with the `docker run` command. S2I supports incremental builds, which re-use
previously downloaded dependencies, previously built artifacts, etc.
The new image incorporates the base image (the builder) and built source, and is
ready to use with the `docker run` command. S2I supports incremental builds, which
re-use previously downloaded dependencies, previously built artifacts, etc.

The advantages of S2I include:

Expand All @@ -49,8 +54,8 @@ source, so the image needs to be able to process tarred content.
Speed:: With S2I, the assemble process can perform a large number of complex
operations without creating a new layer at each step, resulting in a fast
process. In addition, S2I scripts can be written to re-use artifacts stored in a
previous version of the application image rather than having to download or
build them each time the build is run.
previous version of the application image, rather than having to download or
build them, each time the build is run.

Patchability:: S2I allows you to rebuild the application consistently if an
underlying image needs a patch due to a security issue.
Expand All @@ -63,7 +68,7 @@ Operational security:: Building an arbitrary *_Dockerfile_* exposes the host
system to root privilege escalation. This can be exploited by a malicious user
because the entire docker build process is run as a user with docker privileges.
S2I restricts the operations performed as a root user, and can run the scripts
as an non-root user.
as a non-root user.

User efficiency:: S2I prevents developers from performing arbitrary `yum
install` type operations during their application build, which results in slow
Expand All @@ -74,7 +79,7 @@ best practices for your applications.

[#custom-build]
=== Custom Build
Custom builds allow developers to define the specific builder image responsible
for the entire build process. The custom builder image is a plain Docker image
with embedded build process logic, such as building RPMs or building base Docker
images.
link:../../creating_images/custom.html[Custom build] allows developers to define
a specific builder image, responsible for the entire build process. The custom
builder image is a plain Docker image with embedded build process logic, such as
building RPMs or building base Docker images.
40 changes: 40 additions & 0 deletions creating_images/custom.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
= Custom builder
{product-author}
{product-version}
:data-uri:
:icons:
:experimental:
:toc: macro
:toc-title:

toc::[]

== Overview
link:../architecture/core_objects/builds.html#custom-build[Custom build] is designed
to fill the gap that was created when everybody jumped into creating docker images.
Still there is a requirement to produce packages with all the software being installed
and most importantly one needs to be able to build base docker images. This is
where Custom build is the perfect match to fill in that gap.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't really how i'd describe custom builder... the role it fills is basically anything we may not have anticipated in our STI and Docker builders. In addition the most likely use case is not to produce a new docker image (use Docker or STI builds for that), it's to produce individual artifacts like jars, wars, installable zips, etc. The point is that it can perform any steps you want to produce some output artfact, and then is responsible for pushing that artifact to some location (eg scp'ing it to a server).

It also provides a backdoor way of introducing a CI/CD flow since you could write a custom builder that runs unit or integration tests, in theory. (sort of like the Jenkins custom job type)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update this in a follow up PR.
On Jun 15, 2015 6:58 PM, "Ben Parees" notifications@github.com wrote:

In creating_images/custom.adoc
#492 (comment)
:

+{product-author}
+{product-version}
+:data-uri:
+:icons:
+:experimental:
+:toc: macro
+:toc-title:
+
+toc::[]
+
+== Overview
+link:../architecture/core_objects/builds.html#custom-build[Custom build] is designed
+to fill the gap that was created when everybody jumped into creating docker images.
+Still there is a requirement to produce packages with all the software being installed
+and most importantly one needs to be able to build base docker images. This is
+where Custom build is the perfect match to fill in that gap.

this isn't really how i'd describe custom builder... the role it fills is
basically anything we may not have anticipated in our STI and Docker
builders. In addition the most likely use case is not to produce a new
docker image (use Docker or STI builds for that), it's to produce
individual artifacts like jars, wars, installable zips, etc. The point is
that it can perform any steps you want to produce some output artfact, and
then is responsible for pushing that artifact to some location (eg scp'ing
it to a server).

It also provides a backdoor way of introducing a CI/CD flow since you
could write a custom builder that runs unit or integration tests, in
theory. (sort of like the Jenkins custom job type)


Reply to this email directly or view it on GitHub
https://github.com/openshift/openshift-docs/pull/492/files#r32441856.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


To fully utilize the power of Custom build one needs to be understand how to
create a builder image that will be capable of building desired objects.

== Custom builder image
The builder image upon invocation receives following environment variables with
the information needed to proceed with the build:

.Custom Builder Environment Variables
[cols="4a,6a",options="header"]
|===

|Variable name |Description

|`*BUILD*`
|This variable specifies the entire serialized link:../rest_api/openshift_v1.html#v1-build[Build] object.

|`*SOURCE_REPOSITORY*`
|This variable specifies the URL to a repository with sources to build.

|`*DOCKER_SOCKET*`
|This variable specifies the path to docker socket, if exposing docker socket was enabled on BuildConfig.
|===
56 changes: 29 additions & 27 deletions creating_images/sti.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,12 @@ code as an input and produce a new image that runs the assembled application as
output.

The main advantage of using S2I for building reproducible Docker images is the
ease of use for developers. As a builder image author, you must be aware of the
two basic requirements for the best possible S2I performance: the required image
contents and S2I scripts.
ease of use for developers. As a builder image author, you must understand two
basic concepts in order for your images to provide the best possible S2I performance:
link:#build-process[the build process] and link:#s2i-scripts[S2I scripts].

== Required Image Contents
[#build-process]
== Build Process
The build process consists of the following three fundamental elements, which
are combined into a final Docker image:

Expand All @@ -36,11 +37,11 @@ location specified with the `--destination` flag or the `*io.s2i.destination*`
label from the builder image, with the default location being the
*_/tmp_* directory.

For this *tar* process to happen, your image must supply the *tar* archiving
For this process to happen, your image must supply the *tar* archiving
utility (the `tar` command available in `*$PATH*`) and the command line
interpreter (the `/bin/sh` command); this allows your image to use the fastest
possible build path. If the `tar` or `/bin/sh` command is not available, the
`sti build` script is forced to automatically perform an additional Docker build
`sti build` process is forced to automatically perform an additional Docker build
to put both the sources and the scripts inside the image, and only then run the
usual `sti build` procedure.

Expand All @@ -49,12 +50,12 @@ See the following diagram for the basic S2I build workflow:
.Build Workflow
image::sti-flow.png[S2I workflow]

////
* Run build's responsibility is to untar the sources, scripts and artifacts (if such exist) and invoke `assemble` script. If this is second run (after catching `tar`/`/bin/sh` error) it's responsible only for invoking `assemble` script, since both scripts and sources are already there.
////
* Run build's responsibility is to untar the sources, scripts and artifacts (if such exist) and invoke the `assemble` script. If this is the second run (after catching `tar`/`/bin/sh` not found error) it is responsible only for invoking `assemble` script, since both scripts and sources are already there.


[#s2i-scripts]
== S2I Scripts
You can write S2I scripts in any programming language as long as the scripts are
You can write S2I scripts in any programming language, as long as the scripts are
executable inside the builder image. S2I supports multiple options providing
`assemble`/`run`/`save-artifacts` scripts. All of these locations are checked on
each build in the following order:
Expand All @@ -63,7 +64,7 @@ each build in the following order:
2. A script found in the application source `.sti/bin` directory
3. A script found at the default image URL (`io.s2i.scripts-url` label)

Both `io.s2i.scripts-url` label specified in the image and `--scripts-url` flag
Both the `io.s2i.scripts-url` label specified in the image and the `--scripts-url` flag
can take one of the following form:

- `image://path_to_scripts_dir` - absolute path inside the image to a directory where the S2I scripts are located
Expand All @@ -86,7 +87,7 @@ or `io.s2i.destination` label applies only to sources and artifacts.
and places them into appropriate directories inside the image. The workflow for
this script is:

. Restore build artifacts. If you want to support incremental builds, make sure to define *_save-artifacts_* as well.
. Restore build artifacts. If you want to support incremental builds, make sure to define *_save-artifacts_* as well (optional).
. Place the application source in the desired location.
. Build the application artifacts.
. Install the artifacts into locations appropriate for them to run.
Expand All @@ -100,8 +101,8 @@ this script is:
|The *_save-artifacts_* script gathers all dependencies that can speed up the
build processes that follow. For example:

- For Ruby, *gems* is installed by Bundler.
- For Java, *.m2* contents are installed.
- For Ruby, *gems* installed by Bundler.
- For Java, *.m2* contents.

These dependencies are gathered into a tar file and streamed to the standard
output.
Expand All @@ -119,16 +120,15 @@ image is working correctly. The proposed flow of that process is:
. Build the image.
. Run the image to verify the *_usage_* script.
. Run `sti build` to verify the *_assemble_* script.
. Run `sti build` again to verify the *_save-artifacts_* script and the *_usage_* script's restore artifacts functionality. (optional)
. Run `sti build` again to verify the *_save-artifacts_* and *_assemble_* scripts save and restore artifacts functionality. (optional)
. Run the image to verify the test application is working.

See the link:sti_testing.html[Testing S2I Images] topic for more information.

NOTE: The suggested location to put the test application built by your
*_test/run_* script is the *_test/test-app_* directory in your image repository.
See the
https://github.com/openshift/source-to-image/blob/master/docs/cli.md#sti-create[S2I
documentation] for more information.
See the https://github.com/openshift/source-to-image/blob/master/docs/cli.md#sti-create[S2I documentation]
for more information.
|===

*Example S2I Scripts*
Expand Down Expand Up @@ -213,19 +213,21 @@ example:
See the https://docs.docker.com/reference/builder/#onbuild[Docker documentation]
for more information on `ONBUILD`.

S2I has a different strategy when a Docker image with `ONBUILD` instructions is
used as a builder image for the application source code. During the S2I build,
Upon start S2I detects whether the builder image uses `ONBUILD` instructions.
If there are none, the regular S2I link:#build-process[build] is performed,
otherwise a different strategy is chosen. During such a S2I build,
all `ONBUILD` instructions are executed in the order they were defined in the
builder image Dockerfile. The S2I scripts are not required for this strategy,
but they can be used as supplementary scripts to existing `ONBUILD`
instructions.
builder image's Dockerfile. The S2I scripts are not required for this strategy,
but they can be used as a supplement to existing `ONBUILD` instructions.

Many official Docker images that use `ONBUILD` do not declare the image `CMD` or
`ENTRYPOINT`, and for that, S2I must know how to run your application. There are
two methods for defining the `ENTRYPOINT`:

- Include the *_run_* script in your application root folder. S2I then recognizes it and sets it as the application image `ENTRYPOINT`.
- Include the *_run_* script in your application root folder. S2I recognizes it
and sets it as the application image `ENTRYPOINT`.

- Use the S2I scripts. If you provide the URL from where the S2I can fetch the scripts, the S2I *_run_* script is then
set as an image `ENTRYPOINT`. If the S2I scripts location also includes the *_assemble_* script, the script is then
executed as the last instruction of the Docker build.
- Use the link:#s2i-scripts[S2I scripts]. If you provide them, the *_run_* script
is set as an image `ENTRYPOINT`. If the S2I scripts location also includes the
*_assemble_* script, that script is executed as the last instruction of the underlying
Docker build.
43 changes: 23 additions & 20 deletions creating_images/sti_testing.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -36,19 +36,19 @@ invoked by the OpenShift S2I image builder and it could be a simple Bash script
or a static Go binary.

The *_test/run_* script performs the S2I build, so you must have the S2I binary
available on your system. If required, follow the installation instructions in
the S2I
https://github.com/openshift/source-to-image/blob/master/README.md#installation[README].
available in your `*$PATH*`. If required, follow the installation instructions in
the https://github.com/openshift/source-to-image/blob/master/README.md#installation[S2I README].

S2I combines the application source code and Docker image, so in order to test
S2I combines the application source code and builder image, so in order to test
it you need a sample application source to verify that the source successfully
converts into a Docker image. The sample application should be simple, but it
should also exercise the `assemble` command.
transforms into a runnable Docker image. The sample application should be simple,
but it should exercise the crucial steps of `assemble` and `run` scripts.

== Generating Scripts and Tools
The S2I tooling comes with powerful generation tools to speed up the process of
creating a new S2I image. The `sti create` command produces all necessary S2I
creating a new S2I image. The `sti create` command produces all the necessary S2I
scripts and testing tools along with the *_Makefile_*:

****
`$ sti create _<image name>_ _<destination directory>_`
****
Expand Down Expand Up @@ -84,7 +84,7 @@ test:
====

== Basic Testing Workflow
The *_test_* script assumes you have already built the image that you want to
The *_test_* script assumes you have already built the image you want to
test. If required, first build the S2I image using:

****
Expand All @@ -98,7 +98,7 @@ The following steps describe the default workflow to test S2I image builders:
====

****
`$ docker run _<BUILDER_IMAGE_NAME>_`
`$ docker run _<BUILDER_IMAGE_NAME>_ .`
****
====

Expand All @@ -112,7 +112,8 @@ The following steps describe the default workflow to test S2I image builders:
****
====

. If you support *_save-artifacts_*, then execute step 2 again to verify that restoring artifacts works properly.
. Optionally, if you support *_save-artifacts_*, execute step 2 once again to
verify that saving and restoring artifacts works properly.

. Run the container:
+
Expand All @@ -125,16 +126,17 @@ The following steps describe the default workflow to test S2I image builders:

. Verify the container is running and the application is responding.

Executing these steps is generally enough to tell if the S2I scripts are
operating properly.
Executing these steps is generally enough to tell if the builder image is
working as expected.

== Using OpenShift Build for Automated Testing
Another way you can execute the S2I image tests is to use the OpenShift platform
itself as a continuous integration system. The OpenShift platform is capable of
building Docker images and is highly customizable.

To set up an S2I image builder continuous integration system, define a special `*CustomBuild*` and use the
*openshift/sti-image-builder* image. This image executes all the steps mentioned
To set up an S2I image builder continuous integration system, define a
link:../architecture/core_objects/builds.html#custom-build[Custom build] and use
the *openshift/sti-image-builder* image. This image executes all the steps mentioned
in the link:#basic-testing-workflow[Basic Testing Workflow] section and creates
a new S2I builder image.

Expand Down Expand Up @@ -193,6 +195,7 @@ a new S2I builder image.
}

----

====

You can use the `oc create` command to create this `*BuildConfig*`. After you create the `*BuildConfig*`, you can start the build using the following command:
Expand All @@ -204,10 +207,10 @@ You can use the `oc create` command to create this `*BuildConfig*`. After you cr
****
====

If your OpenShift instance is hosted on a public IP address, then the build is
triggered each time you push into your S2I builder image GitHub repository.
If your OpenShift instance is hosted on a public IP address, the build can be
triggered each time you push into your S2I builder image GitHub repository. See
link:../dev_guide/builds.html#webhook-triggers[webhook triggers] for more information.

You can also use the `*CustomBuild*` to trigger a rebuild for your applications
based on the S2I image you updated. To do this you must specify the `Output`
field in the `parameters` section and define to which Docker registry the image
should be pushed after a successful build.
You can also use the `*CustomBuild*` to trigger a rebuild of your application
based on the S2I image you updated. See link:../dev_guide/builds.html#image-change-triggers[image change triggers]
for more information.
Loading