Skip to content
This repository was archived by the owner on Jun 14, 2019. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 77 additions & 113 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,94 +4,62 @@ The CI Operator consumes a configuration file that describes how release artifac
are built from a repository's branch. An example file is shown below with all
optional fields present:

```json
{
"tag_specification": {
"cluster": "",
"namespace": "",
"name": "",
"tag": "",
"tag_overrides": {

}
},
"base_images": {
"": {
"cluster": "",
"namespace": "",
"name": "",
"tag": ""
}
},
"test_base_image": {
"cluster": "",
"namespace": "",
"name": "",
"tag": ""
},

"canonical_go_repository": "",

"binary_build_commands": "",
"test_binary_build_commands": "",
"rpm_build_commands": "",
"rpm_build_location": "",

"images": [
{
"from": "",
"to": "",
"context_dir": "",
"dockerfile_path": "",
"inputs": {
"": {
"as": "",
"paths": [
{
"source_path": "",
"destination_dir": ""
}
]
}
},
"optional": false
}
],

"tests": [
{
"as": "",
"from": "",
"commands": "",
"artifact_dir": ""
}
],

"raw_steps": [],

"promotion": {
"namespace": "",
"name": "",
"tag": "",
"name_prefix": "",
"additional_images": {
"": ""
}
},

"resources": {
"": {
"requests": {
"cpu": "",
"memory": ""
},
"limits": {
"cpu": "",
"memory": ""
}
}
}
}
```yaml
base_images:
<name>:
cluster: ''
name: ''
namespace: ''
tag: ''
binary_build_commands: ''
canonical_go_repository: ''
images:
- context_dir: ''
dockerfile_path: ''
from: ''
inputs:
<name>:
as: ''
paths:
- destination_dir: ''
source_path: ''
optional: false
to: ''
promotion:
additional_images:
<name>: ''
name: ''
name_prefix: ''
namespace: ''
tag: ''
raw_steps: []
resources:
<name>:
limits:
cpu: ''
memory: ''
requests:
cpu: ''
memory: ''
rpm_build_commands: ''
rpm_build_location: ''
tag_specification:
cluster: ''
name: ''
namespace: ''
tag: ''
tag_overrides: {}
test_base_image:
cluster: ''
name: ''
namespace: ''
tag: ''
test_binary_build_commands: ''
tests:
- artifact_dir: ''
as: ''
commands: ''
from: ''
```

# `tag_specification`
Expand All @@ -100,14 +68,13 @@ tagged into tests for the repository. The Origin CI assembles latest releases
for all components using one `ImageStream` and many tags. To use these releases,
use the following specification:

```json
"tag_specification": {
"cluster": "https://api.ci.openshift.org",
"namespace": "openshift",
"name": "origin-v3.11",
"tag": "",
"tag_overrides": {}
}
```yaml
tag_specification:
cluster: https://api.ci.openshift.org
name: origin-v3.11
namespace: openshift
tag: ''
tag_overrides: {}
```

There are two primary modes for assembling a release:
Expand Down Expand Up @@ -151,15 +118,13 @@ streams and one tag are used to assemble a release.
from the repository. The field is a mapping from pipeline image name to remote
`ImageStream` specification. A common base image might be an operating system:

```json
"base_images": {
"os": {
"cluster": "https://api.ci.openshift.org",
"namespace": "openshift",
"name": "centos",
"tag": "7"
}
}
```yaml
base_images:
os:
cluster: https://api.ci.openshift.org
name: centos
namespace: openshift
tag: '7'
```

The key in this mapping is the name that can be used in `"from"` fields elsewhere
Expand All @@ -178,13 +143,12 @@ be used as the build environment for the source code cloning and any downstream
builds like compilation or unit tests. Commonly, the `openshift/release` image
is used:

```json
"test_base_image": {
"cluster": "https://api.ci.openshift.org",
"namespace": "openshift",
"name": "release",
"tag": "golang-1.10"
}
```yaml
test_base_image:
cluster: https://api.ci.openshift.org
name: release
namespace: openshift
tag: golang-1.10
```

# `canonical_go_repository`
Expand Down Expand Up @@ -361,4 +325,4 @@ the test fails to `malloc()`.
`cpu` is the maximum CPU, with the unit, often in millicores (`100m`).

### `resources.$name.limits.memory`
`memory` is the maximum RAM, with the unit, often in MiB (`200Mi`).
`memory` is the maximum RAM, with the unit, often in MiB (`200Mi`).
77 changes: 29 additions & 48 deletions ONBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ as expected (you need to be logged in to a cluster, e.g. to
[api.ci](https://api.ci.openshift.org)):

```
./ci-operator --config config.json --git-ref openshift/<repo>@<revision>
./ci-operator --config config.yaml --git-ref openshift/<repo>@<revision>
```

After you make sure everything works, you need to create a subdirectory
Expand All @@ -42,22 +42,19 @@ and injects the source code into the base image specified by the
`test_base_image` key. The base image should contain all build dependencies of
the tested component, so the it will often be a `openshift/release:<tag>` image.

```json
{
"test_base_image": {
"cluster": "https://api.ci.openshift.org",
"namespace": "openshift",
"name": "release",
"tag": "golang-1.10"
}
}
```yaml
test_base_image:
cluster: https://api.ci.openshift.org
name: release
namespace: openshift
tag: golang-1.10
```

Given your component can be built in the context of the `openshift/release`
image, you can test building the `src` target:

```
$ ./ci-operator --config example.json --git-ref=openshift/<component>@<revision> --target=src
$ ./ci-operator --config example.yaml --git-ref=openshift/<component>@<revision> --target=src
```

### Test targets
Expand All @@ -68,21 +65,14 @@ example of two test targets, each performing a different test by calling
different `make` target in a `src` image (of course, a `Makefile` in your
component repository would need to have these targets for this to work).

```json
{
"tests": [
{
"as": "unit",
"from": "src",
"commands": "make test-unit"
},
{
"as": "performance",
"from": "src",
"commands": "make test-performance"
}
]
}
```yaml
tests:
- as: unit
commands: make test-unit
from: src
- as: performance
commands: make test-performance
from: src
```

By default, ci-operator runs all specified test targets, building all their
Expand All @@ -105,28 +95,19 @@ Here, `unit` and `integration` targets will both be built from a `test-bin`
image, which will be a result of running `make instrumented-build` over a `src`
image, while `performance` test target will be run from a `bin` image:

```json
{
"binary_build_commands": "make build",
"test_binary_builds_commands": "make instrumented-build",
"tests": [
{
"as": "unit",
"from": "test-bin",
"commands": "make test-unit"
},
{
"as": "integration",
"from": "test-bin",
"commands": "make test-integration",
},
{
"as": "performance",
"from": "bin",
"commands": "make test-performance"
}
]
}
```yaml
binary_build_commands: make build
test_binary_builds_commands: make instrumented-build
tests:
- as: unit
commands: make test-unit
from: test-bin
- as: integration
commands: make test-integration
from: test-bin
- as: performance
commands: make test-performance
from: bin
```

### Using Separate Build Environment and Release Environment Images
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ ci-operator is mainly intended to be run automatically by the CI system, but
after you build it, you can also run it locally:

```
./ci-operator --config component.json --git-ref=openshift/{repo}@master
./ci-operator --config component.yaml --git-ref=openshift/{repo}@master
```

For more information about ci-operator options, use the `--help` parameter:
Expand Down
9 changes: 5 additions & 4 deletions cmd/ci-operator-checkconfig/main.go
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
package main

import (
"encoding/json"
"flag"
"fmt"
"io/ioutil"
"os"
"path/filepath"

"github.com/ghodss/yaml"

"github.com/openshift/ci-operator/pkg/api"
)

Expand All @@ -26,8 +27,8 @@ func main() {
fmt.Printf("prevent panic by handling failure accessing a path %q: %v\n", configDir, err)
return err
}
if filepath.Ext(path) == ".json" {
// we assume any JSON in the config dir is a CI Operator config
if filepath.Ext(path) == ".yaml" || filepath.Ext(path) == ".json" {
// we assume any JSON or YAML in the config dir is a CI Operator config
name, err := filepath.Rel(configDir, path)
if err != nil {
return fmt.Errorf("could not determine relative path name for %s: %v", path, err)
Expand All @@ -39,7 +40,7 @@ func main() {
}

var config api.ReleaseBuildConfiguration
if err := json.Unmarshal(data, &config); err != nil {
if err := yaml.Unmarshal(data, &config); err != nil {
return fmt.Errorf("invalid configuration from %s: %v\nvalue:%s", name, err, string(data))
}

Expand Down
Loading