kola: Support loading tests from /usr/lib/coreos-assembler/tests/kola #1441
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In continuing work on external tests, passing `-E` gets awkward for a few reasons. First, it ties the tests too heavily to the source directory. This supports a model like https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests and Debian autopkg test etc. where the tests are installed. This ensures a clean separation from the source directory, which also helps us in e.g. supporting built binaries as is done in ostreedev/ostree#2048 For example, one thing we can do after this is extend the coreos-assembler container with the results of `make install-tests` from relevant git repositories (or make RPMs of them).
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: cgwalters The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This was referenced May 11, 2020
darkmuggle
reviewed
May 12, 2020
| // can be placed; for example, a project like ostree can install | ||
| // tests at /usr/lib/coreos-assembler/tests/kola/ostree/... | ||
| // and this will be automatically picked up. | ||
| const InstalledTestsDir = "/usr/lib/coreos-assembler/tests/kola" |
Contributor
There was a problem hiding this comment.
nit: could we make this a var instead, e.g.
const defaultInstalledTestDirs = "/usr/lib/coreos-assembler/tests/kola"
var InstalledTestsDir = defaultInstalledTestDir
Such that go run -ldflags '-X kola.harness.InstalledTestsDir="foo" ...' works?
Member
Author
There was a problem hiding this comment.
The idea is that this is a standard directory for other components to install to; we already have a way to do "find tests from elsewhere" via -E right?
darkmuggle
approved these changes
May 12, 2020
Contributor
darkmuggle
left a comment
darkmuggle
left a comment
There was a problem hiding this comment.
Completely optional nit.
/lgtm
Member
Author
|
/refresh |
cgwalters
added a commit
to cgwalters/ostree
that referenced
this pull request
May 14, 2020
This builds on coreos/coreos-assembler#1441 to install our tests rather than running them from the source directory. This model will more cleanly allow us to ship our tests along with a test container or elsewhere, separate from the source directory. Also prep for ostreedev#2048
cgwalters
added a commit
to cgwalters/ostree
that referenced
this pull request
May 15, 2020
This builds on coreos/coreos-assembler#1441 to install our tests rather than running them from the source directory. This model will more cleanly allow us to ship our tests along with a test container or elsewhere, separate from the source directory. Also prep for ostreedev#2048
cgwalters
added a commit
to cgwalters/ostree
that referenced
this pull request
May 15, 2020
This builds on coreos/coreos-assembler#1441 to install our tests rather than running them from the source directory. This model will more cleanly allow us to ship our tests along with a test container or elsewhere, separate from the source directory. Also prep for ostreedev#2048
cgwalters
added a commit
to cgwalters/ostree
that referenced
this pull request
May 15, 2020
This builds on coreos/coreos-assembler#1441 to install our tests rather than running them from the source directory. This model will more cleanly allow us to ship our tests along with a test container or elsewhere, separate from the source directory. Also prep for ostreedev#2048
cgwalters
added a commit
to cgwalters/rpm-ostree
that referenced
this pull request
May 23, 2020
Switch to the "installed" model introduced by: coreos/coreos-assembler#1441 It's hard to support running tests *both* from the srcdir and installed; in this case because we have a symlink that needs to be followed, which kola knows how to do from the srcdir but not when installed. Let's establish a new convention of `tests/kolainst`. In our case we follow the symlink manually for now. That bit will be cleaned up when we eventually switch entirely to kola tests.
cgwalters
added a commit
to cgwalters/rpm-ostree
that referenced
this pull request
May 23, 2020
Switch to the "installed" model introduced by: coreos/coreos-assembler#1441 It's hard to support running tests *both* from the srcdir and installed; in this case because we have a symlink that needs to be followed, which kola knows how to do from the srcdir but not when installed. Let's establish a new convention of `tests/kolainst`. In our case we follow the symlink manually for now. That bit will be cleaned up when we eventually switch entirely to kola tests.
cgwalters
added a commit
to cgwalters/rpm-ostree
that referenced
this pull request
May 26, 2020
Switch to the "installed" model introduced by: coreos/coreos-assembler#1441 It's hard to support running tests *both* from the srcdir and installed; in this case because we have a symlink that needs to be followed, which kola knows how to do from the srcdir but not when installed. Let's establish a new convention of `tests/kolainst`. In our case we follow the symlink manually for now. That bit will be cleaned up when we eventually switch entirely to kola tests.
openshift-merge-robot
pushed a commit
to coreos/rpm-ostree
that referenced
this pull request
May 26, 2020
Switch to the "installed" model introduced by: coreos/coreos-assembler#1441 It's hard to support running tests *both* from the srcdir and installed; in this case because we have a symlink that needs to be followed, which kola knows how to do from the srcdir but not when installed. Let's establish a new convention of `tests/kolainst`. In our case we follow the symlink manually for now. That bit will be cleaned up when we eventually switch entirely to kola tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In continuing work on external tests, passing
-Egets awkwardfor a few reasons. First, it ties the tests too heavily to the
source directory.
This supports a model like https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests
and Debian autopkg test etc. where the tests are installed.
This ensures a clean separation from the source directory,
which also helps us in e.g. supporting built binaries as is done
in ostreedev/ostree#2048
For example, one thing we can do after this is extend the
coreos-assembler container with the results of
make install-testsfrom relevant git repositories (or make RPMs of them).