-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: Stub in sharness framework #180
base: master
Are you sure you want to change the base?
Conversation
test/t1001-generate-template.t
Outdated
ocitools generate --template template | jq . >actual && | ||
cat <<-EOF >expected && | ||
{ | ||
\"ociVersion\": \"1.0.0-rc1\", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't like the escaping that we do here :/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Wed, Aug 03, 2016 at 04:06:28PM -0700, Mrunal Patel wrote:
@@ -0,0 +1,59 @@
+#!/bin/sh
+
+test_description='Test generate template'
+
+. ./sharness.sh
+
+test_expect_failure CAT,ECHO,HEAD,JQ 'Test ocitools generate --template with an empty template' "
- echo '{}' >template &&
- ocitools generate --template template | jq . >actual &&
- cat <<-EOF >expected &&
{
\"ociVersion\": \"1.0.0-rc1\",
I don't like the escaping that we do here :/
There are only so many quotes we can use ;). Would you rather me wrap
the test in single quotes and use double quotes inside it? That makes
using variables in the test tricky, but we don't do any of that yet.
The sharness framework looks cool, I need time to absorb it. It is different test frame topic with #66, right? It is more like a unit test. If so, I think we |
On Thu, Aug 04, 2016 at 05:34:29AM -0700, 梁辰晔 (Liang Chenye) wrote:
Yes. That is about “how do we validate a runtime?” and this is about
The goal of this PR is to test the ocitools command line as a black |
+1 for the black box test |
Is this dead, or worth a rebase? |
Compared with importing sharness, how about importing a go based framework, like go-check? |
On Wed, Oct 12, 2016 at 11:04:24PM -0700, Ma Shimiao wrote:
A shell-based test harness lets us use a very convenient API for ocitools --help | head -n2 >actual && |
Please contribute to master branch |
I can rebase onto master and change the target branch without filing a new PR and fragmenting discussion. Can you re-open this PR so I can do that? |
@@ -15,3 +15,4 @@ script: | |||
- git-validation -run DCO,short-subject -v -range ${TRAVIS_COMMIT_RANGE} | |||
- make | |||
- make test | |||
- PATH="${PATH}:${PWD}" make -C test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this will be failed. As from test/README.md, we need do 'git submodule update --init' first. But you didn't do such preparation in travis.yml
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
... we need do 'git submodule update --init' first.
Travis does that for us.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, got it.
test/README.md
Outdated
# oci-runtime-tool integration tests | ||
|
||
This project uses the [Sharness][] test harness, installed as a [Git | ||
submodule]submodule]. To setup the test installation after a clone, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm missing a [
here. Will update later tonight.
I'm OK with this patchset, but generate related tests are all failed. Would you mind update these tests based on current generate output? |
They look like they passed to me. Can you link to what you'd like fixed? |
My local test result:
|
Ah, I probably need to rebase around #431. |
It's GPLv2+ [1], so using a submodule keeps the separation from our Apache 2.0 source clear. And it's an isolated enough dependency that I prefer to keep the Git histories separate. Generated with: $ mkdir test $ git submodule add git://github.com/mlafeldt/sharness.git test/sharness $ (cd test/sharness && git checkout v1.0.0) $ git add test/sharness $ cd test $ ln -s sharness/aggregate-results.sh $ ln -s sharness/sharness.sh $ ln -s sharness/test/Makefile $ emacs README.md $ git add README.md Makefile sharness.sh aggregate-results.sh [1]: https://github.com/mlafeldt/sharness/tree/v1.0.0#license Signed-off-by: W. Trevor King <[email protected]>
Signed-off-by: W. Trevor King <[email protected]>
Signed-off-by: W. Trevor King <[email protected]>
Using jq to format the output so we don't have to worry about oci-runtime-tool's default tab indents or lack of trailing newlines, neither of which play nicely with <<-EOF here documents. Signed-off-by: W. Trevor King <[email protected]>
Signed-off-by: W. Trevor King <[email protected]>
On Wed, Aug 02, 2017 at 04:04:30AM +0000, W. Trevor King wrote:
Ah, I probably need to rebase around #431.
|
Spun off from this discussion. This currently ignores most of the
generate
options and doesn't touchvalidate
at all, but I thought I'd float it early for some feedback before sinking too much time into it. If the current work looks solid, I'm happy to land it now and follow up with additional PRs to fill it out, or to keep this PR open until the coverage is more complete.Details on the individual changes in the commit messages.