Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-OS: Provide binaries #585

Merged
merged 71 commits into from
Jun 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
39f240e
Multi-OS: Provide binaries
May 10, 2020
af50e67
Make cross-compilation build work (on macOS locally, try CI separately)
May 10, 2020
cffc0bc
deal with cc-test-reporter
May 10, 2020
1e33748
oh, Windows
May 10, 2020
3b0a2a1
Add docs and start of matrix / test-plan
May 10, 2020
b4b5371
cross-platform CI
May 10, 2020
3bf369d
debug
May 10, 2020
fc0e37c
save
May 10, 2020
6f04931
oops
May 10, 2020
284705a
debug
May 10, 2020
25b2e67
oci runtime error try2
May 10, 2020
59b9573
no windows support for codeclimate
May 10, 2020
6f04e71
missed that
May 10, 2020
f9011ff
bash
May 10, 2020
24177a5
Windows find is, of course, different
May 10, 2020
4621bbf
CI
May 10, 2020
91f8f03
fix pasta
May 10, 2020
955c3bb
fix windows binary name
May 10, 2020
daa336f
Move to ci folder to avoid cluttering root. Implement fmt and test to…
May 10, 2020
1ac0fc2
pass through
May 10, 2020
0fbb844
try just bash
May 10, 2020
46678fd
oops2
May 10, 2020
35aa225
TIL go fmt {package} is a thing
May 10, 2020
6b9344f
no longer needed
May 10, 2020
670cdd4
windows
May 10, 2020
cc06c9d
fix table
May 10, 2020
77cc3bf
fix
May 10, 2020
bcea1ce
pasta
May 10, 2020
1ba5427
format table
May 10, 2020
02ed9cb
revert to master
May 11, 2020
09586c3
revert
May 11, 2020
4ec3fa6
make travis_tag consistent for local builds compared to previously
May 11, 2020
a2c784a
:facepalm:
May 11, 2020
8486de1
flesh out parity doc
May 11, 2020
78b539c
expand
May 11, 2020
330bd1c
tweak
May 11, 2020
75eb18a
elaborate
May 11, 2020
8227028
I added tests
May 11, 2020
fb3921b
simplify I think?
May 14, 2020
e35b1b8
Name the binaries to denote alphaness
May 14, 2020
b939438
Output warning before/after, and fail unless GOSS_ALPHA=1
May 14, 2020
4f145e9
elaborate slightly
May 14, 2020
ee45539
oops, CI verbiage
May 14, 2020
988558f
cleaner formatting
petemounce May 17, 2020
1a98dfc
support each shell
petemounce May 24, 2020
f7c2d6f
Test Windows by adding automation
petemounce May 24, 2020
7bc01d7
+x
petemounce May 24, 2020
9a24740
oops!
petemounce May 24, 2020
7a28410
debug
petemounce May 24, 2020
b1f60f5
bash and debug
petemounce May 24, 2020
cfa1584
actually building the binary helps when running it!
petemounce May 25, 2020
1e7aa4e
apparently it's not on travis
petemounce May 25, 2020
9e94b49
Darwin coverage
petemounce May 25, 2020
19d5914
fix
petemounce May 25, 2020
926203f
enable
petemounce May 25, 2020
9a4c64c
rename
petemounce May 25, 2020
bc7eaa4
skip
petemounce May 25, 2020
94248a8
skip
petemounce May 25, 2020
f406321
no skip
petemounce May 25, 2020
d8042f3
adjust for darwin
petemounce May 25, 2020
2bd23c3
skip the not-working bits
petemounce May 25, 2020
17d2a39
Clarify wording, set support expectations
petemounce May 25, 2020
fae7f38
oops
petemounce May 25, 2020
92c20cf
n/a + docs
petemounce May 25, 2020
43f6313
try to appease codeclimate
petemounce May 25, 2020
50c8328
Surely the codeclimate duplication gods approve now
petemounce May 26, 2020
97c4250
well, perhaps the gods like this one?
petemounce May 26, 2020
143e895
Return -1 as 'not applicable'
petemounce Jun 10, 2020
9ebab4a
Remember go env
petemounce Jun 15, 2020
ed0c665
oops
petemounce Jun 15, 2020
7a762a1
now unnecessary
petemounce Jun 15, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,38 +9,47 @@ go:
- 1.13.x

os:
- osx
- linux
- windows

dist: trusty

services:
- docker

before_install:
- if [[ "${TRAVIS_OS_NAME}" == "windows" ]]; then choco install make; fi
# bash from macOS is too old to have readarray. Install newer version.
- if [[ "${TRAVIS_OS_NAME}" == "osx" ]]; then brew install bash; fi

install:
- go get -u golang.org/x/lint/golint
- curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
- chmod +x ./cc-test-reporter
- ./ci/install.sh

before_script:
- ./cc-test-reporter before-build
- ./ci/before-build.sh

script:
- make all
- ./ci/build.sh

after_script:
- ./cc-test-reporter after-build --exit-code $TRAVIS_TEST_RESULT -d
- ./ci/after-build.sh

deploy:
provider: releases
token:
secure: hEHCC4EN7iHz7pIWKRn2qw22NTqUxnuBp59wfAlLBtV26j5rHMzSu8mlxkJInusDUGLJiNLrZPRWN0mzOdIXalbUeLhlX7EflJgEj6Q0MchUR69LzCAp0KMIFL1Sfq0v81VgujRLUUy5utxDL8Er4tZknn2PpXAMzpO+ozjNRDhhSEM4iMXfY3bcOIMnx6XRgCjFCb036wlBgOfdgv5fwm2PP638DTKar4W6ZZbqCQByhJ5RyL3BMDPTT0moA/tYbG+FA6p6Rme1OcBkMnpsiJZoB3u8gxsNiEJ43/C2RcULW/18qqp2UVD5FipSDYP7GQ5ugKCbgpWXb0Ctl8o4hv1UsNl0XoyJhAt0PRp6vqnyy6LWB2FzX30Xj/vGIhO/IfiJvspHxpatTk7Esjr46K4u9ao/x63LX6F6yI1ZTfbzt2MhRYRjwh4ORNfqhysuzXChftX1S9hj6s6gO0/zqoOsRK/PK8DProbUn4bxrGOBzi16P0GEk4agWWUm74Pis9qCThXNW8MXEV936KvE1wb1RxTACYvFBtO2IM5eQ26t2Y7mGJd7FJup9LR4oUtUTSbYo5P2Sal6xntBKH5P4nwEtM+TtHoeSCKQ3X5i1VSdvAH7soEAly6rP5d5wwPhqqx9mgUPYO/3ulvxLJOYHamrbj6nlHDXnCEoj1ZMxX4=
file:
- release/goss-alpha-darwin-amd64
- release/goss-alpha-darwin-amd64.sha256
- release/goss-linux-amd64
- release/goss-linux-amd64.sha256
- release/goss-linux-386
- release/goss-linux-386.sha256
- release/goss-linux-arm
- release/goss-linux-arm.sha256
- release/goss-alpha-windows-amd64.exe
- release/goss-alpha-windows-amd64.exe.sha256
- extras/dgoss/dgoss
- extras/dgoss/dgoss.sha256
skip_cleanup: true
Expand Down
31 changes: 7 additions & 24 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,21 @@ export GO15VENDOREXPERIMENT=1
exe = github.com/aelsabbahy/goss/cmd/goss
pkgs = $(shell ./novendor.sh)
cmd = goss
TRAVIS_TAG ?= "0.0.0"
GO_FILES = $(shell find . \( -path ./vendor -o -name '_test.go' \) -prune -o -name '*.go' -print)
GO111MODULE=on

.PHONY: all build install test release bench fmt lint vet test-int-all gen centos7 wheezy precise alpine3 arch test-int32 centos7-32 wheezy-32 precise-32 alpine3-32 arch-32

all: test-all dgoss-sha256
all: test-short-all test-int-all dgoss-sha256

test-all: fmt lint vet test test-int-all
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduced test-short-all to represent all the short tests like lints and unit-tests, which can run on each platform.

The integration tests can't run on non-linux unless we come up with a different approach, but I definitely didn't want to include that scope here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I give up on trying to have a proper integration test for Windows/OSX

Solid unit tests + a small smoke test might be sufficient.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, 👍. If we think it becomes worthwhile to do more than unit-test, I reckon it'll be reasonably straightforward to achieve via TestMain and blackbox'ing against some golden files.

Copy link
Member

@aelsabbahy aelsabbahy May 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious on your Test mail suggestion. Whenever you have a chance (no rush). Can you link me a psudocode gist example of what you mean.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/aelsabbahy/goss/pull/407/files#diff-2e387891f2d94197d8fc70573226de67R17-R34 - the job of the TestMain is to make sure that the binary is present; in this example it's done via go build, but it doesn't matter what the implementation is.

https://github.com/aelsabbahy/goss/pull/407/files#diff-2e387891f2d94197d8fc70573226de67R1 - use a build-tag so that these tests only run via go test -tags integration - this would allow the tests to only be run after the build automation has had a chance to build the release binaries, at which point TestMain could just choose which one to use based on runtime.GOOS/runtime.GOARCH.

https://github.com/aelsabbahy/goss/pull/407/files#diff-2e387891f2d94197d8fc70573226de67R13 - rendon/testcli is a package that looked the most similar to a setup I'm familiar with from another language's ecosystem. It makes a small wrapper around os.exec and then allows one to assert against stdout/stderr.

https://github.com/aelsabbahy/goss/pull/407/files#diff-2e387891f2d94197d8fc70573226de67R36-R63 - table-driven tests that run the binary and compare the output to some golden files, with the option to update those.

test-short-all: fmt lint vet test

install: release/goss-linux-amd64
$(info INFO: Starting build $@)
cp release/$(cmd)-linux-amd64 $(GOPATH)/bin/goss

test:
$(info INFO: Starting build $@)
{ \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to inside travis, factored these into scripts where it becomes easier to deal with the cross-platform differences.

set -e ;\
go test -coverprofile=c.out ${pkgs} ;\
cat c.out | sed 's|github.com/aelsabbahy/goss/||' > c.out.tmp ;\
mv c.out.tmp c.out ;\
}
./ci/go-test.sh $(pkgs)

lint:
$(info INFO: Starting build $@)
Expand All @@ -36,30 +29,20 @@ vet:

fmt:
$(info INFO: Starting build $@)
{ \
set -e ;\
fmt=$$(gofmt -l ${GO_FILES}) ;\
[ -z "$$fmt" ] && echo "valid gofmt" || (echo -e "invalid gofmt\n$$fmt"; exit 1)\
}
./ci/go-fmt.sh

bench:
$(info INFO: Starting build $@)
go test -bench=.



# Pattern rule for platform builds.
# `subst` substitutes space for -, thus making an array
# firstword, and word select indexes from said array.
release/goss-%: $(GO_FILES)
CGO_ENABLED=0 GOOS=$(firstword $(subst -, ,$*)) GOARCH=$(word 2, $(subst -, ,$*)) go build -ldflags "-X main.version=$(TRAVIS_TAG) -s -w" -o $@ $(exe)
sha256sum $@ > [email protected]
release/goss-%:
./release-build.sh $*

release:
$(MAKE) clean
$(MAKE) build

build: release/goss-linux-386 release/goss-linux-amd64 release/goss-linux-arm
build: release/goss-alpha-darwin-amd64 release/goss-linux-386 release/goss-linux-amd64 release/goss-linux-arm release/goss-alpha-windows-amd64

gen:
$(info INFO: Starting build $@)
Expand Down
15 changes: 9 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ Goss is a YAML based [serverspec](http://serverspec.org/) alternative tool for v

## Installation

Note: macOS and Windows binaries are new and considered alpha-quality. Some functionality may be missing, some may be broken. Refer to <platform-feature-parity.md> (Enhancements and bug-reports welcome, please see [#551: Multi-OS support](https://github.com/aelsabbahy/goss/issues/551))

This will install goss and [dgoss](https://github.com/aelsabbahy/goss/tree/master/extras/dgoss).

**Note:** Using `curl | sh` is not recommended for production systems, use manual installation below.
Expand All @@ -45,8 +47,8 @@ curl -fsSL https://goss.rocks/install | GOSS_VER=v0.3.6 GOSS_DST=~/bin sh

### Manual installation


#### Latest

```bash
curl -L https://github.com/aelsabbahy/goss/releases/latest/download/goss-linux-amd64 -o /usr/local/bin/goss
chmod +rx /usr/local/bin/goss
Expand All @@ -58,6 +60,7 @@ chmod +rx /usr/local/bin/dgoss
```

#### Specific Version

```bash
# See https://github.com/aelsabbahy/goss/releases for release versions
VERSION=v0.3.10
Expand Down Expand Up @@ -88,7 +91,7 @@ An initial set of tests can be derived from the system state by using the [add](

Let's write a simple sshd test using autoadd.

```
```txt
# Running it as root will allow it to also detect ports
$ sudo goss autoadd sshd
```
Expand Down Expand Up @@ -132,7 +135,7 @@ Now that we have a test suite, we can:

* Run it once

```
```txt
goss validate
...............

Expand All @@ -142,19 +145,19 @@ Count: 15, Failed: 0

* Edit it to use [templates](https://github.com/aelsabbahy/goss/blob/master/docs/manual.md#templates), and run with a vars file

```
```txt
goss --vars vars.yaml validate
```

* keep running it until the system enters a valid state or we timeout

```
```txt
goss validate --retry-timeout 30s --sleep 1s
```

* serve the tests as a health endpoint

```
```txt
goss serve &
curl localhost:8080/healthz

Expand Down
8 changes: 8 additions & 0 deletions ci/after-build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail

os_name="$(go env GOOS)"

if [[ "${os_name}" != "windows" ]]; then
./cc-test-reporter after-build --exit-code "${TRAVIS_TEST_RESULT}" -d
fi
8 changes: 8 additions & 0 deletions ci/before-build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail

os_name="$(go env GOOS)"

if [[ "${os_name}" != "windows" ]]; then
./cc-test-reporter before-build
fi
13 changes: 13 additions & 0 deletions ci/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -euo pipefail

os_name="$(go env GOOS)"

# darwin & windows do not support integration-testing approach via docker, so on those, just run fast tests.
if [[ "${os_name}" == "darwin" || "${os_name}" == "windows" ]]; then
make test-short-all release/goss-alpha-${os_name}-amd64
integration-tests/run-tests-alpha.sh "${os_name}"
else
# linux runs all tests; unit and integration.
make all
fi
24 changes: 24 additions & 0 deletions ci/go-fmt.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -euo pipefail

os_name="$(go env GOOS)"

# gofmt must be on PATH
command -v gofmt

if [[ "${os_name}" == "windows" ]]; then
echo "Skipping go-fmt on Windows because line-endings cause every file to need formatting."
echo "Linux is treated as authoritative."
echo "Exiting 0..."
exit 0
fi

fmt="$(go fmt github.com/aelsabbahy/goss/...)"

if [[ -z "${fmt}" ]]; then
echo "valid gofmt"
else
echo "invalid gofmt:"
echo "${fmt}"
exit 1
fi
10 changes: 10 additions & 0 deletions ci/go-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail

command -v go

go test -coverprofile="c.out" "${1}"

sed 's|github.com/aelsabbahy/goss/||' <"c.out" >"c.out.tmp"

mv "c.out.tmp" "c.out"
11 changes: 11 additions & 0 deletions ci/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail

os_name="$(go env GOOS)"

go get -u golang.org/x/lint/golint

if [[ "${os_name}" != "windows" ]]; then
curl -L "https://codeclimate.com/downloads/test-reporter/test-reporter-latest-${os_name}-amd64" > "./cc-test-reporter"
chmod +x "./cc-test-reporter"
fi
Loading