Skip to content

Commit

Permalink
docs: remove shellspec mention and todo which is not currently on any…
Browse files Browse the repository at this point in the history
… to-do list
  • Loading branch information
thisislawatts committed Jan 6, 2025
1 parent 35ec148 commit da018ac
Showing 1 changed file with 0 additions and 33 deletions.
33 changes: 0 additions & 33 deletions test/smoke/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@

Design goal is to have a single test suite, that can detect if CLI is not working properly - before and after it's released. Defects it should catch are e.g. [broken pkg builds for a specific platform](https://github.com/snyk/snyk/issues/670), [misaligned dependencies](https://github.com/snyk/snyk/issues/1261) or [issues with a specific installation targets](https://github.com/snyk/snyk/issues/1270).

CLI is being tested by a series of tests using [Shellspec](https://shellspec.info). See them in a `test/smoke/spec` folder.

Spec in this folder is used as a

1. **"Smoke test" step in CircleCI** to verify that built CLI can run
Expand All @@ -13,8 +11,6 @@ Spec in this folder is used as a

Smoke tests should be fast. Ideally tests only things that could break when CLI is being built and packaged into binaries. Functionality should be tested our _other_ tests.

Before you start adding specs, those files are bash scripts, it's recommended to have a [ShellCheck](https://www.shellcheck.net) installed and running in your IDE. See [Shellspec reference](https://github.com/shellspec/shellspec/blob/master/docs/references.md#expectation) for available commands.

It's recommended to have a branch named `smoke/_SOMETHING_`, as [this branch will run the GitHub Action](https://github.com/snyk/snyk/blob/f35f39e96ef7aa69b22a846315dda015b12a4564/.github/workflows/smoke-tests.yml#L3-L5).

To run these tests locally you may use `npm run test:smoke`:
Expand All @@ -25,10 +21,6 @@ To run these tests locally you may use `npm run test:smoke`:

2. Install dependencies for the local fixture `test/fixtures/basic-npm` with `npm install --prefix test/fixtures/basic-npm`

3. Run shellspec locally:

Note that you will need shellspec version `0.28.1` or higher.

From the root of the `snyk` repo, run:

```sh
Expand All @@ -45,31 +37,6 @@ This will meddle with your `snyk config` file as Smoke Tests are checking functi

This will open a browser in one instance unless it's disabled with `SMOKE_TESTS_SKIP_TEST_THAT_OPENS_BROWSER=1` envvar. Opening of a browser is disabled by default when running Smoke tests with npm command `npm run test:smoke`.

## TODO

### Wishlist

- [ ] be able to run against PR

### Missing scenarios

- [x] basics: version, help, config
- [x] auth [TOKEN]
- [x] test [--json][npm project]
- [ ] test [--json][java-goof]
- [ ] policy, ignore
- [x] monitor [--json][npm project]
- [ ] monitor [--json][java-goof]
- [ ] wizard - possibly impossible? (maybe a basic test that it even loads)

### Missing environments

- [x] Alpine binary
- [ ] Docker: current images can't output a clear stderr, because of an extraneous --json flag. Also released version is currently lagging behind the latest GitHub tag by a few hours
- [x] yarn installation
- [ ] scoop package
- [x] homebrew

## Current workarounds and limitations

### Alpine
Expand Down

0 comments on commit da018ac

Please sign in to comment.