Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 12 additions & 28 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@

Notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/).
This project mirrors major Elm versions. So version 0.18.\* of this project will
be compatible with Elm 0.18.\*.
The format is based on [Keep a Changelog](http://keepachangelog.com/). This project mirrors major Elm versions. So version 0.18.\* of this project will be compatible with Elm 0.18.\*.

## 0.19.1-revision4 - 2020-09-21

Expand Down Expand Up @@ -101,8 +99,7 @@ be compatible with Elm 0.18.\*.

### Fixed

- All reporters were considered "machine readable", resulting in the error
stream being ignored (#161)
- All reporters were considered "machine readable", resulting in the error stream being ignored (#161)

### Added

Expand All @@ -117,8 +114,7 @@ be compatible with Elm 0.18.\*.

### Changed

- Imports in the `Example.elm` file are now sorted for compatibility with
`elm-format@exp`
- Imports in the `Example.elm` file are now sorted for compatibility with `elm-format@exp`

## 0.18.5 - 2017-06-11

Expand All @@ -128,8 +124,7 @@ be compatible with Elm 0.18.\*.

### Added

- If your project depends on `elm-lang/html`, `elm-test init` will add a
dependency to `eeue56/elm-html-test` so you can test your HTML. (#154)
- If your project depends on `elm-lang/html`, `elm-test init` will add a dependency to `eeue56/elm-html-test` so you can test your HTML. (#154)

## 0.18.4 - 2017-05-30

Expand All @@ -142,41 +137,30 @@ be compatible with Elm 0.18.\*.

### Added

- `--add-dependencies target/elm-package.json` flag to add any missing
dependencies _from_ the `elm-package.json` file in the current director _to_ the
`target/elm-package.json` file. (#28)
- `--add-dependencies target/elm-package.json` flag to add any missing dependencies _from_ the `elm-package.json` file in the current director _to_ the `target/elm-package.json` file. (#28)
- `Test.todo` to mark tests as _not yet implemented_. (#104)
- `--fuzz` flag to override the default fuzz count. (#77)
- `Test.only` and `Test.skip` to limit which tests will be executed.

### Changed

- `elm-test init` now adds all dependencies from the package `elm-package.json`
to the generated `tests/elm-package.json` file. (#68)
- You no longer write a `tests/Main.elm` file. Rather, you pass the paths to
your tests to the `elm-test` executable to run just those files, or you run
`elm-test` without arguments which will look for all elm files under `tests/`
and `test/`. (#72)
- `elm-test init` now adds all dependencies from the package `elm-package.json` to the generated `tests/elm-package.json` file. (#68)
- You no longer write a `tests/Main.elm` file. Rather, you pass the paths to your tests to the `elm-test` executable to run just those files, or you run `elm-test` without arguments which will look for all elm files under `tests/` and `test/`. (#72)
- All exposed values of type `Test` are executed by the test-runner.
- Duplicate titles/descriptions fail the test run. (#115)
- Empty describes are no longer allowed. (#95)

### Fixed

- Ignores `elm-stuff` (#100)
- Tests that throw a runtime exception fail with the exception message as
failure, rather than crashing the runner. (#69)
- Tests that throw a runtime exception fail with the exception message as failure, rather than crashing the runner. (#69)

#### Migrating from `0.18.2`

- Upgrade the runner `npm i -g elm-test`
- Remove `tests/Main.elm`
- Remove the dependency on `rtfeldman/node-test-runner` from
`tests/elm-package.json`
- Bump the dependency on `elm-community/elm-test` to `4.2.0 <= v < 5.0.0` in
`tests/elm-package.json`
- Ensure your test files expose each test you want to run, and that those values
are of type `Test`
- Make sure those tests aren't defined twice (for example: once as a top-level
value, and again in a `describe` block) or they will be executed twice.
- Remove the dependency on `rtfeldman/node-test-runner` from `tests/elm-package.json`
- Bump the dependency on `elm-community/elm-test` to `4.2.0 <= v < 5.0.0` in `tests/elm-package.json`
- Ensure your test files expose each test you want to run, and that those values are of type `Test`
- Make sure those tests aren't defined twice (for example: once as a top-level value, and again in a `describe` block) or they will be executed twice.
- run `elm-test` to execute your tests.
194 changes: 135 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,175 @@
# node-test-runner [![Version](https://img.shields.io/npm/v/elm-test.svg)](https://www.npmjs.com/package/elm-test)

Runs [elm-test](https://package.elm-lang.org/packages/elm-explorations/test/latest) suites from Node.js
Runs [elm-explorations/test] suites in Node.js.

When people say “elm-test” they usually refer to either:

- This CLI tool for running tests.
- [elm-explorations/test] – an Elm package for defining tests that this CLI tool can run.

[elm-explorations/test]: https://package.elm-lang.org/packages/elm-explorations/test/latest

## Installation

```shell
npm install -g elm-test
```
npm install --save-dev elm-test
```

## Usage
## Quick start

```shell
elm-test install foo/bar # Install the foo/bar package to "test-dependencies"
elm-test init # `elm-test install elm-explorations/test` and create tests/Example.elm
elm-test # Run all exposed Test values in *.elm files in tests/
elm-test Foo.elm # Run all exposed Test values in Foo.elm
```
Install [elm-explorations/test] and create `tests/Example.elm`:

### Command Line Arguments
npx elm-test init

#### `install`
Run tests in the `tests/` folder:

Like `elm install`, except it installs to the `test-dependencies` field of your project's `elm.json` file instead of `dependencies`.
npx elm-test

```shell
elm-test install elm/regex
```
Run tests in one particular file:

#### `init`
npx elm-test tests/Example.elm

Runs `elm-test install elm-explorations/test` and then creates a `tests/Example.elm`
example test to get you started.
Run tests in files matching a [glob](https://github.com/isaacs/node-glob#glob-primer):

Afterwards, you can run `elm-test` with no arguments to try out the example.
npx elm-test "src/**/*Tests.elm"

#### `--compiler`
> Note: The double quotes are important! Without quotes, your shell might expand the globs for you. With quotes, elm-test expands the globs. This way the watcher can pick up new tests matching the globs, and it will work cross-platform.

The `--compiler` flag can be used to use a version of the Elm compiler that
has not been installed globally.
Run in watch mode:

```shell
npm install elm
elm-test --compiler ./node_modules/.bin/elm
```
npx elm-test --watch

#### `--seed`
## Where to put tests

Allow running the tests with a predefined seed, rather than a randomly generated seed. This is especially helpful when trying to reproduce a failing fuzz-test.
### Locating files containing tests

```shell
elm-test --seed=12345
```
There are 3 places you could put your tests:

#### `--fuzz`
1. In the `tests/` folder.

Define how many times a fuzzer should run. Defaults to `100`
This is the default and requires no extra setup.

```shell
elm-test --fuzz=500
```
2. In any source directory (`"source-directories"` in `elm.json` for applications, `src/` for packages) as separate files.

#### `--report`
A convention is to put test files next to the file it tests with a `Tests` suffix. For example, you could have `src/LoginForm.elm` and `src/LoginFormTests.elm`.

Specify which reporter to use for reporting your test results. Valid options are:
This requires telling elm-test which folders/files to run. Examples:

- `console` (default): pretty, human readable formatted output
- `json`: every event will be written to stdout as a json-encoded object
- `junit`: junit-compatible xml will be written to stdout
npx elm-test "src/**/*Tests.elm"
npx elm-test test/frontend/elm

```shell
elm-test --report=json
```
You might also need to configure your editor to understand that the `"test-dependencies"` in your `elm.json` are available in these files.

3. In already existing source files.

This allows testing internal functions without exposing them. (Be aware that testing implementation details can sometimes be counter-productive.)

This requires moving everything in `"test-dependencies"` in your `elm.json` into regular `"dependencies"`, so your project still compiles. This also helps your editor. Note that this approach isn’t suitable for packages, since you don’t want your package to unnecessarily depend on [elm-explorations/test].

You can mix all three variants if you want:

#### `--version`
npx elm-test tests "src/**/*Tests.elm" app

Displays the version of the current elm-test.
> In this example, `"src"` and `"app"` need to be in `"source-directories"` in `elm.json`.

```shell
$ elm-test --version
0.19.1
### Locating tests within files

For elm-test to find tests in your files you need to:

1. Create top-level values of the type [Test](https://package.elm-lang.org/packages/elm-explorations/test/latest/Test#Test). You can name the values anything – the only thing that matters is that their type is `Test`.
2. Expose them.

Example:

```elm
module LoginForm exposing (alreadyLoggedInTests, tests)

import Test exposing (Test)


tests : Test
tests =
-- ...


alreadyLoggedInTests : Test
alreadyLoggedInTests =
-- ...
```

#### `--watch`
Some prefer to expose a single `Test` value and group everything using [describe](https://package.elm-lang.org/packages/elm-explorations/test/latest/Test#describe). Some prefer to expose several `Test` values.

**Also check out the [elm-explorations/test quick-start](https://github.com/elm-explorations/test#quick-start) guide!**

## Command Line Arguments

These are the most common commands and flags. Run `elm-test --help` for an exhaustive list.

**Note:** Throughout this section, the `npx` prefix is omitted for brevity.

Starts the runner in watch mode. Upon changing any currently watched source
files (either in your your source-directories or in your tests'
source-directories), your tests will get rerun.
### install

```shell
elm-test --watch
Like `elm install`, except elm-test will install to `"test-dependencies"` in your `elm.json` instead of to `"dependencies"`.

elm-test install elm/regex

### init

Runs `elm-test install elm-explorations/test` and then creates a `tests/Example.elm` example test to get you started.

`elm-test init` requires an `elm.json` file up the directory tree, so you will need to run `elm init` first if you don’t already have one.

After initializing elm-test in your project, try out the example by running `elm-test` with no arguments.

elm init
elm-test init
elm-test

### --watch

Start the runner in watch mode. Your tests will automatically rerun whenever your project changes.

elm-test --watch

### --seed

Run with a specific fuzzer seed, rather than a randomly generated seed. This allows reproducing a failing fuzz-test. The command needed to reproduce (including the `--seed` flag) is printed after each test run. Copy, paste and run it!

elm-test --seed 336948560956134

### --fuzz

Define how many times each fuzz-test should run. Defaults to `100`.

elm-test --fuzz 500

### --report

Specify which format to use for reporting test results. Valid options are:

- `console` (default): pretty, human readable formatted output.
- `json`: newline-delimited json with an object for each event.
- `junit`: junit-compatible xml.

```
elm-test --report json
```

#### `--help`
### --compiler

Displays all the available options and commands.
If `elm` is _not_ in your `$PATH` when elm-test runs, or the Elm executable is called something other than `elm`, you can use this flag to point to your installation.

### Travis CI
elm-test --compiler /path/to/elm

To run a tool installed locally using `npm` you can use `npx`:

npx elm-test

`npx` adds the local `node_modules/.bin/` folder to `$PATH` when it executes the command passed to it. This means that if you have installed `elm` locally, `elm-test` will automatically find that local installation.

As mentioned in [Installation](#installation) we recommend installing elm-test locally in every project. This ensures all contributors and CI use the same version, to avoid nasty “works on my computer” issues.

## Travis CI

If you want to run your tests on Travis CI, [here's a good starter `.travis.yml`](https://docs.travis-ci.com/user/languages/elm/):

Expand All @@ -104,3 +178,5 @@ language: elm
elm:
- 0.19.1
```

Here is an example [`travis.yml`](https://github.com/rtfeldman/elm-css/blob/master/.travis.yml) configuration file for running tests in CI.
20 changes: 10 additions & 10 deletions lib/elm-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -90,41 +90,41 @@ function main() {
.name('elm-test')
.usage('[options] [globs...]')
.description(examples)
.option('--watch', 'Run tests on file changes', false)
// For example `--seed` and `--fuzz` only make sense for the “tests” command
// and could be specified for that command only, but then they won’t show up
// in `--help`.
.option(
'--compiler <path>',
'Use a custom path to an Elm executable (default: elm)',
undefined
)
.addOption(
new Option('--seed <int>', 'Run with a previous fuzzer seed')
new Option('--seed <int>', 'Run with a specific fuzzer seed')
.default(Math.floor(Math.random() * 407199254740991) + 1000, 'random')
.argParser(parsePositiveInteger)
)
.option(
'--fuzz <int>',
'Run with each fuzz test performing this many iterations',
'Define how many times each fuzz-test should run',
parsePositiveInteger,
100
)
.addOption(
new Option(
'--report <format>',
'Print results to stdout in the given format'
'Specify which format to use for reporting test results'
)
.default('console')
.choices(Report.all)
)
.option('--watch', 'Run tests on file changes', false)
.option(
'--compiler <path>',
'Use a custom path to an Elm executable (default: elm)',
undefined
)
.version(packageInfo.version, '--version', 'Print version and exit')
.helpOption('-h, --help', 'Show help')
.addHelpCommand('help [command]', 'Show help');

program
.command('init')
.description('Create example tests')
.description('Install elm-explorations/test and create tests/Example.elm')
.action(() => {
const options = program.opts();
const pathToElmBinary = getPathToElmBinary(options.compiler);
Expand Down
1 change: 1 addition & 0 deletions prettier.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
module.exports = {
singleQuote: true,
proseWrap: 'never',
overrides: [
{
files: '*.js',
Expand Down