Skip to content
This repository has been archived by the owner on Sep 14, 2023. It is now read-only.

feat: integrate examples as deno tests for ci #510

Merged
merged 1 commit into from
Jan 25, 2023

Conversation

ryanleecode
Copy link
Contributor

@ryanleecode ryanleecode commented Jan 23, 2023

Resolves #524

Description

  • Creates a script to run all examples in parallel except those specified in .ignore.

  • Maps the output into deno tests so they can be integrated in CI.

  • Fixes extrinstic.test.ts to not depend on the balance of alice or bob since they can be changed while the examples are running in parallel

  • Fixes the batch.ts and polkadot_js_signer.ts to use dave as the signer as to not interfere with transaction nonces created by alice. Otherwise the node will throw rpc errors as shown below.

[examples/batch.ts] RPC ERROR {
[examples/batch.ts]   jsonrpc: "2.0",
[examples/batch.ts]   error: {
[examples/batch.ts]     code: 1014,
[examples/batch.ts]     message: "Priority is too low: (13654 vs 2174)",
[examples/batch.ts]     data: "The transaction has too low priority to replace another transaction already in the pool."
[examples/batch.ts]   },
[examples/batch.ts]   id: 3
[examples/batch.ts] }
[examples/batch.ts] RPC ERROR {
[examples/batch.ts]   jsonrpc: "2.0",
[examples/batch.ts]   error: {
[examples/batch.ts]     code: 1014,
[examples/batch.ts]     message: "Priority is too low: (13654 vs 2174)",
[examples/batch.ts]     data: "The transaction has too low priority to replace another transaction already in the pool."
[examples/batch.ts]   },
[examples/batch.ts]   id: 3
[examples/batch.ts] }

@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 76fb204 to d6f22e4 Compare January 23, 2023 22:39
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from d6f22e4 to cf704c8 Compare January 23, 2023 22:50
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from cf704c8 to 5965ee1 Compare January 23, 2023 22:50
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 5965ee1 to 7fcaec1 Compare January 24, 2023 15:43
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 7fcaec1 to 8e0a0d4 Compare January 24, 2023 15:43
@ryanleecode ryanleecode marked this pull request as ready for review January 24, 2023 15:50
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 8e0a0d4 to 77b64c8 Compare January 24, 2023 18:39
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 77b64c8 to 49f2fc7 Compare January 24, 2023 18:40
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 49f2fc7 to 2b8483e Compare January 24, 2023 18:44
@ryanleecode ryanleecode requested a review from tjjfvi January 24, 2023 18:48
.github/workflows/test.yml Outdated Show resolved Hide resolved
deno.jsonc Outdated Show resolved Hide resolved
util/constants.ts Outdated Show resolved Hide resolved
examples/examples.test.ts Outdated Show resolved Hide resolved
examples/examples.test.ts Outdated Show resolved Hide resolved
examples/examples.test.ts Outdated Show resolved Hide resolved
examples/batch.ts Outdated Show resolved Hide resolved
examples/.ignore Outdated Show resolved Hide resolved
examples/examples.test.ts Outdated Show resolved Hide resolved
examples/examples.test.ts Outdated Show resolved Hide resolved
@tjjfvi
Copy link
Contributor

tjjfvi commented Jan 24, 2023

Also, this doesn't (and shouldn't) resolve #509. This is orthogonal to that issue, which is about adding assert calls into the examples themselves.

_tasks/test_examples.ts Outdated Show resolved Hide resolved
examples/.ignore Outdated Show resolved Hide resolved
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from 84783c8 to e14909d Compare January 25, 2023 18:42
@ryanleecode ryanleecode force-pushed the feat/integrated-example-ci-testing branch from e14909d to 522c5c8 Compare January 25, 2023 20:07
@ryanleecode
Copy link
Contributor Author

Added a progress bar and aggregated the failed tests results to the end. Looks something like this now

19/20 examples completed 31.7s [                                                  ] 95.00%----- output end -----
./_tasks/test_examples.ts => examples ... batch.ts ... ok (31s)
./_tasks/test_examples.ts => examples ... watch_events.ts ... ok (20s)
./_tasks/test_examples.ts => examples ... read_era_rewards.ts ... ok (2s)
./_tasks/test_examples.ts => examples ... read_bonded.ts ... ok (12s)
./_tasks/test_examples.ts => examples ... watch_blocks.ts ... ok (9s)
./_tasks/test_examples.ts => examples ... read_events.ts ... ok (1s)
./_tasks/test_examples.ts => examples ... fee_estimate.ts ... ok (12s)
./_tasks/test_examples.ts => examples ... balance.ts ... FAILED (12s)
    error: AssertionError: task failed with status code: 1
        throw new AssertionError(msg);
              ^
        at assert (https://deno.land/[email protected]/testing/asserts.ts:139:11)
        at fn (file:///workspaces/capi/_tasks/test_examples.ts:60:11)
        at async testStepSanitizer (deno:cli/js/40_testing.js:448:7)
        at async TestContext.step (deno:cli/js/40_testing.js:1338:13)
        at async Promise.all (index 7)
        at async file:///workspaces/capi/_tasks/test_examples.ts:37:3
./_tasks/test_examples.ts => examples ... polkadot_js_signer.ts ... ok (31s)
./_tasks/test_examples.ts => examples ... rpc_call.ts ... ok (2s)
./_tasks/test_examples.ts => examples ... derived.ts ... ok (3s)
./_tasks/test_examples.ts => examples ... raw_rpc_client_call.ts ... ok (11s)
./_tasks/test_examples.ts => examples ... metadata.ts ... ok (12s)
./_tasks/test_examples.ts => examples ... rpc_subscription.ts ... ok (19s)
./_tasks/test_examples.ts => examples ... read_block.ts ... ok (11s)
./_tasks/test_examples.ts => examples ... first_ten_keys.ts ... ok (12s)
./_tasks/test_examples.ts => examples ... ticker.ts ... ok (19s)
./_tasks/test_examples.ts => examples ... block_author.ts ... FAILED (4s)
    error: AssertionError: task failed with status code: 1
        throw new AssertionError(msg);
              ^
        at assert (https://deno.land/[email protected]/testing/asserts.ts:139:11)
        at fn (file:///workspaces/capi/_tasks/test_examples.ts:60:11)
        at async testStepSanitizer (deno:cli/js/40_testing.js:448:7)
        at async TestContext.step (deno:cli/js/40_testing.js:1338:13)
        at async Promise.all (index 17)
        at async file:///workspaces/capi/_tasks/test_examples.ts:37:3
./_tasks/test_examples.ts => examples ... raw_rpc_client_subscription.ts ... ok (19s)
./_tasks/test_examples.ts => examples ... transfer.ts ... ok (31s)
------- output -------

--------------FAILED EXAMPLES OUTPUT START ----------------
Task run deno run -A "examples/block_author.ts"
1zugcaiwmKdWsfuubmCMBgKKMLSef2TEC3Gfvv5GxLGTKMN
error: Uncaught (in promise) Error: failure1
throw new Error("failure1")
      ^
    at file:///workspaces/capi/examples/block_author.ts:43:7
Task run deno run -A "examples/balance.ts"
{
  value: {
    nonce: 0,
    consumers: 0,
    providers: 1,
    sufficients: 0,
    data: { free: 10000000000000000n, reserved: 0n, miscFrozen: 0n, feeFrozen: 0n }
  }
}
error: Uncaught (in promise) Error: failure2
throw new Error("failure2")
      ^
    at file:///workspaces/capi/examples/balance.ts:10:7
-------------- FAILED EXAMPLES OUTPUT END -----------------
./_tasks/test_examples.ts => examples ...----- output end -----
./_tasks/test_examples.ts => examples ... FAILED (31s)

 ERRORS 

examples => ./_tasks/test_examples.ts:19:6
error: Error: 2 test steps failed.
    at runTest (deno:cli/js/40_testing.js:840:11)
    at async runTests (deno:cli/js/40_testing.js:1094:22)

 FAILURES 

examples => ./_tasks/test_examples.ts:19:6

FAILED | 0 passed (18 steps) | 1 failed (2 steps) (31s)

error: Test failed

@ryanleecode
Copy link
Contributor Author

./_tasks/test_examples.ts (uncaught error)
error: Error: Inappropriate ioctl for device (os error 25)
    return Deno.consoleSize().columns;
                ^
    at Object.consoleSize (deno:runtime/js/40_tty.js:[14](https://github.com/paritytech/capi/actions/runs/4009550921/jobs/6885037673#step:9:15):9)
    at ProgressBar.get ttyColumns (https://deno.land/x/[email protected]/mod.ts:237:17)
    at ProgressBar.render (https://deno.land/x/[email protected]/mod.ts:[15](https://github.com/paritytech/capi/actions/runs/4009550921/jobs/6885037673#step:9:16)7:12)
    at file:///home/runner/work/capi/capi/_tasks/test_examples.ts:27:14./_tasks/test_examples.ts (uncaught error)
error: Error: Inappropriate ioctl for device (os error 25)
    return Deno.consoleSize().columns;
                ^
    at Object.consoleSize (deno:runtime/js/40_tty.js:[14](https://github.com/paritytech/capi/actions/runs/4009550921/jobs/6885037673#step:9:15):9)
    at ProgressBar.get ttyColumns (https://deno.land/x/[email protected]/mod.ts:237:17)
    at ProgressBar.render (https://deno.land/x/[email protected]/mod.ts:[15](https://github.com/paritytech/capi/actions/runs/4009550921/jobs/6885037673#step:9:16)7:12)
    at file:///home/runner/work/capi/capi/_tasks/test_examples.ts:27:14

😦

@ryanleecode ryanleecode merged commit 74a9eb2 into main Jan 25, 2023
@ryanleecode ryanleecode deleted the feat/integrated-example-ci-testing branch January 25, 2023 20:59
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

create deno task for running all examples in CI as tests
2 participants