`wasm-pack test` running tests multiple times #511

chinedufn · 2019-01-18T10:43:10Z

🐛 Bug description

My tests seems to be running multiple times in Firefox instead of just once.

$ wasm-pack test crates/virtual-dom-rs --firefox --headless

  [1/6] 🦀  Checking `rustc` version...
  [2/6] 🎯  Adding WASM target...
  info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
  [3/6] 🌀  Compiling tests to WASM...
      Finished dev [unoptimized + debuginfo] target(s) in 0.47s
  wasm-bindgen 0.2.31 (b04f60cf2)
  [4/6] ⬇️  Installing wasm-bindgen...
  [5/6] Getting geckodriver...
  [6/6] Running tests in Firefox...
      Finished dev [unoptimized + debuginfo] target(s) in 0.44s
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/virtual_dom_rs-9c24f0dc38062301.wasm
  no tests to run!
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/create_element-c831bfe39cb431f1.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test create_element::click_event ... ok
  test create_element::div_with_properties ... ok
  test create_element::nested_divs ... ok

  test result: ok. 3 passed; 0 failed; 0 ignored
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/diff_patch-ac2fa5a869b98ee0.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test diff_patch::replace_with_children ... ok
  test diff_patch::append_sibling_text_nodes ... ok
  test diff_patch::append_text_node ... ok
  test diff_patch::text_node_siblings ... ok
  test diff_patch::append_children ... ok
  test diff_patch::remove_attributes ... ok
  test diff_patch::truncate_children ... ok
  test diff_patch::replace_child ... ok

  test result: ok. 8 passed; 0 failed; 0 ignored
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/events-35fa407b92336d6e.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test events::on_input ... ok
  test events::on_input_custom ... ok

  test result: ok. 2 passed; 0 failed; 0 ignored
|
no tests to run!
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 3 tests

test create_element::click_event ... ok
test create_element::div_with_properties ... ok
test create_element::nested_divs ... ok

test result: ok. 3 passed; 0 failed; 0 ignored
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 8 tests

test diff_patch::replace_with_children ... ok
test diff_patch::append_sibling_text_nodes ... ok
test diff_patch::append_text_node ... ok
test diff_patch::text_node_siblings ... ok
test diff_patch::append_children ... ok
test diff_patch::remove_attributes ... ok
test diff_patch::truncate_children ... ok
test diff_patch::replace_child ... ok

test result: ok. 8 passed; 0 failed; 0 ignored
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 2 tests

test events::on_input ... ok
test events::on_input_custom ... ok

test result: ok. 2 passed; 0 failed; 0 ignored

🤔 Expected Behavior

Tests run once

👟 Steps to reproduce

git clone https://github.com/chinedufn/percy
wasm-pack test crates/virtual-dom-rs --firefox --headless

🌍 Your environment

Include the relevant details of your environment.
wasm-pack version: 0.6.0
rustc version: $ rustc 1.33.0-nightly (2d3e909e4 2018-12-22)

The text was updated successfully, but these errors were encountered:

ashleygwilliams · 2019-01-18T15:07:21Z

@chinedufn thanks for filing this! we'll take a look soon- i've assigned @fitzgen because he knows the most about the test command. i'll attempt to repro now and comment here with my experience. could you let us know what operating system youre on? it might not matter but it's always nice to have all the possible info!

ashleygwilliams · 2019-01-18T15:09:05Z

oh also- looks like you are running test on https://github.com/chinedufn/percy ? this may be an odd side effect of running the command in the workspace (again not totally sure, but some relevant info for sure!)

ashleygwilliams · 2019-01-18T15:13:35Z

can reproduce on my machine!

Ashleys-MacBook-Pro-2:percy ag_dubs$ wasm-pack test crates/virtual-dom-rs --firefox --headless

[1/6] 🦀 Checking rustc version...
[2/6] 🎯 Adding WASM target...
info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
[3/6] 🌀 Compiling tests to WASM...
Finished dev [unoptimized + debuginfo] target(s) in 0.28s
[4/6] ⬇️ Installing wasm-bindgen...
[5/6] Getting geckodriver...
[6/6] Running tests in Firefox...
Finished dev [unoptimized + debuginfo] target(s) in 0.25s
Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/virtual_dom_rs-9e58ab35df2eb75a.wasm
no tests to run!
Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/create_element-c363d3307c12c5d1.wasm
Executing bindgen...
Visiting http://127.0.0.1:51206...

test create_element::click_event ... ok
test create_element::div_with_properties ... ok
test create_element::nested_divs ... ok

test result: ok. 3 passed; 0 failed; 0 ignored
Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/diff_patch-bea8606093b0cd09.wasm
Executing bindgen...
Visiting http://127.0.0.1:51242...

test diff_patch::replace_with_children ... ok
test diff_patch::append_sibling_text_nodes ... ok
test diff_patch::append_text_node ... ok
test diff_patch::text_node_siblings ... ok
test diff_patch::append_children ... ok
test diff_patch::remove_attributes ... ok
test diff_patch::truncate_children ... ok
test diff_patch::replace_child ... ok

test result: ok. 8 passed; 0 failed; 0 ignored
Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/events-2dad164255426644.wasm
Executing bindgen...
Visiting http://127.0.0.1:51279...

test events::on_input ... ok
test events::on_input_custom ... ok

test result: ok. 2 passed; 0 failed; 0 ignored
|
no tests to run!
Running headless tests in Firefox with /Users/ag_dubs/Library/Caches/.wasm_pack/geckodriver-daada96429827dae/geckodriver
running 3 tests

test create_element::click_event ... ok
test create_element::div_with_properties ... ok
test create_element::nested_divs ... ok

test result: ok. 3 passed; 0 failed; 0 ignored
Running headless tests in Firefox with /Users/ag_dubs/Library/Caches/.wasm_pack/geckodriver-daada96429827dae/geckodriver
running 8 tests

test diff_patch::replace_with_children ... ok
test diff_patch::append_sibling_text_nodes ... ok
test diff_patch::append_text_node ... ok
test diff_patch::text_node_siblings ... ok
test diff_patch::append_children ... ok
test diff_patch::remove_attributes ... ok
test diff_patch::truncate_children ... ok
test diff_patch::replace_child ... ok

test result: ok. 8 passed; 0 failed; 0 ignored
Running headless tests in Firefox with /Users/ag_dubs/Library/Caches/.wasm_pack/geckodriver-daada96429827dae/geckodriver
running 2 tests

test events::on_input ... ok
test events::on_input_custom ... ok

test result: ok. 2 passed; 0 failed; 0 ignored

Ashleys-MacBook-Pro-2:percy ag_dubs$

chinedufn · 2019-01-18T17:06:44Z

what operating system you're on

MacOS

Definitely stealing this dropdown for more details idea!

this may be an odd side effect of running the command in the workspace

Sounds good - other than that migrating my workspace tests to wasm-pack has been seamless.

ashleygwilliams · 2019-01-18T19:08:06Z

cool (i'm also on OSX and yeah details tags are handy)!

is this a behavior you just started seeing happen in the workspace with 0.6.0? if that's the case, i would imagine that means the workspace is the culprit! (it'd be helpful to know if you just started seeing this and it wasn't happening before on a single crate, just to help eliminate some stuff)

chinedufn · 2019-01-18T19:52:23Z

So it turns out that I was previously using the wasm-bindgen and wasm-bindgen-test binaries directly since all of my projects use workspaces.
I just started migrating to wasm-pack this morning.

Sorry that's all I've got on that front - happy to provide any other info that might be useful though!

ashleygwilliams · 2019-01-18T19:59:33Z

no worries, no more info is a type of info still :) - glad to have you using wasm-pack! would love to hear if you have any other suggestions that may have come up during your migration!

fitzgen · 2019-01-18T20:53:22Z

If I strace -f -e trace=execve wasm-pack test --firefox --headless I only get wasm-pack exec'ing cargo test one time, which makes me think this is the wasm-bindgen-test-runner doing something funky maybe. Or a bad interaction between wasm-pack and wasm-bindgen-test-runner somehow.

chinedufn · 2019-01-18T21:06:02Z

Hmmm thank you to both of you for looking into this!

So a few quick things that I'm noticing in case it helps.

The first time we see a batch of tests we're also printing the path to the .wasm binary, the second time we are not:

Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/diff_patch-ac2fa5a869b98ee0.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test diff_patch::replace_with_children ... ok

Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 8 tests

test diff_patch::replace_with_children ... ok

There is also slightly different copy. Waiting for test to finish... vs. running 8 tests so that could also be a clue.

Second thing that I see is this pipe character in the output

  test result: ok. 2 passed; 0 failed; 0 ignored
|
no tests to run!

From a quick search of the codebase the only pipe that I can find that gets printed out is in src/progressbar.rs

cd wasm-pack
rg '\|'
// ... snip ...
src/progressbar.rs
93:                .tick_chars("/|\\- ")

Of course - absolutely no idea if any of this is at all relevant - just pointing out anything that might help y'all think about where this could be stemming from.

fitzgen · 2019-01-22T17:26:52Z

I suspect it is just the output being printed twice, and not the tests actually being run twice. Will test this hypothesis later today.

Also introduces testing of our CLI's output via `assert_cmd`. Expect some follow ups to get more of our testing infrastructure using this incredible crate! Fixes rustwasm#511

Also introduces testing of our CLI's output via `assert_cmd`. Expect some follow ups to get more of our testing infrastructure using this incredible crate! Fixes #511

chinedufn changed the title ~~Disabling NO_HEADLESS mode in wasm-pack test~~ wasm-pack test running tests multiple times Jan 18, 2019

ashleygwilliams added user report needs reproduction wasm-pack test labels Jan 18, 2019

ashleygwilliams modified the milestones: 0.6.1, 0.7.0 Jan 18, 2019

ashleygwilliams assigned fitzgen Jan 18, 2019

ashleygwilliams added bug Something isn't working and removed needs reproduction labels Jan 18, 2019

fitzgen mentioned this issue Jan 23, 2019

Only print cargo test output the once #521

Merged

ashleygwilliams added the PR attached there's a PR open for this issue label Jan 23, 2019

ashleygwilliams modified the milestones: 0.7.0, 0.6.1 Jan 23, 2019

fitzgen closed this as completed in #521 Jan 23, 2019

ashleygwilliams pushed a commit that referenced this issue Feb 27, 2019

Only print cargo test output the once

a5c872b

Also introduces testing of our CLI's output via `assert_cmd`. Expect some follow ups to get more of our testing infrastructure using this incredible crate! Fixes #511

ashleygwilliams pushed a commit that referenced this issue Feb 27, 2019

Only print cargo test output the once

4a7cd6a

Also introduces testing of our CLI's output via `assert_cmd`. Expect some follow ups to get more of our testing infrastructure using this incredible crate! Fixes #511

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`wasm-pack test` running tests multiple times #511

`wasm-pack test` running tests multiple times #511

chinedufn commented Jan 18, 2019 •

edited

Loading

ashleygwilliams commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

chinedufn commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

chinedufn commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

fitzgen commented Jan 18, 2019

chinedufn commented Jan 18, 2019 •

edited

Loading

fitzgen commented Jan 22, 2019

wasm-pack test running tests multiple times #511

wasm-pack test running tests multiple times #511

Comments

chinedufn commented Jan 18, 2019 • edited Loading

🐛 Bug description

🤔 Expected Behavior

👟 Steps to reproduce

🌍 Your environment

ashleygwilliams commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

chinedufn commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

chinedufn commented Jan 18, 2019

ashleygwilliams commented Jan 18, 2019

fitzgen commented Jan 18, 2019

chinedufn commented Jan 18, 2019 • edited Loading

fitzgen commented Jan 22, 2019

`wasm-pack test` running tests multiple times #511

`wasm-pack test` running tests multiple times #511

chinedufn commented Jan 18, 2019 •

edited

Loading

chinedufn commented Jan 18, 2019 •

edited

Loading