-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(test): flaky t0182-circuit-relay.sh #8967
base: master
Are you sure you want to change the base?
Conversation
Would be interesting to see why we don't get a reservation: |
2022-05-12 conversation: this may be relevant for the release if it means a new version of go-libp2p is needed. It may just be a problem with the tests though. |
Who's maintaining https://github.com/ipfs/iptb? cc @BigLep |
@schomatis afaik nobody – it was not touched since I remember (2018 or later) |
2022-05-26 conversation: we're not viewing this as release blocking for 0.13 because even if there are some issues with circuit relay v2, we at least have mostly working hole punching. 0.13 is better than before. The ideal quick thing to do is to see if this is a new regression or long standing. We could use an older version and run Lidel's loop command to see if the flakiness manifests. |
@lidel seems to have already pinpoint this to #8868 per OP. I can get the detailed logs for @marten-seemann early next week but I'm going to push for either us really owning IPTB or replacing it altogether (with testground or whatever we use these days). |
Sounds upstream tests are flaky as well libp2p/go-libp2p#1576 (comment) |
e68bd2f
to
43d520b
Compare
This one-liner hack was the easiest I've found to get the logs of IPTB nodes. After the failed sharness test the logs are in
Note that a successful sharness test run removes this directory (so it's left only after the failed one, copy the log before re-running) and restarting the node ( I'm seeing an error in node 0, but you'd likely need to extend the log subsystem to get more details:
|
2022-06-14 conversation: we're going to wait until the next go-libp2p release before investigating this further since there are changes coming. |
go-libp2p v0.21 landed in #9074, rebasing this PR to see if the problem is still present |
Removes fixup introduced in #8868 (comment) so we can dig into the underlying cause
43d520b
to
ea4d3a9
Compare
This PR removes fixup introduced in #8868 (comment) so we can dig into the underlying cause.
About the flaky test
It started failing every 3-10 run after we switched to go-libp2p 0.19.0 in #8868 – tests in
go-ipfs/test/sharness/t0182-circuit-relay.sh
became flaky.ipfsi 2 swarm connect /p2p/$PEERID_1/p2p-circuit/p2p/$PEERID_0
sometimes fails with:NO_RESERVATION
error is returned, the relay will never work, no matter how long we wait and retry again (only way to fix it is to reboot of the relay node / testbed).Repro (keep running
t0182-circuit-relay.sh
until the error):$ killall ipfs ; i=0; while ./t0182-circuit-relay.sh -v; do echo -n " -----> $i <----\n\n\n"; sleep 1; ((i=i+1)) ; done
cc @aschmahmann @marten-seemann @schomatis – any ideas how to debug this?