Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

service/block, node/p2p: Fix race conditions in TestExtendedHeaderBroadcast and TestFull_P2P_Streams #288

Merged
merged 4 commits into from
Dec 23, 2021

Conversation

ynewmann
Copy link
Contributor

node/p2p/addrs.go Outdated Show resolved Hide resolved
@Bidon15
Copy link
Member

Bidon15 commented Dec 14, 2021

Thank you so much for the PR @jenyasd209 ! 🥳
It would be great if you could:

  1. Update the changelog.md
  2. Fix the lint error on maAnnounce[:]
  3. Take a look on new error thrown during failure of TestExtendedHeaderBroadcast

@ynewmann
Copy link
Contributor Author

ynewmann commented Dec 14, 2021

3. Take a look on new error thrown during failure of TestExtendedHeaderBroadcast

Updated. The problem is I have no failures locally (go v1.16)

Update: checked locally with go v1.17 and still have no issues. As I can see the error is context deadline exceeded, for some reasons 15 seconds weren't enough for the test. Does the test need strictly 15 seconds?

@Wondertan
Copy link
Member

Wondertan commented Dec 15, 2021

@jenyasd209, thank you for taking over bugs!

I am fine with the Race fixes, but unsure about #232 being fixed. The bug described there is not reproducible and happened only once in an unusual situation. Furthermore, I am not sure why this bag happened in the first place. There should not be any panics as append(on the line 50) should not produce such errors, due to the way appends works. Also, a lot of things changed from the point the bug appeared.

Let's proceed by removing your fix for #232 in this PR and closing the issue, as there should be no panics on the code as it is right now and the fix does not really fix anything.

@Wondertan
Copy link
Member

@jenyasd209, also, the error you were facing is test flakiness and can be resolved by re-running all tests using github actions page

@ynewmann
Copy link
Contributor Author

@jenyasd209, thank you for taking over bugs!

I am fine with the Race fixes, but unsure about #232 being fixed. The bug described there is not reproducible and happened only once in an unusual situation. Furthermore, I am not sure why this bag happened in the first place. There should not be any panics as append(on the line 50) should not produce such errors, due to the way appends works. Also, a lot of things changed from the point the bug appeared.

Let's proceed by removing your fix for #232 in this PR and closing the issue, as there should be no panics on the code as it is right now and the fix does not really fix anything.

I tagged #232 because it looks similar to #286 (logs https://gist.github.com/Bidon15/9ec686472ed21ee77662fca18dc5ecf2). There is race condition at the same line where panic happens. Due to append is not thread safe panic can happen if there are a couple calls of this func.

@Wondertan
Copy link
Member

Wondertan commented Dec 16, 2021

@jenyasd209, ooh, I see. Thanks for pointing me to the connection with the races. I made a deeper look and there is clearly a bug/typo. We weren't even using maAnnounce(as you pointed in #288 (comment)) after the append, as you can see there. We should append there not to the maAnnounce, but to maListen, i.e: maListen = append(maListen, maAnnounce...). At the first sight, I was even thinking that this happens. Also, this resolves races, as no more appending of the shared slice will happen.

Wondertan
Wondertan previously approved these changes Dec 20, 2021
Copy link
Member

@Wondertan Wondertan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you!

renaynay
renaynay previously approved these changes Dec 21, 2021
Copy link
Member

@renaynay renaynay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small nit but otherwise good. Unfortunately TestExtendedHeaderBroadcast will no longer be needed as header production will be removed from BlockService anyway with #277 .

CHANGELOG-PENDING.md Outdated Show resolved Hide resolved
@Wondertan
Copy link
Member

@jenyasd209, can you pls make a rebase?

add locks for maps in `mockFetcher` structure

Issue celestiaorg#283
Copy `announce multi addresses` to separate array to avoid data race and return them with `listen multi addresses`.

Issue celestiaorg#286
…s in `mockFetcher`. Add item to CHANGELOG-PENDING.md
@ynewmann ynewmann dismissed stale reviews from renaynay and Wondertan via f68b864 December 23, 2021 12:00
@ynewmann ynewmann force-pushed the jenyasd209/fix-race-conditions branch from 3d6b2c4 to f68b864 Compare December 23, 2021 12:00
@Wondertan Wondertan merged commit 6035222 into celestiaorg:main Dec 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants