-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/tools/gopls/test: flakes due to analyzer panics #56035
Comments
Found new dashboard test flakes for:
2022-09-29 22:28 dragonfly-amd64-622 tools@d49f960b go@a14ac863 x/tools/gopls/test (log)
2022-09-29 22:28 freebsd-amd64-race tools@d49f960b go@66165739 x/tools/gopls/test (log)
2022-09-29 22:28 freebsd-amd64-race tools@d49f960b go@9faf6b79 x/tools/gopls/test (log)
2022-09-29 22:28 freebsd-amd64-race tools@d49f960b go@2cbcf36e x/tools/gopls/test (log)
2022-09-29 22:28 freebsd-amd64-race tools@d49f960b go@9d2c73a9 x/tools/gopls/test (log)
2022-09-29 22:28 freebsd-amd64-race tools@d49f960b go@a14ac863 x/tools/gopls/test (log)
2022-09-29 22:28 freebsd-arm64-dmgk tools@d49f960b go@f6d84451 x/tools/gopls/test (log)
2022-09-29 22:28 openbsd-amd64-70 tools@d49f960b go@157e018f x/tools/gopls/test (log)
2022-09-29 22:28 windows-amd64-race tools@d49f960b go@bf0a1ffb x/tools/gopls/test (log)
2022-09-30 13:58 freebsd-amd64-race tools@b180effd go@9e0149da x/tools/gopls/test (log)
2022-09-30 13:58 freebsd-amd64-race tools@b180effd go@13159fef x/tools/gopls/test (log)
2022-09-30 13:58 freebsd-amd64-race tools@b180effd go@9910f8a4 x/tools/gopls/test (log)
2022-09-30 13:58 windows-amd64-race tools@b180effd go@4e6f9634 x/tools/gopls/test (log)
2022-09-30 13:58 windows-amd64-race tools@b180effd go@9e0149da x/tools/gopls/test (log)
2022-09-30 13:58 windows-amd64-race tools@b180effd go@f6d84451 x/tools/gopls/test (log)
2022-09-30 17:40 freebsd-amd64-race tools@ed98f10c go@4a49af57 x/tools/gopls/test (log)
2022-09-30 17:40 windows-amd64-race tools@ed98f10c go@4a49af57 x/tools/gopls/test (log)
2022-09-30 19:58 dragonfly-amd64-622 tools@3e0355b8 go@cc90e45f x/tools/gopls/test (log)
2022-09-30 19:58 freebsd-amd64-race tools@3e0355b8 go@17622b89 x/tools/gopls/test (log)
2022-09-30 19:58 freebsd-amd64-race tools@3e0355b8 go@e7e554e5 x/tools/gopls/test (log)
2022-09-30 19:58 freebsd-amd64-race tools@3e0355b8 go@18888751 x/tools/gopls/test (log)
2022-09-30 19:58 freebsd-amd64-race tools@3e0355b8 go@738a2caf x/tools/gopls/test (log)
2022-09-30 19:58 freebsd-amd64-race tools@3e0355b8 go@f6d84451 x/tools/gopls/test (log)
2022-09-30 19:58 freebsd-amd64-race tools@3e0355b8 go@c9a62b7e x/tools/gopls/test (log)
2022-09-30 19:58 openbsd-amd64-70 tools@3e0355b8 go@738a2caf x/tools/gopls/test (log)
2022-09-30 19:58 windows-amd64-race tools@3e0355b8 go@17622b89 x/tools/gopls/test (log)
2022-09-30 19:58 windows-amd64-race tools@3e0355b8 go@51297dd6 x/tools/gopls/test (log)
2022-09-30 19:58 windows-amd64-race tools@3e0355b8 go@5f566d35 x/tools/gopls/test (log)
2022-09-30 19:58 windows-amd64-race tools@3e0355b8 go@738a2caf x/tools/gopls/test (log)
2022-09-30 19:58 windows-amd64-race tools@3e0355b8 go@f6d84451 x/tools/gopls/test (log)
2022-10-03 15:32 openbsd-386-68 tools@55e5cff6 go@9d2c73a9 x/tools/gopls/test (log)
2022-10-03 15:32 openbsd-amd64-70 tools@55e5cff6 go@f6d84451 x/tools/gopls/test (log)
2022-10-03 15:32 windows-amd64-race tools@55e5cff6 go@6d8ec893 x/tools/gopls/test (log)
2022-10-04 16:03 freebsd-amd64-race tools@40dabfa8 go@f6d84451 x/tools/gopls/test (log)
2022-10-04 16:03 freebsd-amd64-race tools@40dabfa8 go@9d2c73a9 x/tools/gopls/test (log)
2022-10-04 16:03 windows-amd64-race tools@40dabfa8 go@fa463cc9 x/tools/gopls/test (log)
2022-10-04 16:03 windows-amd64-race tools@40dabfa8 go@f6d84451 x/tools/gopls/test (log)
2022-10-04 16:03 windows-amd64-race tools@40dabfa8 go@9d2c73a9 x/tools/gopls/test (log)
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@89566448 x/tools/gopls/test (log)
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@dfd8aa46 x/tools/gopls/test (log)
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@e9017c24 x/tools/gopls/test (log)
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@fa463cc9 x/tools/gopls/test (log)
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@9d2c73a9 x/tools/gopls/test (log)
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@f6d84451 x/tools/gopls/test (log)
2022-10-04 16:29 windows-amd64-race tools@7f79a022 go@e9017c24 x/tools/gopls/test (log)
2022-10-04 16:29 windows-amd64-race tools@7f79a022 go@0a723816 x/tools/gopls/test (log)
2022-10-04 16:29 windows-amd64-race tools@7f79a022 go@9d2c73a9 x/tools/gopls/test (log)
|
The cause of this is a recent change to report analysis panics during testing (https://go.dev/cl/426018). In light of the frequency of these flakes, we may need to roll that back following investigation. CC @adonovan |
Change https://go.dev/cl/439116 mentions this issue: |
…rash We recently started using bug.Errorf to report analyzer crashes, causing tests to fail sporadically. Unfortunately the log included only the "panicked" message but not the relevant stack of the panic or of the other goroutines, giving limited evidence on which to investigate the underlying cause, suspected to be in package loading, not the analyzer itself. This change causes such crashes to display all stacks so that we can gather more evidence over the next few days before we suppress the crashes again. Updates golang/go#54762 Updates golang/go#56035 Change-Id: I2d29e05b5910540918816e695b458463a05f94d9 Reviewed-on: https://go-review.googlesource.com/c/tools/+/439116 Reviewed-by: Bryan Mills <[email protected]> Run-TryBot: Alan Donovan <[email protected]> TryBot-Result: Gopher Robot <[email protected]> Reviewed-by: Robert Findley <[email protected]> gopls-CI: kokoro <[email protected]>
Change https://go.dev/cl/440178 mentions this issue: |
It has a known bug, and the purpose of the PanicOnBugs is to find unknown bugs. Updates golang/go#54762 Updates golang/go#56035 Change-Id: Id9fdfff478ef52b1d864acee366991808baeb574 Reviewed-on: https://go-review.googlesource.com/c/tools/+/440178 Run-TryBot: Alan Donovan <[email protected]> Reviewed-by: Robert Findley <[email protected]> gopls-CI: kokoro <[email protected]>
Change https://go.dev/cl/440179 mentions this issue: |
We now have a plausible model for the cause of the analysis crashes and are disabling this check until we have submitted a plausible fix. (Specifically: actionHandle.promise may be a stale promise for an older package with the same ID, since snapshot.actions uses the package ID as a key. This causes us to re-use stale inspectors, whose syntax nodes won't be found in TypesInfo.) Updates golang/go#54762 Updates golang/go#56035 Change-Id: I1a7cc1b02b8c322692b1a6bee03f6cd858c08ea0 Reviewed-on: https://go-review.googlesource.com/c/tools/+/440179 TryBot-Result: Gopher Robot <[email protected]> gopls-CI: kokoro <[email protected]> Run-TryBot: Alan Donovan <[email protected]> Reviewed-by: Bryan Mills <[email protected]> Auto-Submit: Alan Donovan <[email protected]>
Change https://go.dev/cl/420538 mentions this issue: |
Most invalidation happens in snapshot.clone, but to be safe we were also invalidating package data when new metadata is set via packages.Load. At the time, I wasn't sure why this was necessary. Now I understand: with experimentalUseInvalidMetadata it is possible that we re-compute invalidated data using stale metadata in between the initial clone and the subsequent reload. I noticed that it was also possible to have stale analysis results after the Load results arrive. Fix this by invalidating analysis results on Load, in addition to packages. Factor out invalidation into a new helper method. Since we believe this may fix analyzer panics, re-enable strict handling of analysis panics during tests. For golang/go#56035 For golang/go#54762 For golang/go#42857 Change-Id: I8c28e0700f8c16c58df4ecf60f6127b1c05d6dc5 Reviewed-on: https://go-review.googlesource.com/c/tools/+/420538 Run-TryBot: Robert Findley <[email protected]> TryBot-Result: Gopher Robot <[email protected]> gopls-CI: kokoro <[email protected]> Reviewed-by: Alan Donovan <[email protected]>
Found new dashboard test flakes for:
2022-10-04 16:29 freebsd-amd64-race tools@7f79a022 go@8bd803fd x/tools/gopls/test (log)
2022-10-04 16:29 windows-amd64-race tools@7f79a022 go@89566448 x/tools/gopls/test (log)
2022-10-04 16:29 windows-amd64-race tools@7f79a022 go@947091d3 x/tools/gopls/test (log)
2022-10-04 18:24 dragonfly-amd64-622 tools@dc88e7b4 go@ddc7d2a8 x/tools/gopls/test (log)
2022-10-04 18:24 freebsd-amd64-race tools@dc88e7b4 go@7d3a5a50 x/tools/gopls/test (log)
2022-10-04 18:24 freebsd-amd64-race tools@dc88e7b4 go@c591d82e x/tools/gopls/test (log)
2022-10-04 18:24 freebsd-amd64-race tools@dc88e7b4 go@d1187438 x/tools/gopls/test (log)
2022-10-04 18:24 freebsd-amd64-race tools@dc88e7b4 go@e7d203f4 x/tools/gopls/test (log)
2022-10-04 18:24 freebsd-amd64-race tools@dc88e7b4 go@947091d3 x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@47a53884 x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@9f78c3cb x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@7d3a5a50 x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@89566448 x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@c591d82e x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@3380ee25 x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@058f019e x/tools/gopls/test (log)
2022-10-04 18:24 windows-amd64-race tools@dc88e7b4 go@841873f2 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 freebsd-amd64-race tools@9ffaf69c go@89566448 x/tools/gopls/test (log)
2022-10-13 20:55 openbsd-amd64-68 tools@9ffaf69c go@89566448 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 openbsd-386-68 tools@9ffaf69c go@005c78d8 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 windows-amd64-race tools@9ffaf69c go@947091d3 x/tools/gopls/test (log)
2022-10-13 20:55 windows-amd64-race tools@9ffaf69c go@005c78d8 x/tools/gopls/test (log)
2022-10-13 20:55 windows-amd64-race tools@9ffaf69c go@89566448 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 openbsd-amd64-70 tools@9ffaf69c go@c79b2009 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 windows-amd64-race tools@9ffaf69c go@7feb6872 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 dragonfly-amd64-622 tools@9ffaf69c go@a4b4717f x/tools/gopls/test (log)
|
Change https://go.dev/cl/442794 mentions this issue: |
Found new dashboard test flakes for:
2022-10-13 20:55 openbsd-amd64-70 tools@9ffaf69c go@201759f9 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 freebsd-amd64-13_0 tools@9ffaf69c go@e9cdca9a x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 freebsd-amd64-race tools@9ffaf69c go@947091d3 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 aix-ppc64 tools@9ffaf69c go@e612d212 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 openbsd-386-70 tools@9ffaf69c go@947091d3 x/tools/gopls/test (log)
|
Found new dashboard test flakes for:
2022-10-13 20:55 openbsd-386-70 tools@9ffaf69c go@947091d3 x/tools/gopls/test (log)
|
We now understand the longstanding race causing panics in gopls analysis (the cache key for analysis handles is not correct, and can lead to cache hits for analysis of stale packages). It will take a little while to redesign our analysis caching, the simplest solution being to hang analysis results off of the actual package they are analyzing. In the meantime, suppress the panic as before, to eliminate flakes. For golang/go#55201 For golang/go#56035 Change-Id: I96600f186f9f31b67dae10c0a95a14fa73630571 Reviewed-on: https://go-review.googlesource.com/c/tools/+/442794 Run-TryBot: Robert Findley <[email protected]> Reviewed-by: Alan Donovan <[email protected]> gopls-CI: kokoro <[email protected]> TryBot-Result: Gopher Robot <[email protected]>
Closing optimistically, as we think this is fixed now. (thankfully, watchflakes will reopen if not!) |
Found new dashboard test flakes for:
2022-11-10 18:37 linux-ppc64le-buildlet tools@d56532ab go@1e11eaa9 x/tools/gopls/test (log)
|
That most recent one seems to be a symptom of #56793. |
Issue created automatically to collect these failures.
Example (log):
— watchflakes
The text was updated successfully, but these errors were encountered: