haskellPackages: re-enable tests for many packages

[alexfmpe]

I've been prototyping a snippet that returns all derivations for which doCheck = false, but with the flag set to true instead. Then I build the whole bunch against haskellPackages and see which ones succeed. Out of ~300 ones, close to 100 built.

I didn't remove a good chunk of them that had scary comments, e.g.

• 32 bit only behavior (not sure how we want to test that)
• mutual cycles between test packages
• non determinism in tests

These ones also built but they have doCheck = false right in hackage-packages.nix. Not sure why

• Cabal-3.10
• Cabal-3.12
• Cabal-3.14
• haskell-src-exts
• mwc-random
• req
• websockets

Before and after this PR there's no failures when running

nix-build -E 'with (import ./. {}).haskell.packages.ghc98; [ HList Random123 configuration-tools css-text data-hash direct-sqlite dlist docopt doctest-parallel domain-auth fft fsnotify ghcide haeredes hath hedn hindent hspec-contrib implicit isocline memcache metrics mfsolve monad-dijkstra mysql-haskell netpbm network network-transport-tcp path-io pipes-websockets raven-haskell safecopy sequence-formats servant-auth-client socket store snap-core strive system-fileio system-filepath systemd th-printf token-bucket tree-diff typed-process unagi-chan unicode-data unordered-containers xml-picklers wai-logger webdriver wuss yi-keymap-vim zip-archive ]'

and there are exactly these failures when using ghc910: HList, hindent ghcide

Given the large volume, I wouldn't be surprised if we eventually end up having to add a few back, but hard to tell the reasons for a doCheck when there's no comment.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[alexfmpe]

Note I have only tested on x86_64-linux so far.
Even GHC 9.8 is apparently not cached yet for aarch64-darwin on haskell-updates.

[maralorn]

I fear that some of them are flaky. But I still think we should bite this bullet.

[alexfmpe]

I fear that some of them are flaky. But I still think we should bite this bullet.

My thoughts exactly. I kept a ton that "built on my machine" because their comments said they were flaky, but for the ones that offer no reason, I'd err on the side of taking them off and possibly adding back when they bite us with an explanation rather than keep them until some eventual manual fine-grained sweep.

Maybe we ought to even require a reason added (e.g. issue/pull link) for new overrides so they don't stick around forever. Not too long ago, configuration-common was just over 3k LOC, it's 10% less at this point

[maralorn]

Love it.

[vcunat]

So, I was researching why .tree-diff failed on Hydra. Why did you reenable tests for it? It linked to upstream issue that hasn't been resolved, so why are we doing this dance again?

[vcunat]

I mean, if even upstream doesn't care for years to fix a test failure, will we do something about it ourselves? (except keep restarting the builds manually) When I enable tests, I think about what we do in case they fail, because otherwise they're more harm than good.

[maralorn]

I guess the why is that the test passed and alex probably overlooked the fact that the issue stated it was flaky. Let’s disable them.

[sternenseemann]

I think I'll just disable this test case (specifically) again. Looks like a bug in the exprParser w.r.t. whitespace handling. The parser doesn't seem that important in the first place.

Also, it seems to me that this dontCheck shouldn't have been removed in the first place with the reasoning given in this PR (removing those overrides that give no explanation/upstream issue). I've also noticed in practice that it is difficult making the connection to this change again when running into these types of failures, so I more often than not just restart them instead of remembering that I should probably re-insert a dontCheck…