Version bounds are too strict; broken with stackage; broken on NixOS; can be updated

[Kiwi]

Hi, I (try to) maintain matterhorn for NixOS. I frequently have to jailbreak (tell cabal to kick rocks and use newer versions than the .cabal wants) matterhorn and a few of its dependencies to get it to build because nixpkgs defaults to the stackage version of packages and if something needs a version only on hackage or the cabal file is too strict it requires special treatment or else it will break (and maybe even then it might still break or not be fixable). Then at some point the versions get updated on stackage or in the projects and and then our patches aren't required so they get removed (or it could happen that the jailbreak doesn't work at all because api change means new version doesn't work) and then stackage updates and then they break again. 🙃 So overrides are supposed to have a comment to an issue/pr so we know why a patch is there and when it isn't necessary anymore (so here I am) comment explaining the issues Anyway I'm supposed to say that it would be easier if it was on stackage but I know some people don't like stack/stackage or have a reason they don't want to use it and I am one of those people so it's hard for me to feel good suggesting it to people. ahem. >.>

This is what happens with no jailbreak.

Setup: Encountered missing or private dependencies:
aeson >=1.2.3.0 && <1.5,
aspell-pipe ==0.4.*,
skylighting-core ==0.8.*,
strict ==0.3.*,
vty ==5.31.*

After I jailbreak it will build and run.

[kiwi@mvp-nixos nixpkgs]$ nix-build-haskell matterhorn
/nix/store/0rsqzhgrh6b4n9hwihd3hc86x92hvbag-matterhorn-50200.11.0

kiwi@mvp-nixos nixpkgs]$ nix-build-haskell aeson aspell-pipe skylighting-core strict vty
/nix/store/kdlv4s53whzllsvv5xqnn15f6lv9z7zi-aeson-1.5.4.1
/nix/store/m9ppl1f9m9qbirls684rd28854ll8sih-aspell-pipe-0.6
/nix/store/lk68kzpnjfk84w778z56ff28xlyiikgp-skylighting-core-0.10.0.3
/nix/store/n0g173ccqf0rj6dwkdjc13l8pixp5df9-strict-0.4
/nix/store/3mxj19fqdb6ramv0xyczzarx546rbd9j-vty-5.32

I played around with cabal-edit and cabal-bounds and I think a lot of the bounds have updates and can be updated to the newest. I had to tweak the cabal file before/after (depending on which tool I used) sometimes to make it work. There are a few things they don't like about the existing cabal file. I could maybe make a PR if that'd help but I don't know this project at all and I might change things more than you like. :)

I don't even use matterhorn heh. This is all @evils fault! 😝

tl;dr it would be really great if the version bounds were updated and kept updated so that it keeps working with newer api compatible version upgrades 😄 ❤️

[jtdaugherty]

Thanks for your report! I can see how our practices are causing some difficulty.

I want to do what I can to help, but I want to just state for the record that we do not intend to make matterhorn buildable with an arbitrary range of dependency versions. We deliberately maintain very tight bounds. We are happy to advance those bounds, but we will still want to maintain tight bounds even if the upper bounds are raised. We just don't have the resources to QA a wide variety of build configurations, and since we don't intend for anyone to use matterhorn as a library, we don't think maintaining wide dependency bounds is necessary.

I also want to state for the record that supporting Stackage is not one of our goals. If matterhorn builds against the set of things in a particular version of Stackage, that's great! But if it doesn't, that's too bad but changing our dependency versions to accommodate that is not really compatible with our goals due to our QA limits that I mentioned above. In this case it just looks like Stackage has moved on to newer and better versions of various things (like aspell-pipe, which we wrote and maintain just for matterhorn) but it will be a little while before we put out a matterhorn release that uses those newer packages. The next release of matterhorn will upgrade its bounds for some of the packages you listed above.

I am curious to know what sort of Cabal file issues you're running into.

[Kiwi]

That's hilarious that about aspell-pipe lol. I know there are a lot of people that don't have the goal to be on stackage and they have good reasons I do not fault them for and likely agree. :) I've... strongly disliked stack/stackage for longer than I've even known about NixOS and even more so now. :|

Uh with cabal-edit it would introduce a symbol (^>=) some places that's only valid in cabal-version: >= 2.0 and there's not a lower bound on vector and then sometimes I'd have to change things afterwards or it wouldn't build because it'd reject dependencies. I don't know that it's so much an error in the cabal file as it is a bug in the tools. maybe some of both. idk

[nix-shell:~/projects/github.com/matterhorn-chat/matterhorn]$ cabal-edit lint
aeson : Consider upgrading upper bound to latest version 1.5.4.1
bytestring : Consider upgrading upper bound to latest version 0.11.0.0
mattermost-api : Explicit version detected. Considering allowing a range.
random : Consider upgrading upper bound to latest version 1.2.0
skylighting-core : Consider upgrading upper bound to latest version 0.10.0.3
vector : Upper bound only for package with >1.0 admits all previous versions. Add a lower bound to major version.

[jtdaugherty]

Okay, thanks. Looking at the list of messages, we could definitely add a lower bound for vector. We can also see about upgrading some of the others.