Add support for loading multiple components into one repl session

[mpickering]

There are several parts to this patch which are logically distinct but work together to support the overal goal of starting a GHCi session with multiple packages loaded at once.

1. When a user writes "cabal repl " then if the user is using a compiler > ghc-9.4.* then we will attempt to start a multi-session which loads the selected targets into one multi-package session of GHC. 1a. The closure property states that in order to load components p and q into
   the same session that if p depends on z and z depends on q
   then z must also be loaded into the session.
   1b. Only inplace packages are able to be loaded into a multi session (if a component
   z exists then it is already made into an inplace package by
   cabal). Therefore cabal has already engineered that there is source
   code locally available for all packages which we will want to load
   into a session.

2. It is necessary to modify ./Setup configure to allow users to configure a package without having previously built the dependency. Instead, we promise to the configure phase that we will have built it by the time we build the package. This allows us to configure all the packages we intend to load into the repl without building any dependenices which we will load in the same session, because the promise is satisifed due to loading the package and it's dependency into one multi-session which ensures the dependency is built before it is needed.

   A user of ./Setup configure specifies a promised dependency by prepending a "+" to a normal dependency specification. For example:

   '--dependency=+cabal-install-solver=cabal-install-solver-3.9.0.0-inplace'

2a. The ./Setup repl command is modified to allow a user to defer
starting the repl and instead instruct the command to write the
necessary build flags to a file. The option is called
--repl-multi-file <FILEPATH>.

`cabal-install` then invokes this command for each component which
will populate the session and starts a multi-session with all the
arguments together.

3. The solver is unmodified, the solver is given the repl targets and creates a build plan as before. After the solver is completed then in setRootTargets and pruneInstallPlan we modify the install plan to enforce the closure property and mark which dependencies need to be promised.

   • Mark the current components as BuildInPlaceOnly InMemory, which indicates to the compiler that it is to be built in a GHC multi-session.
   • Augment the component repl targets to indicate that components required by the closure property (in addition to normal targets) will be loaded into the repl.
   • Modify the dependency edges in compLibDependencies to indicate which dependencies are the promised ones (which is precisely components which are BuildInPlaceOnly InMemory build styles). This is the field which is eventually used to populate the --dependency argument to ./Setup configure.

---

Please include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
• The documentation has been updated, if necessary.

Please also shortly describe how you tested your change. Bonus points for added tests!

[mpickering]

Tests and documentation are coming, here to test against CI.

[mpickering]

CI seems broken currently:

  ghcup: /usr/local/.ghcup/cache/ghcup-0.0.7.yaml: setModificationTime:setFileTimes: permission denied (Operation not permitted)

[Mikolaj]

Yes, thank you, CI was broken. Please kindly rebase. I hope we have hacked around that.

[Mikolaj]

Actually, this time, the third CI fix will really fix your PR's run. Let me rebase myself...

[Mikolaj]

@mergify rebase

[mergify]

⚠️ This pull request got rebased on behalf of a random user of the organization.
This behavior will change on the 1st February 2023, Mergify will pick the author of the pull request instead.

To get the future behavior now, you can configure bot_account options (e.g.: bot_account: { author } or update_bot_account: { author }.

Or you can create a dedicated github account for squash and rebase operations, and use it in different bot_account options.

[mergify]

rebase

✅ Branch has been successfully rebased

[mpickering]

This should be passing CI and ready for review now @Mikolaj

I am writing a blog post to describe this feature which might be helpful to read as well. I will post a link once it is finished.

[mpickering]

Can anyone explain why this is failing?

https://github.com/haskell/cabal/actions/runs/4262053788/jobs/7417094108

[ulysses4ever]

Very unfortunate :-( The linker warning looks like what was reported on ghc-devs recently. But I'm not sure if this is the reason for the failure.

[Mikolaj]

The only error I can find in this log is

2023-02-24T11:55:18.7344280Z Resolving dependencies...
2023-02-24T11:55:19.4365280Z Error: cabal: Could not resolve dependencies:
2023-02-24T11:55:19.4469500Z [__0] trying: hackage-repo-tool-0.1.1.3 (user goal)
2023-02-24T11:55:19.4474820Z [__1] trying: unix-2.8.0.0/installed-2.8.0.0 (dependency of hackage-repo-tool)
2023-02-24T11:55:19.4477990Z [__2] trying: hackage-security-0.6.2.3 (dependency of hackage-repo-tool)
2023-02-24T11:55:19.4479130Z [__3] trying: hackage-security:+lukko
2023-02-24T11:55:19.4480030Z [__4] next goal: lukko (dependency of hackage-security +lukko)
2023-02-24T11:55:19.4481050Z [__4] rejecting: lukko-0.1.1.3 (conflict: unix =>
2023-02-24T11:55:19.4483550Z base==4.18.0.0/installed-4.18.0.0, lukko => base>=4.5 && <4.18)
2023-02-24T11:55:19.4487260Z [__4] rejecting: lukko-0.1.1.2 (conflict: unix =>
2023-02-24T11:55:19.4488390Z base==4.18.0.0/installed-4.18.0.0, lukko => base>=4.5 && <4.15)
2023-02-24T11:55:19.4489700Z [__4] rejecting: lukko-0.1.1.1, lukko-0.1.1, lukko-0.1 (conflict: unix =>
2023-02-24T11:55:19.4492340Z base==4.18.0.0/installed-4.18.0.0, lukko => base>=4.5 && <4.14)
2023-02-24T11:55:19.4493740Z [__4] fail (backjumping, conflict set: hackage-security, lukko, unix,
2023-02-24T11:55:19.4494550Z hackage-security:lukko)
2023-02-24T11:55:19.4495290Z After searching the rest of the dependency tree exhaustively, these were the
2023-02-24T11:55:19.4495980Z goals I've had most trouble fulfilling: base, hackage-security,
2023-02-24T11:55:19.4496490Z hackage-repo-tool, unix, lukko, hackage-security:lukko

[Mikolaj]

This may or may not fix it: haskellari/lukko#33

If that's it, for now I'd not worry and review and even merge with this failure.

[Mikolaj]

But, the other failures are not about GHC 9.6, so I'd worry about those.

[Mikolaj]

But there is no long. It just says "This job failed". Doh.

[Mikolaj]

Which probably means this is a GHA outage. Restarting the job may help.

[fgaz]

All done. I ended up implementing the "good" fix.

Now we need a maintainer to go over my fixup commits, then I can squash them and finally merge this

#8726 (comment)

[Mikolaj]

The fix indeed looks very civilized, though I got lost reading the code due to how long the replAction functions is. I have low confidence in my assesment, but LGTM.

[ocharles]

🎉 Great work everyone, this is gonna be mega!

[Mikolaj]

Top notch PR and top notch teamwork. Thank you everyone.