Module naming conventions for GHC base libraries

[simonpj]

This proposal suggests naming conventions for the libraries

• ghc-internal
• ghc-experimental
• base
• ghc

Rendered form here

[Ericson2314]

I am not so sure about this. bifurcating GHC into "public API" vs "internals" is just one way to go about organizing its code (and I personally don't think it is a good one). Regardless of whether it is good or bad, the module naming I think should be independent of such matter of internal organization: modules in GHC should be named the same way whether they are more or less stable.

(If we really had a stable deivions to make, we would have a separate library per https://nikita-volkov.github.io/internal-convention-is-a-mistake/, but we are nowhere near that.)

"API" is also, I think, bad bargain, the "application" and "programming" don't mean anything in particular, and "interface" is quite redundant when all module naming is for interface purposes.

---

GHC. I still think is the best name for the compiler proper. GHC stands for Glasgow Haskell Compiler --- it's in the name.

The counterparts to ghc-internals/ghc-prim are called things like libgcc (GCC) or libcompiler-rt (Clang). The lib prefix of C/C++ libraries is doing some work here, compiler-rt is rather generic. but I think the rt is good. We have "RTS" for just the C part, but it might almost be better to think of the runtime as something encompassing all of rts + ghc-prim + ghc-internals. None of this stuff is part of the compiler proper, but all of it is unstable "support code" propping up the code the user actually wrote. And functionality does in fact move between the various parts of it with some fluidity --- for example @dcoutts is working on moving some IO stuff back into C for the threaded runtime (like the unthreaded runtime does today) for more performance io_uring support.

So I don't yet have an alternative name I really like, but given the above, maybe something like GHR for "Glasgow Haskell Runtime" would make sense.

[gbaz]

"bifurcating GHC into "public API" vs "internals" is just one way to go about organizing its code" -- I think/hope that the ghc api proposal is not this. Rather, it is to create a new set of modules with new functions that promise to be a stable (and usable) api, and which are not intended for internal use by ghc. That is to say, ghc itself does not dogfood the ghc api (although ghci may choose to do so) and instead the functions provided by the api are designed from the start for external consumption, perhaps glossing over certain tricky but rare bits at first, especially ones highly dependent on less stable datatypes. As such, having a distinct namespace of some sort for the API is very important regardless.

I can imagine such a package perhaps evolving over time to be a distinct package on top of ghc, but that seems orthogonal to this discussion. And, well, now that I think about it, since the API proposal currently is stalled, we probably need not worry about it at all in this discussion -- I just wanted to clarify what the idea is, last I understood it, for the sake of future discussion.

[simonpj]

"bifurcating GHC into "public API" vs "internals" is just one way to go about organizing its code" -- I think/hope that the ghc api proposal is not this. Rather, it is to create a new set of modules with new functions that promise to be a stable (and usable) api, and which are not intended for internal use by ghc. That is to say, ghc itself does not dogfood the ghc api (although ghci may choose to do so) and instead the functions provided by the api are designed from the start for external consumption, perhaps glossing over certain tricky but rare bits at first, especially ones highly dependent on less stable datatypes. As such, having a distinct namespace of some sort for the API is very important regardless.

Yes, @gbaz has it exactly right. GhcAPI.* is a shim around the internal modules GHC.*, one that has stronger stability guarantees. But those internal modules must remain available because we can't predict everything that a client of the ghc library may want to do. I hope that if someone finds they can only do something through GHC.* they will petition the GHC API working group (still in the womb) to add a suitable function to GhcAPI.*.

We must be able to reorganise and refactor GHC's internals without constraint. At the moment we simply don't know what clients of ghc are relying on, so we may well mess up their lives without ever knowing. If we had GhcAPI.* we'd know which modules we needed to take (much) more care with.

The details are not important; I'm just using this proposal as a way establish, in principle. a namespace for a stable GHC API.

[Ericson2314]

I like ignoring it for now; I would like it if this proposal didn't mention that at all, and if/when we create that stable layer on top we worry about its module names then.

That brings us back to using GHC.* both for the compiler (internals) and for ghc-internals/ghc-prim. In another thread, this was brought up as a deficiency. This proposal doesn't yet talk about this problem, but shouldn't it?

[gbaz]

Thanks Simon. All this looks pretty eminently rational to me. I especially like proposal 3 that we shouldn't go around renaming modules that already exist rapidly, but rather should over time have individual discussions as we start to disentangle things and migrate them across packages.

[nomeata]

I’d recommend to make the Experiment a suffix.

Rationale: Data.Tuple.Experimental is an companion/extension of Data.Tuple; some exports may move from one to the other. Many developers sort their imports alphabetically. Making this a suffix means all Data.Tuple-related imports are next to each other.

Ok (omitting explicit import lists and qualifiers):

import Control.Applicative
import Control.Arrow
import Experimental.Control.Applicative
import Experimental.Foreign.C
import Data.Tuple
import Foreign.C

Better:

import Control.Applicative
import Control.Applicative.Experimental
import Control.Arrow
import Data.Tuple
import Foreign.C
import Foreign.C.Experimental

Also, maybe people will use the idiom

import Data.List qualified as L
import Data.List.Experimental qualified as L

which is also nicer if both qualified as L are next to each other.

[simonpj]

Clearly this is a matter of taste. Personally I find it easier to think of Experimental.* as a complete sub-tree of modules, all experimental. And it's consistent with using a prefix GHC.* or GhcAPI.* elsewhere.

But it doesn't matter what I feel provided whatever we do

• Satisfies the maximum number of users
• Is carried through consistently (e.g. all modules in ghc-experimental end in .Experimental.

I would love to hear from others about prefix-vs-postfix.

[nomeata]

Ah, below you write

• This sort of naming is conventionally used to distinguish modules within a package, not between packages.

which is only partly true – the module namespace prefixes are not always package prefixes; quite a few packages have modules in various logical places (Data. and Control.). The module namespace groups things by concept (or at least tries to).

And due to ghc-experimental’s nature it’s expected that it defines things both in Data and Control and Foreign.

[Ericson2314]

I also think suffix is better. We've had very good luck not having package-distinct prefixes --- I am not sure why practice has gone better than theory! --- so I think it is OK to keep on doing that here.

[adamgundry]

I'm also in favour of the .Experimental suffix. I imagine Data.List.Experimental could re-export the contents of Data.List and add some extra experimental goodies; it's then a very small delta to switch between import Data.List and import Data.List.Experimental, whereas import Experimental.Data.List would appear somewhere completely different if imports are sorted.

[tek]

I think the reasoning for using a suffix is convincing.

[googleson78]

Is there some particular reason to use a "merged prefix"(GhcAPI.*) instead of using the module structure to express the relationship (GHC.API.* - this is the API of GHC)? The latter feels more natural.

[nomeata]

This is cosmetics, but I find it strange that suddenly Ghc in GhcAPI is cased like this.

I would find GHC.API. more natural, but I understand that you want to avoid having the public API inside the namespace for the rest of GHC.

There is the Language. prefix commonly used by most compiler-like-libraries (e.g. haskell-src-exts, ghc-parser, ghc-opts). We could reasonably join this namespace instead of inventing our own, maybe using Language.Haskell.GHC.* or, to avoid overly long names, Language.GHC.*.

[adamgundry]

None of the alternatives for proposal 4 are really completely convincing. Moreover, I don't think we need to make a decision now, given that there is no immediate prospect of the GHC API redesign happening. If it does happen, whoever drives it forward will be best placed to decide on naming. So I think this proposal could simply establish the principle that it will have a distinguishable module naming convention, but not yet make a concrete choice.

[Ericson2314]

Yes I would prefer it only talk about libraries we've planned on making.

[adamgundry]

I'm also in favour of the .Experimental suffix. I imagine Data.List.Experimental could re-export the contents of Data.List and add some extra experimental goodies; it's then a very small delta to switch between import Data.List and import Data.List.Experimental, whereas import Experimental.Data.List would appear somewhere completely different if imports are sorted.

[adamgundry]

None of the alternatives for proposal 4 are really completely convincing. Moreover, I don't think we need to make a decision now, given that there is no immediate prospect of the GHC API redesign happening. If it does happen, whoever drives it forward will be best placed to decide on naming. So I think this proposal could simply establish the principle that it will have a distinguishable module naming convention, but not yet make a concrete choice.

[adamgundry]

Is this really the case? At the moment ghc-internals doesn't yet exist, so nobody depends on it! And the whole point is that most users should not depend on it, they should depend on base instead!

So why couldn't we establish the convention that ghc-internals uses GHC.Internal.* as its preferred module prefix, at least for new modules? We could still move over existing modules from base without renaming them if that was practically easier, and then rename them later. Renaming modules in ghc-internals should be fairly cheap as users aren't supposed to import them anyway, so the only package affected should be base.

[tek]

If we land on GHC.Internal and Foo.Experimental, I feel like the package should be named ghc-internal without the s, for consistency

[simonpj]

Is this really the case? At the moment ghc-internals doesn't yet exist, so nobody depends on it! And the whole point is that most users should not depend on it, they should depend on base instead!

I anticipate that many of the modules in base will move to ghc-internals, leaving behind a shim module. But I suppose it is true that when we move, say GHC.Base into ghc-internals we could

• Rename it to GHC.Internal.Base, in ghc-internals
• Leave behind a shim module in base that
  • Is called GHC.Base
  • Imports GHC.Internal.Base and re-exports it all

Hmm. That's true. Moreover, because for a long time (possibly forever) we will have modules like GHC.Base in base (for back-compat reasons) it would be good if the module in ghc-internals had a different name.

[Ericson2314]

I think it's OK if deprecated modules have non-standard names. If anything it is good; it adds an extra signal saying what ought to be used.

[adamgundry]

• Leave behind a shim module in base that
  • Is called GHC.Base
  • Imports GHC.Internal.Base and re-exports it all

Yes. Ideally, GHC.Base will re-export individual declarations from GHC.Internal.Base using an explicit export list (not a module export). That way any new definitions added to GHC.Internal.Base won't accidentally leak into the base API.

So I think it's positively helpful to give the internal modules new names. Another example: we could add module-level DEPRECATED pragmas to the base shim modules prior to removing them.

[simonpj]

I think that only six people have contributed to this discussion so far. I bet that many more have an opinion. Please do say what you think! Reminder:

This proposal suggests naming conventions for the libraries

• ghc-internals
• ghc-experimental
• base
• ghc

Rendered form here

[simonpj]

I'm also in favour of the .Experimental suffix. I imagine Data.List.Experimental could re-export the contents of Data.List and add some extra experimental goodies; it's then a very small delta to switch between import Data.List and import Data.List.Experimental, whereas import Experimental.Data.List would appear somewhere completely different if imports are sorted.

So to be clear, you advocate:

• Use a .Experimental suffix in ghc-experimental
• Use a GHC.Internal. prefix in ghc-internals

Is that right? It seems superficially inconsistent, and yet each on its own makes sense.

Modules in the ghc library (GHC API) are all called GHC.* (e.g. GHC.Tc.Gen.App) and I don't think we can change that.

What do others think? It's all about what colour to paint the bikeshed, but we do need to agree it, otherwise we'll get a mess of inconsistent approaches in different modules.

[adamgundry]

So to be clear, you advocate:

• Use a .Experimental suffix in ghc-experimental
• Use a GHC.Internal. prefix in ghc-internals

Is that right? It seems superficially inconsistent, and yet each on its own makes sense.

Exactly. I'm not particularly worried about making these consistent, since they serve quite different purposes.

[michaelpj]

If we adopt the proposal of having a module warning for the internal modules, then perhaps we don't need a ghc-internals-specific module prefix and we could just use GHC. Not sure if that's a good idea.

[Ericson2314]

That is true, but the "broader runtime" (which this is) vs the compiler itself are very different things. I think GHC.Internals isn't even different enough (say we wanted to use GHC.Internals for extra unstable parts of GHC itself?) but at least it is something to differentiate.

[michaelpj]

Is that right? It seems superficially inconsistent, and yet each on its own makes sense.

I think there is a logic to it:

• Foo.Experimental is a close cousin or variant of of Foo: you might well want to use both together, and in the long run the latter will hopefully migrate to the former
• GHC.Internal and GHC may have little to do with each other apart from the fact that the latter uses the former. There might be GHC.Foo and GHC.Internal.Foo, but more likely there will be completely independent sets of modules.

[numlit2]

Use a .Experimental suffix in ghc-experimental

Data.List.Experimental is not as strong as Experimental.List. In a code review with sorted imports scattered X.Y.Experimental are easier to miss than a group of Experimental.*.

If you want to optimize for experts do X.Y.Experimental, if you want to make it more obvious and easier to spot use Experimental.*. I think the introduction of unstable dependencies should be as prominent as possible.

[tek]

it would be possible to expose both…

[nomeata]

if you want to make it more obvious and easier to spot use Experimental.*

Or EXPERIMENTAL. maybe? Too bad we can't include ⚠️ in the module name.

[JakobBruenker]

Instead of relying on the module name to warn users, another solution could be to add {-# WARNING in "x-experimental" Data.List.Experimental "<some message>" #-}, and have users use Wno-x-experimental if they are sure they want to use it. I don't think we currently have module-scoped warnings though.
(Edit: I think technically ghc-proposals/ghc-proposals#134 allows you to do this, although probably not in a super convenient way)

[vanceism7]

I like the idea of X.Y.Experimental, it feels more appropriately categorized and I kind of doubt someone would miss it. A simple search/replace would probably find them anyways. Perhaps we could even add an experimental warning flag that could be triggered on when desired so the compiler could show warnings for anything deemed experimental that was being used in the code.

On another note:
I'm just bike-shedding here, but it seems like ghc-experimental should actually be called "base-experimental". Because the code living in it has a life-goal of moving into base, and users are encouraged (or atleast not forbidden) to use it in their own code.

I think this naming change helps to enforce the policy that users shouldnt be depending on ghc libraries but the base libs instead

[tomjaguarpaw]

it seems like ghc-experimental should actually be called "base-experimental". Because the code living in it has a life-goal of moving into base

Whose life-goal? If anyone's, it's the goal of GHC HQ. It's not the goal of CLC, who have purview over base. Therefore I don't think the name "base" should be co-opted here.

users are encouraged (or atleast not forbidden) to use it in their own code

I truly hope the name "ghc-experimental" sounds discouraging because I wouldn't like users to depend on this library lightly!

[vanceism7]

Whose life-goal? If anyone's, it's the goal of GHC HQ. It's not the goal of CLC, who have purview over base. Therefore I don't think the name "base" should be co-opted here.

I truly hope the name "ghc-experimental" sounds discouraging because I wouldn't like users to depend on this library lightly!

ghc-experimental definitely sounds like it's something users shouldn't be referencing in their code. Even base-experimental sounds pretty scary to me. But the quote from the rendered makes it seem like it's ok to use it

• Functions and types in here are usually candidates for later transfer into base. But not necessarily: if a collection of functions is not adopted widely enough, it may not be proposed for a move to base.
• It is user-facing (user are encouraged to depend on it), but its API is less stable than base.

Based on that wording, it sounds like the intent is that the code from experimental eventually matures and takes a home in base, and that it's ok to use it with the warning that it might change on you.

If that's the case, I think base-experimental is more appropriate. If it's supposed to be super scary and normal devs shouldn't be depending on it, then I agree with calling it ghc-experimental. But ghc-experimental sounds like it shouldn't even be user facing

[Bodigrim]

I'm just bike-shedding here, but it seems like ghc-experimental should actually be called "base-experimental". Because the code living in it has a life-goal of moving into base, and users are encouraged (or atleast not forbidden) to use it in their own code.

It seems you are reading the purpose of ghc-experimental as being a staging ground for inclusion into base. I don't think it's correct: the primary goal for ghc-experimental is to experiment with API required to implement GHC proposals. It may never end up in base (after all, it is only aspirational, "ultimate" goal to do so), and it's more likely than not that, building upon results of experiments, the final design approved for inclusion in base will be much different from one in ghc-experimental.

If it's supposed to be super scary and normal devs shouldn't be depending on it, then I agree with calling it ghc-experimental.

It depends whether experiments scare you, right? If you depend on ghc-experimental, you agree to participate in an experiment. Sure, this experiment is not supposed to be "super scary", and it is designed to bring positive outcomes in mind, but there is always a chance, and we want consumers to be conscious about it.

The heuristics is simple: ghc-internal for GHC developers, base for CLC, and ghc-experimental for GHC Steering Committee. If there is ghc in the package name, it is guided by a community body with GHC in its name. I'm quite strongly opposed to renaming to base-experimental.

[vanceism7]

I'm just bike-shedding here, but it seems like ghc-experimental should actually be called "base-experimental". Because the code living in it has a life-goal of moving into base, and users are encouraged (or atleast not forbidden) to use it in their own code.

It seems you are reading the purpose of ghc-experimental as being a staging ground for inclusion into base. I don't think it's correct: the primary goal for ghc-experimental is to experiment with API required to implement GHC proposals. It may never end up in base (after all, it is only aspirational, "ultimate" goal to do so), and it's more likely than not that, building upon results of experiments, the final design approved for inclusion in base will be much different from one in ghc-experimental.

If it's supposed to be super scary and normal devs shouldn't be depending on it, then I agree with calling it ghc-experimental.

It depends whether experiments scare you, right? If you depend on ghc-experimental, you agree to participate in an experiment. Sure, this experiment is not supposed to be "super scary", and it is designed to bring positive outcomes in mind, but there is always a chance, and we want consumers to be conscious about it.

The heuristics is simple: ghc-internal for GHC developers, base for CLC, and ghc-experimental for GHC Steering Committee. If there is ghc in the package name, it is guided by a community body with GHC in its name. I'm quite strongly opposed to renaming to base-experimental.

Got it. I totally agree with the logic. I think some of those finer points should be included in the description of ghc-experimental just to make it a little more clear. Thanks for the explanation!

[nomeata]

The heuristics is simple: ghc-internal for GHC developers, base for CLC, and ghc-experimental for GHC Steering Committee. L

The apparent inevitability of Conway's Law amazes me every time.

[simonpj]

I have done a full refresh of this library-module-naming-convention proposal, and pushed the result.

I think we are done. Any final thoughts?

[adamgundry]

Thanks @simonpj, this looks great.

[clyring]

I don't think this is a good example; the motivation for that proposal is not convincing unless the existing dataToTag# and getTag exposed from ghc-prim and base get improved types.

[simonpj]

OK. I have replaced this example with the exceptions proposal.

[bgamari]

I, for one, am quite happy with where this proposal ended up. Thanks, @simonpj!

[Ericson2314]

I am too.

• Prefix for ghc-internals
• Suffix for ghc-experimental
• GHC API parts are non-normative

Thanks @simonpj!

[simonpj]

Great -- let's take this proposal as decided.

[TeofilC]

Suggested change

	* Modules in GHC's internal libraries (``ghc-prim``, ``ghc-internal`` etc) should be of form ``GHC.Internal*``.
	* Modules in GHC's internal libraries (``ghc-prim``, ``ghc-internal`` etc) should be of form ``GHC.Internal.*``.

I think there's a fullstop missing here

[gbaz]

merging as approved.