Add Monorepo Subdirectory Declaration RFC

[greysteil]

As briefly discussed on https://discordapp.com/channels/317475976369930241/317477684999487489.

[evocateur]

I use the homepage field to point monorepo children to the right place (makes npm home <child> work). I guess that's not exactly the use case described here...

[rdela]

@zkat on Discord:

we're working towards having monorepos in npm by end of the year, and this is among a number of details to address before that goes out.

you might consider having this RFC join forces with a push to get npm to recognize subdirectories in git dependency specs, which currently also work only at the root level, and having the ability to target specific directories is a long-requested feature (idr if there's an RFC for it yet)

[zkat]

I'm with #19 (comment) on this one -- it's unclear what value this adds to any particular thing -- homepage seems to be a perfectly reasonable way to handle this. When you're talking about a monorepo, the repo is the repo, after all. Why not use that?

[ljharb]

It's useful to differentiate between "the code" and "the landing page", the same reason there's a "repo" vs a "homepage" - it'd be useful to be able to point clearly to the location of the machine-cloneable code, without bypassing the convenience of linking to a nice human-readable webpage.

[greysteil]

Yeah, the react-dom example is instructive here - the homepage for it is https://reactjs.org/, which seems sensible, but leaves nowhere to put a link to the repo with the directory.

[styfle]

@zkat The homepage property will not work because it is pointing to a website which is fine as-is. We don't expect monorepos to host different websites for each package. For example, it would be very odd for react and react-dom to use different homepages. Let's leave homepage as-is.

This RFC is supposed to solve the problem for Dependabot and any other tool (maybe npm changelog) which needs to perform a git log between two versions in a monorepo.

At this time there is no canonical way to define the path to the package within the git repository of a monorepo. Some projects use lerna, other's have their own means to publish like DefinitelyTyped just throws a link in the README.md

Take a look at dependabot/feedback#121 for some more info on this use-case.

Thanks 👍

[styfle]

I think this looks good and the alternatives are well said.

I don't think that the npm website needs to handle this right now so maybe that can be optional part of this RFC.

The most significant part of this RFC is adding a property to the package.json spec for 3rd party tools to rely upon.

[felixfbecker]

This field would help Sourcegraph with supporting cross-repository go-to-definition.

[greysteil]

@zkat - is there any update to your review in light of the comments above?

To give a very specific use case example, a user contact Dependabot today asking if we could provide a better commit compare view for @types/react in our PRs. We're unable to do so because @types/react is part of a monorepo - the code lives here. If a field was added to the package.json that specified the subdirectory within the monorepo that this package lives, and was included in the registry API, it would allow Dependabot to link to this commit view, which would be much more relevant to the user. Without it we have no way of programatically figuring out that that subdirectory is the relevant one.

[zkat]

So we talked about it further, and I think this is fine! We're actually glad you RFCed this sort of thing!

As far as implementation: this would basically be a PR to documents of what a package.json is, and that's about it -- you shouldn't need to modify the CLI to support this, and arbitrary fields are handled fine by the CLI in the object format of the repository field.

So I think you can consider this ratified! Congratulations, and thanks for the time to write this up, and the excellent examples and ensuing discussion! I'll merge this as soon as the implementation section is updated with what I mentioned in the previous paragraph, and we're otherwise good to go! Let me know when that's done 👍

[ljharb]

@zkat does that mean the website would be updated to look at "directory"?

[greysteil]

@zkat - done, and I'll put a PR into github.com/npm/cli now for that change.

Having this RFC accepted is enough for me to start putting PRs in to monorepos to start using it, so I don't want to say anything that blocks that. I'm very 👍 on merging this as is.

In addition, in future it would be awesome to build on this RFC by including the directory when constructing the repository link from npmjs.com (e.g., for react-dom the repository link would go here, and perhaps the header should be "Source" to remove ambiguity), and include the directory in the API response (e.g., for react-dom). I'm not sure where to PR those changes - can you point me in the right direction, or if the repos are private let me know how I can put in a request? Hopefully the API change one is a one-liner somewhere, with the contentious stuff already approved courtesy of this RFC 🙂

[felixfbecker]

I agree that the website should use this information, which is one of the best ways for people to discover that this feature exists and to convince them to use it. Some projects like Babel use GitHub blob URLs in repository.url which works to link to the folder on GitHub from npmjs.com but it's obviously semantically wrong, because you can't git clone that URL.

The API should already return package.json contents as they were published afaik.

[greysteil]

@zkat - psyched to have this working, and adopted by a bunch of major monorepos. Thanks for all your help on it.

I've just had algolia/npm-search#320 merged, which will mean the directory can be easily included when generating the repository URL for folks using Algolia. Is there a place that I can PR a change to the npmjs.org website, so that repository links there pick it up, too?