Sites packages rules

[guybedford]

Currently the sites packages examples seem to map into folders, but don't specify the exact JS file to use:

site({
  "jquery":  "https://cdn.example.com/jquery/v/2.1.1"
});

Does this mean that import $ from 'jquery' would this result in a request to:

• https://cdn.example.com/jquery/v/2.1.1
• https://cdn.example.com/jquery/v/2.1.1.js
• https://cdn.example.com/jquery/v/2.1.1/index.js

If the latter, are we enforcing a main style?

Further, can we import subpaths within the package like import sizzle from 'jquery/sizzle.js' or import sizzle from 'jquery/sizzle'?

[caridy]

we have discussed this in the past, and we all agreed that we can't support the same level of ambiguity that node/io.js supports today. in other words, this is in user land, and they can do whatever they want.

site definition as configuration is preferred

If they want to do import $ from 'jquery', then will have to do this:

site({
  "jquery":  "https://cdn.example.com/jquery/v/2.1.1/jquery.js"
});

Otherwise how can you control what distribution do you want to load? Maybe you want the raw version in development, and the minified version in prod, and that should not required changes in your modules. Essentially using something like import $ from "jquery/index.min.js" should be considered a bad practice.

site definition for subpath

I will argue against any composition of the path. Even if we add support for rules, e.g.:

site({
  "jquery":  "https://cdn.example.com/jquery/v/2.1.1/index.js",
  "jquery/*":  "https://cdn.example.com/jquery/v/2.1.1/"
});

which can allow things like:

import $ from 'jquery';
import slideshow from "jquery/plugins/slideshow.js";

I will consider this a footgun, because they will have to explicit call out the extension of the subpath. An alternative solution will be something that I think @guybedford already implemented in SystemJS:

site({
  "jquery":  "https://cdn.example.com/jquery/v/2.1.1/index.js",
  "jquery/*":  "https://cdn.example.com/jquery/v/2.1.1/plugins/*.min.js"
});

which can allow to do:

import $ from 'jquery';
import slideshow from "jquery/slideshow";

[johnjbarton]

@caridy I'm confused by your description. Is my re-write correct?

site definition as configuration is preferred

To import $ from 'jquery', first establish your selected 'jquery' distribution as a site key:

site({
  "jquery":  "https://cdn.example.com/jquery/v/2.1.1/jquery.js"
});

Then import $ using the key:

import $ from 'jquery';

In this way you control what distribution to load. You may want the raw version in development and the minified version in production. That change should not require changes in your modules. Using something like

import $ from "jquery/index.min.js" 

unnecessarily locks your module to a particular distribution format.

[caridy]

sorry @johnjbarton, I'm rushing today jejeje.

your interpretation is correct. the deal is that the site configuration is an imperative call, that means users have the flexibility to compose the configuration depending on the runtime, while modules are portable piece of code that should not worry about the runtime configuration where they run.

[guybedford]

@caridy that makes sense and I see your points. Would it still be possible to define jquery/sizzle as a subpath in the sites table?

[caridy]

@guybedford yes. e.g.:

site({
  "jquery":  "https://cdn.example.com/jquery/v/2.1.1/index.js",
  "jquery/sizzle":  "https://cdn.example.com/sizzle/v/1.2/sizzle.js",
  "jquery/*":  "https://cdn.example.com/jquery/v/2.1.1/plugins/*.min.js",
});

it should take precedence since it has an exact match, but that's an open question. we can formalize the order and precedence if we agree to this approach.

[guybedford]

Here's another question - can the sites table itself map to non-normalized URLs?

site({
  "jquery": "lib/jquery.js"
});

Where the assumption is that the target is just normalized as a URL within the current page path. The implementation I have currently effectively does this (https://github.com/ModuleLoader/es6-module-loader/blob/1.0/src/system.js#L124) so it would be good to check if the behaviour is correct.

[johnjbarton]

Where the assumption is that the target is just normalized as a URL within the current page path.

(Below I use "specifier" to mean the string developers type to indicate a file).

We currently treat module specifiers differently than filename specifiers: relative module specifiers must be explicitly relative. Filename specifiers are ambiguous (or perhaps contextual): foo/bar.js means the same as ./foo/bar.js. Module specifiers assign a different meaning to specifiers beginning with ./, they are relative to the current module path.

If we stick with this difference -- and I think it works well -- then we should be consistent to avoid making the distinction context dependent and thus harder for developers to guess.

So I think the target in the sites table should be analyzed consistently with module specifiers. Thus lib/jquery.js would mean the jquery.js file in the lib package while ./lib/jquery.js would mean the file resolved relative to the URL containing the table.

[guybedford]

Note that with the new spec changes, we treat module specifiers identically to URLs - that is that ./lib/jquery.js normalizes the same as lib/jquery.js the only difference with loading jquery becomes that it can be entry in the sites table to begin with.

[johnjbarton]

So how do I distinguish loading bar.js from the package foo vs the subdirectory foo?

[guybedford]

The sites table acts as a first pass match, falling back to standard URL normalization. So if it matches the sites table is the answer (that is for an entry like foo/*). But ./foo would not be in the sites table so still allows a way to reference the subdirectory.

[caridy]

@guybedford if the url isn't a normalized url after the first pass on the sites table, it should be processed as if you were doing <script src="lib/foo.js" type="module"></script>, they should be reflective. In fact, I foresee people doing:

site({
  "lib/*": "/static/lib/*.js"
});

[guybedford]

Thanks @caridy that sounds great. To clarify the edge case - say there was a mapping:

System.site({
  "lib/*": "./static/lib/*.js"
});

Would I be right in thinking that would be taken to mean the same as static/lib/*.js as relative to the current document.baseURI?

[caridy]

correct

[matthewp]

Can someone explain how site packages differ from the old paths? It sounds almost identical but I might be missing something.

[guybedford]

@matthewp it's the same thing under a different name.

[matthewp]

So it's exactly the same, no difference other than name?

[guybedford]

The implementation of sites is near identical to what was implemented for paths in the previous polyfill work.

[caridy]

@matthewp, renamed, plus some formalization explained in this issue as part of the clean up, that's all.

[guybedford]

What is the status on wildcards being supported in the sites table? I have an implementation, but don't want to be setting the wrong expectations for users either.

(Implementation at https://github.com/ModuleLoader/es6-module-loader/blob/1.0/src/system.js#L104)

[guybedford]

@caridy @dherman I'd still be very interested to hear about the status of wildcards in the sites table. It is one of the blocking issues we have at ModuleLoader/es-module-loader#381 in order to move to the new spec work.

I've been thinking about it, and do actually have a simpler proposal than wildcards which may be suitable. Let me know where might be best to share / discuss this, or if it makes sense to continue to discuss in this thread.

[caridy]

@guybedford I want to hear it, let's chat.

[matthewp]

Why not do it here where others can participate?

[guybedford]

Ok, here's the argument (a) against wildcards in sites, and (b) asking what might replace it.

(a) Why wildcards should not be in sites

The wildcard proposal would be something like:

loader.sites({
  'bootstrap/*': '/path/to/bootstrap/*'
});

This is useful because it allows mapping all subpaths - import 'bootstrap/x' and import 'bootstrap/y' both go to the right thing, which otherwise would have needed to be two separate sites rules.

But paths also allow the ability to add other substitutions as well:

loader.sites({
  'bootstrap/*': '/path/to/bootstrap/*.js'
});

This may seem useful, because it can allow us to write import 'bootstrap/x' and have this automatically add the JS extension for us, but it doesn't quite work out actually.

There are now two issues:

1. We can't import any other file extension from within bootstrap
2. Relative loads within bootstrap don't get js extensions added

(2) is because of the normalization pipeline. Imagine within bootstrap/x.js we had a dependency on ./y:

The normalization then works as:

bootstrap/x -> sites -> /path/to/bootstrap/x.js -> URL normalize -> http://site.com/path/to/bootstrap/x.js.

Now we load ./y relative to http://site.com/path/to/bootstrap/x.js. This uses standard URL normalization giving us http://site.com/path/to/bootstrap/y.

That is, y missed the whole ".js" extension memo.

So because of these two issues (both of which may cause issues for users), I think wildcards may be worth leaving out.

(b) A possible replacement for wildcards

Wildcards were useful though because they saved having to map each and every subpath individually.

Would it make sense to allow a trailing / to indicate a "subpaths" sites mapping over just a single path?

Something like:

loader.sites({
  'bootstrap/': '/path/to/bootstrap/'
});

being a folder-style mapping.

Or would it be worth having this as a new concept entirely?

loader.paths({
  'bootstrap': '/path/to/bootstrap'
});

Now bootstrap/x -> /path/to/bootstrap/y.

Alternatively, do we just leave this out entirely, and leave it to implementations to work out the system best for them?

Feedback welcome.

[guybedford]

The other alternative is always to just accept these behaviors of wildcard paths and implement anyway as well.

[caridy]

In #65 we have deferred the SitePackage implementation, we plan to come back to it as part of Stage 3: https://github.com/whatwg/loader/blob/master/roadmap.md#stage-3-conveniences

Let's keep this issue around in the meantime.