Use cases for module attributes besides module type

[littledan]

The initial use case for module attributes is types, but module attributes had been previously discussed over several years. Some other possible use cases:

• Fetch parameters -- e.g., various policies, authentication options, etc. Unclear how this would interact with caching modules, or how this would be represented
• SRI or other integrity markers -- this interacts poorly with caching, and it may be best to have this out of line in some other yet-to-be-designed resource file
• Marking CJS modules in Node -- this may be a good use of the type field
• Redirection to find the module within a WebPackage -- I don't know the current state of all this; maybe @nyaxt could clarify if this makes any sense
• Built-in module polyfilling -- now that import maps are not proposed to serve this purpose (initially, a complex module specifier was proposed, so this would be analogous)

For each of these, there would be a significantly greater amount of investigative work to make a real proposal. This repository does not aim to do that work, but I'd like to just understand a bit more about the broader space. Some questions to discuss in this issue:

• Does anyone have any other use cases besides type, or comments on the above?
• Do strings make sense as values for the additional possible module attributes?
• Do we really think it's likely that we'll eventually want any of them, or will type alone probably suffice?

[xtuc]

I think another use case would be for tooling; transpilers could tune their code generation based on attributes for instance:

import {a} from "b" with parserPlugin: 'jsx';

or a bundler using loaders:

import {a} from "b" with loader: 'babel';

This proposal allows arbitrary keys so it could be anything.

[littledan]

@xtuc On one hand, this could cause more divergence in the ecosystem. On the other hand, I imagine it may be nicer than maintaining separate configuration files.

[Janpot]

I wonder if it could be useful in testing scenarios? similar to libraries like proxyquire.

import { a } from './my-module.js' with stub: { 'path': './my-path-stub.js' };

It may also be useful for some form of dependency injection?

[weswigham]

Just for reference, this was already discussed in tc39 a bit for the integrity case with these slides.

[littledan]

Thanks for the cross-references, @weswigham. To summarize, @domenic explained why module attributes would be a suboptimal solution for integrity, and the committee agreed. I am not proposing integrity as a primary use case, but given all past discussion, it seemed worth mentioning.

[Jack-Works]

I had done a POC months before to ES import a TypeScript file in the browser directly. I used syntax like

import mod from "/typescript-transpiler.js?path=/src/index.ts"

And I use import.meta.url to get the "self" URL where a path parameter in it. With module attributes, I can write

import mod from '/typescript-transpiler.js' with { path: "/src/index" }; // or whatever the syntax is.
// And I can read import.meta.attributes or whatever in typescript-transpile.js to do the compiler job

This requires re-instantiate the whole module if the module attributes are different but the path is the same.
The previous trick in the URL parameter is working because the browser treats the same URL with different URL parameters as different modules.

I think re-instantiate in module attribute might be a serious problem. Does the current semantic of module attribute support this?

[michaelficarra]

Here's some ideas I have for use cases of per-import-site module attributes:

1. parameters to built-in parsers, such as a reviver for the JSON parser, a content encoding, a start production, etc
2. parameters to built-in validators, such as an expected mime-type, an expected maximum size or object count, a parsing timeout, an expected schema, etc
3. provide a custom parser
4. delayed execution of top-level statements (as in integration of two modules with non-cooperating authors)
5. capabilities
6. specifying security/feature policies to use
7. specifying the realm to execute the module in
8. forced re-execution of a possibly-cached module

The custom parser use case could be solved by having a "text/plain" or "application/octet-stream" type supported where it just gives you what came in off the wire instead of parsing/executing the resource. The capabilities and realms use cases could be solved by instead importing within a Realm as in the Realms proposal.

Notice that none of these would be more appropriately provided via out-of-band metadata because they each may vary at different import sites for the same resource.

[jridgewell]

specifying security/feature policies to use

I brought this up at the end of the December Realms/Compartments meetup in SF. Having the ability to restrict, eg, network access at the import site would be extremely powerful:

const basicThing from 'lib' with { network: false };

Or, just strip all privileged API access:

const basicThing from 'lib' with { privilege: 'none' };

[mstijak]

I'm not sure if it's relevant, but webpack uses magic comments to name chunks or select different modes.

https://webpack.js.org/api/module-methods/#magic-comments

[rubennorte]

Module attributes could also be used to implement asset references (like proposed in https://github.com/tc39/proposal-asset-references).

If a reference to a module is needed (instead of the module itself), it could be imported as:

import moduleRef from 'module' with {reference: 'asset'};

import(moduleRef);
MyCustomLoader.load(moduleRef);

It could be used together with the type attribute:

import jsonRef from 'json-file' with {type: 'json', reference: 'asset'};

import(jsonRef);

We use a custom API for this at Facebook and having a standard syntax for it would be interesting (either with the asset references proposal or with this one).

[ndesmic]

I'm thinking something like this: https://github.com/surma/rollup-plugin-comlink would be nice to support. It basically allows importing code as a worker. Currently it uses special url syntax "comlink:.path" to make it apparent.

[xtuc]

@Somnid it's a use case for https://github.com/tc39/proposal-import-assertions#follow-up-proposal-evaluator-attributes. Assertions are not transforming the module, but asserting it's type.

[7ombie]

A lot of people would like support for specifying alternative parsers, specifically for working with languages that compile to JavaScript. There are existing projects that hack Chromium to permit importing TypeScript directly, so it'd be nice to have a proper API for that kind of thing. However, being able to import the plain code wouldn't really solve the issue. The TypeScript parser would need to convert the imported source to JavaScript source, then call eval on it, which has all kinds of issues.

Ideally, we'd be able to specify that an import is a module (and needs to be executed), but that it is not written in JavaScript, and to specify which parser to use. That parser would then return the JavaScript that would be imported as normal.

Given that this would need to work with static imports, we would probably need to specify a parser with a URL it can be imported from. I'm not precisely sure how that would work. You could maybe require specifying the URL, and allow specifying the name of the export (as an optional second assertion), which would default to default (the default export).

Longer term, it'd be awesome if the parser had the option of returning some form of standardized JavaScript AST, eliminating the need to generate JavaScript source code. This would be a lot cleaner and more efficient, and (IIUC) it would also eliminate the need for source maps (the AST would take its line and column numbers from the source language), so debugging source languages in DevTools (with breakpoints etc) would just work.

Obviously, supporting a standard AST format requires an AST format, which is beyond the scope of import assertions, but supporting source-to-source parsers seems to be something that could be explored here, and it would open up the possibility of source-to-AST-parser support later.

[Jack-Works]

A runtime source-to-source parser+executor heard like "I need eval" to me

[7ombie]

Hey @Jack-Works. Thanks for considering it.

A runtime source-to-source parser+executor heard like "I need eval" to me

Electron already has eval. That's not the issue. Importing stuff like TypeScript is not especially useful, if you're only able to load the source files, specifically because they would then require eval (once compiled) to execute, which is not the same as importing modules.

Maybe there could be some other hook, related to the dynamic import function. I dunno. I'm a vanilla JS guy. I was just considering the usefulness of the usecase.

[7ombie]

@Jack-Works - On reflection, I was wrong about extending the import syntax to support registering alternative parsers etc. It doesn't really belong here.

It would be much more flexible to generalize from language parsers to response adaptors that can intercept any resource, including imports, as well as XHR requests, stylesheets, image/audio/video tags etc.

Rather than do this in import statements, adaptors could be registered in a global registry (like custom HTML elements, or something similar), mapping some non-standard memetype to a handler that can either render the resource fully, or can convert it to a standard resource that the browser would take from there.

That is obviously just a rough outline of how it might work. In any case, while import assertions could be affected by a proposal like that (maybe allowing something like import foo from "./main.funky" assert {usertype: "funky"};, I'm pretty certain now that registering any kind of adaptor/transpiler inside import statements was a bad idea.

Sorry for the noise, and thanks for considering it anyway.

[lino-levan]

I think it'd be pretty cool to allow importing modules as workers with this syntax.

import workerRef from "./worker.js" with { type: "module", worker: true};
// or
const workerRef = await import("./worker.js", { with:  { type: "module", worker: true} });

workerRef.onchange = ()=>{
    console.log("hello world");
}

Definitely deserves some bike shedding on the syntax.

[nicolo-ribaudo]

Vite has something similar to what @lino-levan proposes: https://vitejs.dev/guide/features.html#import-with-query-suffixes

[jimmywarting]

• Thinking it would be useful for the service worker to know about the attributes so a service worker could polyfill stuff.
• And also for servers, could it be sent with some request headers?
• also how about using credentials?

import credentials from './secrets.json' with {type: 'json', credentials: 'include', headers: xyz }

[justinfagnani]

Sending attributes to the server could enable some interesting use cases, like auto-proxying server code to the client.

// The server generates a client module that implements functions
// as RPC calls to the server. server.js is actually a Node module.
import {queryFoo, addFoo} from './server.js' with {serverProxy: true};

// Some imagined query API
const foos = await queryFoo(['name', '==', 'bar']);

[styfle]

I'm not sure if type could be used for this use case, but I think it would be great to replace fs.readFile('./content.txt', 'utf8') with something like:

import txt from './file.txt' with { type: 'text' };

const isScript = txt.startsWith('#!/bin/sh');

Similarly, it would be great to replace fs.readFile('./file.mp3') with something like:

import blob from './file.mp3' with { type: 'blob' };

const ab = await blob.arrayBuffer();

This would effectively standardize file reads and even allow bundlers to inline the content when using { type: 'text' }.

If we can't use type then maybe something like asset or file?

[michaelficarra]

@styfle Since there's no instantiation, linking, or evaluation for text/blob assets, I imagine you would want to use the source keyword from https://github.com/tc39/proposal-source-phase-imports with these imports.

[styfle]

@michaelficarra Thanks! I'm not sure if that proposal would import as a string or blob. It says:

This proposal allows ES modules to import a reified representation of the compiled source of a module when the host provides such a representation:

I then references another proposal but it looks stale: https://github.com/tc39/proposal-asset-references

[michaelficarra]

An asset reference here would just be a URL object. See this slightly outdated presentation. That's not what you're looking for if I understand you correctly.

I'm not sure if that proposal would import as a string or blob.

I was thinking you would use the source keyword in conjunction with { type: 'text' } or { type: 'blob' }. Since instantiation, linking, and evaluation would all be a no-op for such assets, it better conveys to the reader that those steps are skipped.

[shaedrich]

And I use import.meta.url to get the "self" URL where a path parameter in it. With module attributes, I can write

import mod from '/typescript-transpiler.js' with { path: "/src/index" }; // or whatever the syntax is.
// And I can read import.meta.attributes or whatever in typescript-transpile.js to do the compiler job

Wouldn't it make more sense to have the actual file in the from and the transpiler as an attribute of sorts?

I think another use case would be for tooling

Not sure if JS should have "knowledge" of tooling/be dependent on it.

It would be much more flexible to generalize from language parsers to response adaptors that can intercept any resource, including imports, as well as XHR requests, stylesheets, image/audio/video tags etc.

Rather than do this in import statements, adaptors could be registered in a global registry

The import maps WICG proposal talks about why they decided against a programmable resolution hook. I'm not sure as to if and to what extent this also applies to this proposal

Similarly, it would be great to replace fs.readFile('./file.mp3') with something like:

import blob from './file.mp3' with { type: 'blob' };

const ab = await blob.arrayBuffer();

In this context, the parser augmentation proposal might be interesting.

I'm not sure if type could be used for this use case, but I think it would be great to replace fs.readFile('./content.txt', 'utf8') with something like:

import txt from './file.txt' with { type: 'text' };

const isScript = txt.startsWith('#!/bin/sh');

This could be taken even further by including a charset attribute:

- import txt from './file.txt' with { type: 'text' };
+ import txt from './file.txt' with { type: 'text', charset: 'utf8' };

const isScript = txt.startsWith('#!/bin/sh');