disambiguate "web compatibility" #142
Comments
|
I would call Magically switching
There's an implicit third outcome, equally undesirable, where adding support for |
|
@jkrems can I ask for examples that mandates people to import CJS, or presents a path that prevents them from shipping ESM? What you describe is that there a lack of feature parity between the platforms, I claim that is not a compatibility concern since nothing enforces people to do something that must be incompatible with the web. You are describing a migration problem as well, but you are not stating anything that prohibits people from shipping a full ESM graph to the web. Do you have examples of how allowing import of formats not supported by the web prevents importing web supported formats? |
|
@jkrems maybe to clarify a bit, what about |
Nothing. It just makes it immediately apparent that a file containing |
|
@jkrems would the notification of linkage failing and certain parse failures not serve the same purpose? In addition as I mention above |
|
I’ve thought it was odd that Node allows I think there’s a compelling argument to be made for dropping automatic extension resolution and automatic folder Such an approach would pair with the package name maps proposal to provide ways to do things like |
|
@GeoffreyBooth I've put the resolution algorithm into Platform compatibility concerns. |
|
I'd also note that even with the example of |
|
@jkrems you have to do that research regardless, because any module anywhere in the graph might use |
|
@GeoffreyBooth a) browsers don't mandate the extension, they use URLs. if the URL lacks an extension, so too can the import path. b) with package name maps, you'll be able to omit extensions in browsers (just like is the best practice in node/npm), and it will work the same without a build step. This is a good thing. |
I don’t quite follow. I didn’t mean to imply that browsers care about extensions, I only meant that they require fully resolvable paths/filenames (yes, as part of a URL). My example Since it’s not standard for webservers to resolve an URL ending in |
|
It's something node already does by default, and it's something users expect. Whether it's implemented in terms of package name maps or not, I think that it would be extremely hostile to users to omit. |
|
I'm going to modify the pr I opened for file extensions.
Seems based on this conversation that the appropriate behavior would be to
load the path without any resolution, instead of enforcing file extensions
…On Thu, Jun 28, 2018, 6:30 PM Jordan Harband ***@***.***> wrote:
It's something node already does by default, and it's something users
expect. Whether it's implemented in terms of package name maps or not, I
think that it would be extremely hostile to users to omit.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#142 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAecV0KevcwNdZ3LQHGyEYapGdOGNSuAks5uBVjugaJpZM4U7_X5>
.
|
|
@ljharb I hear what you’re saying, and the user hostility is why there should still be some way for them to do it: via a loader, via package name maps, or something else. But put simply, either we’re trying to be equivalent with browsers by default or we’re not. The current behavior certainly wouldn’t change in CommonJS, it’s only ESM we’re discussing here. I’ve come to accept that Node isn’t going to support everything that current transpilers do, at least not without loaders or other hacks/patches. I think the explanation that “Node has added support for |
|
I don't think we should be trying to be equivalent by default with browsers - browsers don't have filesystem access, nor a massive CJS ecosystem it needs to retain compatibility with (they have a much more massive legacy ecosystem to retain compatibility with). |
I'd note that in all scenarios we are not equivalent. using |
|
Is there any advantage to choosing
|
|
I think the distinction is that, we don’t expect equivalence with browsers whenever CommonJS or interoperability with CommonJS is involved; but in all-ESM mode, I would expect Node to behave the way browsers do. If I can do |
|
i think thats a bit of a red herring. both node and browsers have non-esm formats they would like to interact with (wasm, html, cjs, c++, etc). |
|
@zenparsing imo requiring intentional opt-in actually discourages migration :-/ @GeoffreyBooth you can't use |
Sure, not for everything. I don’t expect Node to support Let’s cut to the chase: @ljharb, I assume you want to make And so the question is whether those goals are more or less important than enforcing browser equivalence when it comes to the |
|
That's the primary reason, yes - and I think it's much more important. Package name maps allow the browser to work as ergonomically as node does here, which is great. I'm also happy to have node support package name maps in some form; but I think it's critical that the default implementation work like node already does. |
|
@zenparsing those have counterpoints so it doesn't really aid:
requires knowing the difference in the algorithms, supported formats, loaders, and caches. i presume this is more burden than learning how
clear linking errors removes that point, same as when you import anything that doesn't have a specific named export. surprise is easily fixed.
The only point that seems to have a bigger impact that its counterpoint is the one about surprise and named exports. However, it has clear errors and easy to learn either by documentation or experimentation using @GeoffreyBooth you make a point about module resolution but nothing about loading CJS. If you resolved to In addition, you can load // this line won't work on the web
// linkage fails so this file doesn't even evaluate
// under the argument for being unable to import things the web cannot it is a compatibility problem
// we should use `import.meta.require('http')`
import http from 'http';
// everything in here doesn't work because the web also doesn't ship these APIs
// we once again are not equivalent
http.createServer((req, res) => {
let files = {
'/': {type: 'text/html', body: `<script type=module src=/file></script>`},
// no extension required, just MIME
// can also do similar using .htaccess / CDNs / cloud storage providers like S3 / ...
// <FilesMatch "^[^.]+$"> // change as needed
// ForceType text/javascript
// </FilesMatch>
'/file': {type: 'text/javascript', body: `console.log(1)`},
};
let file = files[req.url];
res.writeHead(200, {'content-type': file.type});
res.end(file.body);
}).listen(9090);addendum: fun fact! even if you set |
|
@bmeck I didn’t mean in my example that import * as myModule from '/modules/my-module.js';Yes, obviously you can write lots and lots of Node code that doesn’t work in browsers, and vice versa. That doesn’t mean there isn’t value in trying to align with browsers where we can. This is one place we can, admittedly at the cost of preventing other use cases that people really want (without loaders/patches). Maybe we’ll decide that browser equivalence here isn’t worth the cost, and we would rather make an exception to the browser equivalence goal for these other use cases that are deemed more important. |
|
There seem to be a couple of fairly strong arguments against both
The max-min implementation must provide some way to load CJS from ESM, but given that both are controversial we might need a third max-min option. Wasn't there a "createRequire" proposal or something floating around? |
|
Also, a meta-note: I'm not really digging the GH emoji responses in the context of this debate. I find them emotionally distracting; I think they make it harder to come to a shared understanding. |
Strong +1 to this (almost used an emoji for it). People have a right to express themselves, of course, but 👍s to support one side of a debate only make arguments more heated. It feels like a lot of 👍s are given to express solidarity with one side or another, which doesn’t help forge consensus. |
|
i like at least 👍 because it lets us gauge agreement without cluttering comments |
|
I think the frustration of "i can't have named imports from CJS, but oh well, i can still import it and destructure in another line" is way way less of a downside than legacy APIs in the new module system. |
|
I can try and expand to comments instead of emojis to give greater clarity in what I agree with within other comments, but already feel I occupy a large amount of text in multiple threads. @GeoffreyBooth you do have a point with some documentation, but there are plenty of others with named exports from CJS, loading CJS using @zenparsing I'm not sure minmax makes sense here because the very nature of being able to load CJS in any form is the heart of the problem if you claim that Node should be equivalent to web browsers. There is not a solution with no downsides because of this. Minmax is about finding the solution with the minimal set of non-controversial features but the very nature of being able to use any of the CJS on Per import Module from 'module'; // web is missing feature here, error
// could also just assign this to a variable, but it ends up the same as import.meta.require
import.meta.require = Module.createRequireFunction(import.meta.url);It moves the web incompatibility to a module rather than all I agree with @ljharb on what you see as controversial. I don't think it is a strong enough point to claim that surprise is controversial when documentation, reflection, error messages, etc. allow multiple paths of finding the solution quickly. It also leaves the solution to the consumer so they don't need to wait on upstream changes. If we were to take the API based approach using |
One of the reasons for my disagreement is a I don't see a differing
This is not the same information, the UMD wrapper is a well known method to feature detect the environment a Script is being run in and is compatible with multiple environment. The script is doing the side effects after detecting the environment it runs at. This is after it is given a MIME and is being evaluated. We have been arguing about MIMEs here and how sometimes multiple MIMEs can apply to a single source text. You don't need such a complex example to show that some source texts can run in different modes, a simple
For all of Node's existence it has been
How so, you have only shown that there are source texts that work between different formats, we have not proven that formats themselves are irrelevant. The fact that the source text is designed to work in different formats doesn't really mean it mustn't be one, but instead just shows that it can be any of the supported ones, such as
I'm not sure we can have a 100% foolproof solution without defining the disambiguation mechanism. As you have shown and others, there are a plethora of source texts that can run in multiple formats, and some that even result in different execution. The solution here is to stay unambiguous, and defaulting the MIME to a specific format is likely part of that solution.
I agree that there is a compatibility concern due to the ambiguity. I do not agree that assuming it is CJS is unsafe because that is how Node treats all |
|
I do not agree, because it is not how browsers operate - browsers ignore the file extension, and require an out of band mechanism (the “type” attribute on a script tag) to differentiate. node choosing the file extension as that mechanism is 100% compatible with browsers, and also with webservers that already use file extensions to differentiate file types. |
Browsers aren’t really the issue here, webservers are. As discussed above, most (if not all) webservers will choose the same MIME type for both
So we at least agree that there’s a problem here, and the issue is what to do about it, which may be nothing. I’ll try to find time on Monday or Tuesday to condense this discussion into a summary that the group can digest. |
|
The concern isn't the MIME type, it's what script tag "type" they'll choose, since that's the mechanism the browsers use to determine how an entry point is parsed. |
The discussion above is around Until I write up a summary, you can just look at the demo, including reading the code in that repo. That explains the incompatibility pretty succinctly, and it has working code you can run. The webserver in that demo behaves as a typical real-world webserver would. |
|
Yes, of course - but you're assuming no build process. Before anything gets to the browser, the files can easily have been preprocessed to do the right thing. |
|
Let me also respond to some of @bmeck’s points:
I think I’m finally starting to understand where you’re coming from. I suppose that under the hood, Node’s implementation of loading files probably shares a lot of code in common between Having a single file differ upon method of consumption already happens, at least for initial entry points. Take my demo’s
The user is opting into new behavior by using the
Truly staying unambiguous would mean that you would never need to disambiguate. Unambiguous on the author side would be Unambiguous on the consumer side would be having |
They actually share very little code, this is coming from someone who has been authoring both ESM and CJS in my Node applications. Having a file of unknown format is problematic, for tooling, dealing with issues around teaching, defensive programming, etc. My expectation as an author is that my file doesn't get interpreted in different ways than what I intended it to be interpreted as. That applies to execution and linking, both forwards compatibility and backwards compatibility.
There are people from WHATWG that regret it being ambiguous, but given how In particular it shouldn't be relevant elsewhere because it means some problems for things like determining what MIME a given file should be. If it should be one MIME if loaded one way but another MIME if loaded another way, that is problematic for understanding how to deal with the file and matches my concerns above about using both CJS and ESM.
It does not inherit
This is a large point of controversy to me, because it doesn't make any plans for disambiguating the I absolutely think |
WHATWG could have required a new MIME type (and therefore, a new file extension) for ESM JavaScript—but they didn’t. WHATWG settled on using This idea that an individual file needs to control how it’s parsed is really the source of the incompatibility here. That’s not the case on the Web, and if Node insists on it, it will lead to this incompatibility and probably others. |
|
That’s not the case on the web because browsers made a mistake. We don’t have to repeat their mistake, nor would correcting it cause an incompatibility. |
That would not have helped,
I'm not sure I understand this point, the web lets you specify how it should be parsed via |
The idea that an individual JavaScript file needs to control its own parse goal, I mean, is the source of the incompatibility. The Web treats both |
|
@GeoffreyBooth and the argument is that we need to preserve this because? To my knowledge the plan is to completely replace the need for In particular |
|
@GeoffreyBooth do coffeescript users type coffeescript in .js files, or in .coffee files? Is everything required or imported from coffeescript parsed as coffeescript? |
WASM is served by webservers as
Because otherwise we have a major incompatibility with the Web. If I as a package author want to publish a JavaScript Module library for wide use on the Web, I want to publish it as a Basically, Node doesn’t need author-defined file-level unambiguity. Consumer-defined disambiguation can work, though you may not prefer its syntax. If I had to choose between conflicting goals of allowing authors to enforce the parse goal of their file, versus more compatibility with the Web, I would choose the latter. Authors can always informally specify the parse goal of their file, via filenames like I agree with this from the TC39’s discussion of the issue:
|
|
The paragraph you quoted also means that node doesn't need to have its choices constrained by browsers' standards body. |
Indeed this is part of why having ambiguity is problematic. I can create a file: // test.wasm
console.log(123);serve it with
I still don't see the incompatibility issue at heart here. We can still have things treated as There certainly is a difference in default behavior if we choose to continue treating
This seems a fine goal, but isn't necessarily related to the default behavior of the web nor node. This could be an opt-in thing and I'm unsure why it being opt-in is problematic when we are already using non-web compatible features like
I think as long as opt-in to treat
I believe the loss of static guarantees about how files are intended to be run is enough to make it a need even if you disagree.
I don't think any of my comments so far have been about syntax needing to be a specific way. They are rooted in ambiguity problems.
You can have both using an opt in mechanism. I don't understand this comment. |
If there’s a way to have Node treat |
|
If there's a way for node to treat |
|
@ljharb I would assume so, yes. In particular I'm interested in handling things like Flow/JSX/etc. that also live in |
The HTML Standard uses both But how does one decide which HTTP headers to send out? In practice, it happens based on the file extension as opposed to on a file-by-file basis. In general, you’re gonna have an easier time during development but also when configuring your server by using |
|
This topic seems to have cooled and is being addressed on a per phase basis. |
similar to "transparent" interop, the use of the term "web compatibility" is a bit muddy in usage.
I think we need to discern 2 main things:
import.metain browsers__filenamewhich are not going to be present in the browserimport.meta.urlI want to be very clear that platform concerns are about how ESM graphs are loaded and what contextual data is provided to those modules. Underlying formats used to ship modules is unaffected and may be overloaded through different container mechanism such as using webpackage or BinaryAST.
Notably, I would like to make a discerning line about if importing non-ESM is a compatibility concern on the Code level or the Platform level.
I believe very firmly that importing non-ESM is neither.
For the Code compatibility concern, the simplest example of this is to show an application that contains only ESM. The Code compatibility concern is about requiring code be written using syntax or APIs that cannot work on the web. If you don't import CJS, you don't have this compatibility concern, even if the feature of importing CJS exists.
For the Platform compatibility concern, people can claim that
import 'foo';resolving to CJS in Node is a platform compatibility concern. I want to claim this is a false concern.fooresolve to ESM is still possible either by porting, bundling, or shipping multiple builds. At no point isfoorequired to be CJS by writingimport 'foo';.importdoes not require the format of the dependency being loaded to be anything , be it CJS or HTML.I will leave migration concerns up to @demurgos 's incoming review of the issue, but felt that we should address the topic of what "web compatibility" is in a way that we can be concrete in what breaks when we talk about features.
I am open to changing definitions above, but want to make us be more mindful of using the term "web compatibility" without explaining what breaks by including a feature. Is it the ability to use ESM loading mechanisms in both places, or is it the ability to run the same source text in both places? We should also seek to prioritize if the ability to run the same source text in both places is mandatory given that any usage of CJS will not be possible to run without some assistance in the browser.
The text was updated successfully, but these errors were encountered: