How do i mark some modules to external?

[garrydzeng]

Choose one: is this a 🐛 bug report or 🙋 feature request?

🙋 feature request

🤔 Expected Behavior

Don't include external module in bundled file everywhere.
Like rollup globals option.
https://rollupjs.org/#big-list-of-options

🌍 Your Environment

Software	Version(s)
Parcel	1.0.3
Node	9.2.0
npm/Yarn	Yarn 1.2.1
Operating System	Windows 10

[devongovett]

Can you elaborate on the behavior you'd like to see here? I'm not sure how the option as documented by rollup is useful. Why not just use the global variable? Why is it an import at all?

[garrydzeng]

@devongovett

Let me take React as an example.

I can use React as global variable in my project, but many third-party library imported React in their code and I can't control that, so, if I include React through a CDN, like <script src="//cdnjs.cloudflare.com/ajax/libs/react/16.2.0/umd/react.production.min.js"></script> and the bundle in my index.html, then browser has to download React twice.

• one copy in the bundle
• one from CDN

In order to avoid this problem, I have to import React in my project. If Parcel can replace React as global variable when it handling third-party library, then it can reduce bundle size & I can use CDN to speed up my website. React can be any library also.

I made an example: parcel-example.zip.
Hope this help!

[devongovett]

I see. This seems somewhat related to aliasing actually. See #25.

[MalcolmDwyer]

Another vote for this issue... I really like what you've done here with Parcel. It was super-easy to get my project 95% set up. This issue, (external resources), along with dev-server proxying are the two things holding me back.

Motivation for both is that I'm developing components that will be used within a larger application. There are external dependencies (that I don't want packaged up in the component build), and there are external APIs I want to hit while developing. In both cases, these are things that will be there when the project is deployed, but which are not a part of this component.

garrydzeng mention Rollup's "globals" option. Webpack similarly has "externals" (https://webpack.js.org/configuration/externals/).

You mentioned babel aliasing. That, I think, is a separate issue, because it's after the packager has decided to include a file, you can tell babel an alternate path to get it. There doesn't appear to be a way to tell babel that it should be ignored entirely. (At least not in a way that will keep other things happy). The "external" concept would really need to be configured at the packager.

I'd be happy to help out and contribute code to solving these issues, but given the 'zero-configuration' mantra, it's not clear to me where or how either could/should be done in Parcel. If you have ideas along those lines, please let me know. --Thanks

[starkwang]

+1 on this issue. The aliases implemented in #850 can not solve this issue. Just like globals in Rollup and externals in webpack, Parcel also need an option to map modules to a global value.

I'd like to help with this issue if needed : )

[derolf]

I have a patched version that allows external scripts. The idea is to mark scripts as “copy” and provide a mapping of globale to modules that. But that still is kind of configuration...

[davidnagli]

@starkwang Cool! I’d be great if you can take this issue 😃

[DeMoorJasper]

@derolf @davidnagli Perhaps change the aliasing a slight bit and add something like this:

aliases {
  "react": false // This will ignore the package
}

This would cause it to skip without adding extra complex configurations. (Of course we still have to implement it and i'm not sure if it's sorta allowed by the standards)
This behaviour would be kinda similar to how browser.fs === false works for fs resolves

[derolf]

I have a proposal to extend the "alias" syntax to support externals:

"<module>": "<file>!<evaluated export term>"

Example (for cesiumjs):

"cesium": "./node_modules/cesium/Build/Cesium/Cesium.js!Cesium"

Semantics:

• copy ./node_modules/cesium/Build/Cesium/** into the dest folder
• lazy load or include in bundle (depending on how it is used)
• create fake module "cesium" with cesium.exports = Cesium

Usage:
HTML inline load: <script src="cesium"></script>

[DeMoorJasper]

@derolf isn't this different than the described issue? The issue is about using external packages from cdn's wouldn't this just be lazyloading everything from the local server or cdn?

[derolf]

@DeMoorJasper

For CDN:

"cesium": "http://cdn.xyz.com/foo/bar/Cesium.js!Cesium"

could work the same way. But instead of copying into dest folder it's is directly linked.

I think the main idea is to somehow refer to "prebuilt" folders -- disable parcel parsing -- and link them to a fake module.

So the exclamation mark disables parcel parsing of the content and the term afterwards creates the export directive for the fake module.

[woubuc]

@derolf I think the ! syntax is fairly non-obvious, at least I haven't seen it used before. Is this 'standard' across similar implementations? Why not just use the key?

[derolf]

@woubuc So, if you look at webpack's external, you need a key plus which global symbol to bind to that key. So, we need to tell parcel which symbol to bind to the fake module.

Example: three-js exports a global symbol called THREE, so the syntax would be:

"three": "https://cdnjs.cloudflare.com/ajax/libs/three.js/90/three.min.js!THREE"

This follows the same approach in webpack.

[MalcolmDwyer]

@DeMoorJasper 's <PackageName>: false makes sense to me (but misses renaming... which may be important in certain cases).
I don't see the advantage of @derolf's <PackageName>: <CDN path>:!<import name>, unless it's solving a different problem than what I was talking about.

Could the webpack externals concept be followed directly, except inside package.json. Then it's a 1-to-1 with a well-known feature, and keeps the ability to rename the import.

package.json:

  "dependencies": { },
  "parcel": { /* Or should externals be at the top level? */
    "externals": {
      "MyProjectConfig": "config",
      "react": "react"
    }
  }
}

Unlike the aliases: {packageName: false}, this allows for renaming. You can import from the proper name of the package (as if you were really packaging it up), but still have the code get linked into whatever the actual named variable is in the CDN/inline script.

My deployed index.html/jsp that pulls in the parceled project...

   <script>
      config = {
        someValue: 123
      }
   </script>
   <script src="://some-cdn/react.min.js"></script>
   <script src="./build/parceled-up-project.js></script>

And finally... some random source file in my project:

import MyProjectConfig from 'MyProjectConfig'
import React, { Component } from 'react'

For both those imports, the packager can find the 'from' in the externals list, and know that it doesn't need to actually fetch/import anything. And the actual variable names can get assigned with the desired renaming.

Something like this makes the most sense to me... but it's obviously getting away from 'zero-config', so I assume it would need some consensus before building it in. What do you think?

[derolf]

Package.json is not available to the parser. Because resolution is done afterwards.

So, you need two things:

• disable parcel’s parsing if you pull in a prebundled script
• link a module to an exported variable

[derolf]

@MalcomDwyer okay, I can implement your “externals” in package.json proposal and create module stubs the same way that Webpack does.

But, what is a good syntax to tell parcel to NOT bundle a script that is already in the public folder?

Like: <script parcel-ignore src=‘abc.js>

[MalcolmDwyer]

But, what is a good syntax to tell parcel to NOT bundle a script

In my example, that "deployed index.html/jsp" would not even be something that parcel ever sees...

I have my frontend project codebase that I use to develop, and test the javascript project. Then in a separate downstream project, I just pick up the bundle.js (src="./build/parceled-up-project.js in my example above) as a pre-built entity. It is in that downstream file that those external imports actually get resolved. (So they are external to parcel, but are part of the downstream project).

So I would probably have parcel building the real bundle.js (which ignores those externals), but also for development, I'd have parcel build a demo/index.html which would have the same imports and should include those files. I can't think of a time I'd be pointing parcel at an html with <script src="x"> and not want it to import "x".

[derolf]

I start bundling from a central HTML file and need to pull in stuff to parcel and stuff that is prebuilt (three js and cesium js).

[DeMoorJasper]

Package.json is not available to the parser. Because resolution is done afterwards.

You shouldn't wait this long right?
You just append the externals to the resolver, this way u can detect it before it even gets to the parser

<script parcel-ignore src=‘abc.js>

No big fan of this, it would add a new sort of config to the ecosystem and limit the abilities it has, this seems to be more a use-case for the false key I described in my first comment on this issue, as this would extend beyond just html (adding the possibility to ignore in JS, for example optional dependencies parcel tries to resolve)

[derolf]

As far as I saw, parsing is done before resolving.

So, how can I tell the parser of the index.html not to bundle a script, but just leave it untouched.

[DeMoorJasper]

In the current codebase u should look at asset.addUrlDep... and the Resolver as those are the only places we use to actually resolve assets.

I'll try to make it a bit more clear, the parser.getAsset() never gets called if the dependency never gets added to the asset in the first place. The package is known to the parent asset, so you have full access to package.json

[derolf]

@DeMoorJasper I looked a lot at the code this weekend and found the asset.addUrlDep. The problem is that the parsing of the asset, processing, and generating the output is done in a Worker. This Worker has NO access to the resolver, but just creates put dependency-links to the bundles that will contain these dependencies into the processed files.

Then later, the bundler picks up the queued dependencies and resolves (including aliases) them and starts another Worker process.

Check this Bundler: loadAsset(asset)

async loadAsset(asset) {
... FIRST PROCESS
    let processed = this.cache && (await this.cache.read(asset.name));
    if (!processed || asset.shouldInvalidate(processed.cacheData)) {
      processed = await this.farm.run(asset.name, asset.package, this.options);
...
    }
... THEN RESOLVE
    // Resolve and load asset dependencies
    let assetDeps = await Promise.all(
      dependencies.map(async dep => {
...
          let assetDep = await this.resolveDep(asset, dep);
...
      })
    );

...
}

To fix this, we would need to expose the resolver to the Worker and let resolution happen there???

[derolf]

Ah, you mean Assert.pkg? You are right, it's having full access to the package.json!

[DeMoorJasper]

I'm pretty sure if the dependency never gets added, the bundler never picks them up anyway? The resolver changes would just be in case the bundler adds some dependencies.
Now that I looked at it resolver changes aren't even neccessary unless u wanna move the responsibility to the bundler, I haven't tested any of the things i say i'm suggesting as a fix.

And the code sample you show makes sense but doesn't have much to do with the ignore or this issue. It will never get to loadAsset if the asset is ignored, as that happens on the loadAsset of the previous/parent asset.

And if it's really necessary u could spin up a resolver per worker similar to how the parser is being used inside workers, although it would be nice to prevent this.

EDIT: Yes i do mean Asset.pkg (of the parent of the ignored/external package)

[derolf]

So, which syntax do we follow now? An external needs TWO things: where is the file located and what does it export.

Example with cesium js:

"externals": {
 "cesium": {
   "exports": "Cesium", // --> this generates a module stub with cesium.exports = Cesium;
   "file": "node_modules/cesium/Build/Cesium/Cesium.js", // ---> that is the file to be src for the script
   "copy": "node_modules/cesium/Build/Cesium", // ---> that tells parcel to copy (or symlink) all that content into the public dir
 }
}

"copy" here is important since Cesium.js includes other files from that folder. So, the WHOLE folder needs to be present!

Makes sense?

[youknowriad]

I wonder if it's the best place to discuss parcel vs webpack for folks following this issue :P

[FlorianRappl]

Actually I don't think it's that difficult to do. We do that in Piral where we use Parcel as our build system integrated in our CLI. For us we need to strip out shared dependencies from frontend modules (called Pilets).

What we do could be (i.e., is definitely) considered a hack (and obviously I wish that there may be a simpler alternative, which may exist with Parcel 2), but it could easily be applied / transported in a plugin.

If you guys are interested I guess I could try to craft a simple plugin on the weekend. Only thing to discuss upfront is what packages to mark as external.

Options I see: peerDependencies (this is what we do in Piral essentially - though its a bit more complicated than that), a new key in package JSON (e.g., externalsPackages) or in a dedicated config / rc file.

Any thoughts?

[devongovett]

Seems like maybe there's two separate but related features here:

1. Excluding a module from a bundle, but leaving the import or require there. Use case would be for e.g. libraries with peer dependencies, builtin modules in node/electron, etc. Maybe this could be solved by excluding peerDependencies.
2. Mapping an import to a global variable that is available somehow. Use case would be for e.g. libraries from a cdn. This might need separate configuration, e.g. with aliases.

See also #3305

[FlorianRappl]

So I hacked this together: https://www.npmjs.com/package/parcel-plugin-externals

Don't know if its useful. If there is some use, please give feedback. Thanks! 🍻

[subhero24]

I created a pull request to support this without a plugin

[thinkloop]

You are officially a fullhero now 👍

[akrizs]

Guys! You are making parcel the easiest tool in the toolbelt to use! That is amazing! Thanks alot!

[mikestopcontinues]

What's the status of this issue? As per @devongovett 's first use-case:

Excluding a module from a bundle, but leaving the import or require there. Use case would be for e.g. libraries with peer dependencies, builtin modules in node/electron, etc. Maybe this could be solved by excluding peerDependencies.

I'm deploying to Lambda, and I don't need to include the tremendous aws-sdk in my package. I just need a way to let require('aws-sdk') access the global available on Amazon's servers.

[cliffordp]

@FlorianRappl #144 (comment)
Saved my bacon to be able to use Parcel for a WordPress plugin that uses jQuery (which comes from WP itself)
THANK YOU!

[cliffordp]

@FlorianRappl

Any idea how to glob-exclude? I don't want these:

[FlorianRappl]

Hi @cliffordp - please create issues at https://github.com/FlorianRappl/parcel-plugin-externals such that they have increased visibility for other users of the plugin.

Have you tried https://github.com/FlorianRappl/parcel-plugin-externals#dynamic-dependency-resolution with a rule factory?

I can imagine that (taking only the @wordpress/* packages into consideration)

const rx = /node_modules\/@wordpress\/(.*?)\//;

module.exports = function(path) {
  const result = rx.exec(path);

  if (result) {
    const package = result[1];
    return `@wordpress/${package} => require('@wordpress/${package}')`;
  }

  return undefined;
};

could help here.

[DeMoorJasper]

#4072 implemented this, for additional ways to exclude modules feel free to open an RFC to open discussion on this

[cliffordp]

@DeMoorJasper that didn't work for me (unless the "library" bit needs to be customized for my use case, in which I'm unsure how-to):

[FlorianRappl]

@cliffordp I think this is a Parcel v2 addition - for Parcel v1 I guess you'll need to stay with the parcel-plugin-externals.

[AndrewRayCode]

Sorry to bump an issue three years later, but this is the first result in Google for

"parcel mark dependency as external global variable"

From looking at this issue, it's hard to tell if this was ever resolved? The cryptic comment above from @FlorianRappl

I think this is a Parcel v2 addition

Was it?

I'm trying not to bundle external dependencies (specifically BabylonJS, but whatever) into my build, and treat resolution of that import as a global variable BABYLON, to support this ecosystem which mostly uses global variables to communicate between scripts.

I've defined targets:

  "targets": {
    "babylon": {
      "source": "src/plugins/babylon/index.ts",
      "includeNodeModules": {
        "babylonjs": false
      }
    }
  }

This errors with:

@parcel/packager-js: External modules are not supported when building for browser

13 | Color4,
14 | } from 'babylonjs';
| ^^^^^^^^^^^

This error doesn't make sense.

If I remove the includeNodeModules line, the bundle works, except Parcel is very obviously doing the wrong thing:

✨ Built in 37.08s

dist/babylon/index.js       ⚠️  4.48 MB    20.88s

Was this ever resolved, or do I need to downgrade to v1 and install https://www.npmjs.com/package/parcel-plugin-externals ? That plugin readme has nothing about if it should still be used or not.

Searching the documentation for "externals" returns nothing.

So either Parcel is not read for prime time, or much, much more likely, I'm missing something and it's just not well documented? Can someone point me in the right direction to help future Googlers?

[danmarshall]

@AndrewRayCode I think this is achieved with aliases in v2.

[FlorianRappl]

@AndrewRayCode aliases achieves this in v2 - for v1 you can use the plugin.