Bundling with Webpack doesn't work well

[jquense]

congrats on 1.0! been waiting a long time to use the shiney new globalize :)

I have been running into issues using Globalize with webpack ( browserify may suffer similarly).

ERROR in ./~/globalize/dist/globalize/number.js
Module not found: Error: Cannot resolve module 'cldr' in \node_modules\globalize\dist\globalize

Investigation shows that this is due to the AMD check running first in the UMD wrapper for the files, since the package has been installed from npm there is no cldr package for webpack to find. there is a cldrjs however and if I tell webpack to ignore amd style modules for globalize it works just fine.

The problem is that as a library author using Globalize and publishing to npm I can't ask consumers of my module to add a special exemption to there build process in order to get my module to load. I am not sure what the best way to "fix" this is other then building a commonjs (or amd) only version of globalize that gets published to npm, instead of the general /dist build that works for both bower and npm

[rxaviers]

Hello, thanks for your interest and for submitting this issue.

I haven't looked deep into it yet. But, as far as I understood, webpack parses the file, finds every occurrence of require (for cjs) or define (for AMD), figures out the module names and tries to crawl them all. Is that correct?

My first question, why doesn't it consider the cjs information rather than the AMD one from the UMD?

[rxaviers]

Some other curiosity-questions... Does wekpack try to include the modules below?

a)

// A:
if ( false ) {
  require( "this-module-should-never-be-loaded" );
}

// B:
// require( "commented-out-module" );

[jquense]

Thanks for the quick response!

I haven't looked deep into it yet. But, as far as I understood, webpack parses the file, finds every occurrence of require (for cjs) or define (for AMD), figures out the module names and tries to crawl them all. Is that correct?

exactly, and then concats it all together, its similar to the requirejs pre-processor step, except that it supports all sorts of module/file types and is plug-able for fancier things.

why doesn't it consider the cjs information rather than the AMD one from the UMD?

It supports amd, so it doesn't mind loading it, however, the npm package doesn't install a dependency called cldr only one called cldrjs, so you get an error because it can't find it. If the question is "why doesn't it then try the commonjs branch of the UMD wrapper" I think its just not that smart.

Does webpack try to include the modules below?

both of those approaches work in webpack. It is smart enough to not try in the case of if (false) and definitely ignores commented code. However this will fail:

var no = false
if ( no ) {
  require( "this-module-should-never-be-loaded" ); //will try and load it
}

[rxaviers]

cc @jzaefferer @scottgonzalez for input.

[scottgonzalez]

Have you talked to the webpack team about this? If Globalize is working properly on its own for CJS and AMD environments, it sounds like webpack has a bug.

[jquense]

hey @scottgonzalez thanks for helping out! Its definately not a webpack bug. The underlying issue is that Globalize defines a dependancy that doesn't exist in the npm package. The webpack "environment" is unique because its one of the few places that both module types are valid , so the UMD wrapper never fails on any of its checks.

The reason it works in node is that the check for define fails so it drops down to the check for module and module.exports. In webpack (and also browserify if amdify is being used) the first check for define succeeds, It then tries to load the amd module, at which point it fails because it is missing a dependency cldr.

The problem is that the cldr dependency is called two different things: cldr and cldrjs in the npm package there is no such thing as that first one.

The brittle fix is just to move the check for cjs before the amd check.

[rxaviers]

@jquense, you are saying that webpack is capable of interpreting both CJS and AMD modules simultaneously and it works. On AMD, cldr is bind to whatever you define on AMD config.paths, it doesn't mean one should find a cldr module on npm. Which config.paths are you using?

[jquense]

I should probably demonstrate with code. If you npm install globalize you get two dependencies installed (in the globalize folder) with the following structure installed:

• dist
• node_modules
  • cldr-data
  • cldrjs

and here is the UMD wrapper globalize defines (this is around: dist/globalize)

       if ( typeof define === "function" && define.amd ) {
        define(["cldr","cldr/event"], factory );
    } else if ( typeof exports === "object" ) {
        module.exports = factory( require( "cldrjs" ) );
    } else {
          // global one omitted
    }

By default Webpack understands both module formats so as it parses this file it hits: define(["cldr","cldr/event"], factory ); first (it won't run the other branches if it succeeds) and so tries to find both cldr and cldr/event. Presumably if I had installed Globalize via bower that dependency would be sitting in my bower_components (I don't ever use AMD or bower so I am not sure on that). However, since I used npm the only place to check is node_modules and it finds no module by that name in node_modules and fails.

if it had run: module.exports = factory( require( "cldrjs" ) ); first it would have found cldrjs in node_modules and bundled it successfully

[rxaviers]

And I'm saying that on AMD there's no hard definition for module names. Let me demonstrate with code for you as well https://github.com/jquery/globalize/blob/master/test/config.js#L4 (on AMD you must tell where the modules come from or it's expected there's a file on baseDir with such name).

Anyway, can you make a test please? If you rename cldr occurences to cldrjs in your localnode_modules/globalize/dist/* files, does it work? I think you'll get a failure on "cldr/event".

[jquense]

And I'm saying that on AMD there's no hard definition for module names

Gotcha, sorry I don't use AMD modules, forgive my ignorance :). Yeah if I change the name it fails on cldr/event.

[rxaviers]

We need to understand how to pass an AMD config to webpack or have webpack to simply ignore the AMD definitions.

[jquense]

So that is easy enough to do as the person webpacking the files. Unfortunately I don't think there is anything module authors can do to define configurations for potential consumers.

How does it work with requirejs? If I release a library that depends on globalize can I set of the paths so someone installing my module doesn't have to (i.e. doesn't need to worry about wiring up my dependencies) or would they have to set it all up themselves as well?

Some context. I'm coming from the React.js community which mostly uses cjs, npm, and webpack to publish and distribute components (both for server/browser). The advantage being consumers of my components will just get all my dependencies installed and webpack/browserify will package them up without a consumer needing to have any knowledge of my dependencies and I don't have to create a bundle to publish I can just keep my package all in modules.

[jquense]

on second thought tho if you are going to configure the amd stuff anyway then my original solution should work? Reordering the UMD to:

if ( typeof exports === "object" ) {
  module.exports = factory( require( "cldrjs" ) );
} else if ( typeof define === "function" && define.amd ) {
  define(["cldr","cldr/event"], factory );
} else {
  // global one omitted
}

[rxaviers]

How does it work with requirejs? If I release a library that depends on globalize can I set of the paths so someone installing my module doesn't have to (i.e. doesn't need to worry about wiring up my dependencies) or would they have to set it all up themselves as well?

Using AMD, they would need to set it all up themselves AFAIK.

Reordering the UMD to:

Does reordering work for you locally please?

[jquense]

Does reordering work for you locally please?

Can confirm that it Works on My Machine™ if I reorder the UMD wrapper

[remyrylan]

@jquense Could you elaborate on how you got Globalize working with Webpack please? I'm having the exact same issue.

[rxaviers]

@JeremyRylan, for now as a workaround, try swapping the AMD and CJS exports of the UMD wrapper (e.g., https://github.com/jquery/globalize/blob/1.0.0/dist/globalize.js#L19-L30), i.e., by placing the CJS in front of the AMD.

[remyrylan]

@rxaviers Perfect! That works for me. Thank you very much for the quick response and for the amazing work with Globalize.

Do you think this is something that will be addressed in a future release or will we rely on having to modify the source?

[rxaviers]

You're welcome. This is definitely something we want to get sorted out. Although, it's important that such solution doesn't break anything else.

It would help if you (or @jquense) could share with us a step-by-step procedure (for example, in a gist) to reproduce your use-case.

[jquense]

@JeremyRylan A less brittle solution you can try is to add this loader to your webpack config

{ test: /globalize/, loader: 'imports?define=>false' }

Which will tell Webpack to ignore AMD imports for modules that match globalize packages (untested).

As I say in above posts, this is an ok solution for app developers, but not feasible for building libraries that depend on globalize.

For the most simple repo of the problem: https://gist.github.com/jquense/10e3d57e1ce8ae0467da

If you are looking for a real life use case you can check out https://github.com/jquense/react-widgets which is a component library I maintain that depends on globalize for date and number pickers

Right now to get this to work with 1.0.0 I either need to

1. bundle all my code up into a monolithic "dist" build
2. Tell my users how to handle building my libraries dependencies

The first is not feasible, as it destroys a lot of the benefits of using modules, and means that users cannot consume my library ala carte. Right now you can do: require('react-widgets/lib/DateTimePicker') if you only want to use the date picker component in an app. Webpack/Browserify will include only the code needed by the date picker, instead of sending all components to a client you only pay for the bytes you use.

Number two is not ideal, but for all the reasons I've said earlier.

[unindented]

@rxaviers Any chance we could see this fixed in the next release of globalize? Proper webpack support would be huge for me.

[unindented]

@rxaviers Here's your node-npm example modified to use webpack: https://github.com/unindented/globalize-hello-world-webpack

The README explains how to reproduce the problem.

[rxaviers]

Thanks for your example. I may reply in a week to 2 weeks timeframe. I do hope we can get this sorted out for the next release yes.

[jamischarles]

@jquense The custom loader thing worked great for me. Thanks!

[rxaviers]

Hi everyone, I've finally spent some time digging into this issue, understood it beeter and I'd like your input.

The problem is that webpack finds the AMD definition first and tries to use this instead of the CJS definition. This is not a problem if resolve.alias are configured accordingly to map the proper AMD paths, but this is cumbersome specially for developers using npm and CJS (which are the majority using webpack).

I understood the two solutions/workarounds that have been raised are:

1. To swap the UMD exports order by specifying the CJS first, then AMD.
2. To specify a custom loader to skip the AMD definition { test: /globalize/, loader: 'imports?define=>false' } (Ref: Bundling with Webpack doesn't work well #441 (comment))

For the first one, I am not comfortable making changes motivated by its side-effect solving another completely different thing.

For the second one, like @jquense said, it's not feasible for building libraries that depend on Globalize. I assume, because it's cumbersome asking developers to include such non-trivial loader.

Going further, I gave a try at @unindented's example and I realized a second issue: the inability to handle feeding Globalize on CLDR data. This is, when CLDR data is static (e.g., https://github.com/unindented/globalize-hello-world-webpack/blob/master/main.js#L6-L9) webpack works just fine. Although, when CLDR data is dynamic (and it is in most of the usecases, e.g., require("cldr-data/main/" + locale + "...") or require("cldr-data").entireMainFor(locale)), webpack enters in its resource context mode and tries to load the entire package to dynamically decide what needs to be loaded, which means tons of JSON data (hundreds of MBs) in the client.

These problems suggests a solution should:

1. Allow developers to use Globalize in webpack without worring about AMD paths.
2. Allow developers to simply write code without worring about loading CLDR data (and let webpack to handle it during build), e.g.:

var Globalize = require("globalize");

Globalize.formatNumber(Math.PI);

Unfortunately, I could not think of any solution that doesn't involve adding configurations to webpack. To minimize the inconvenience though, I thought we could concentrate all in one single GlobalizePlugin. On top of that, a third point, not yet mentioned here, is to allow webpack to optionally generate precompiled formatters and parsers for production (faster and smaller code) instead of including CLDR data and having formatters and parsers to be created dynamically in the client.

I've created it, which certainly needs more iterations given your feedback.

// Webpack config
module.exports = {
    plugins: [
        new GlobalizePlugin({
            production: boolean, // whether or not to generate precompiled runtime code.
            defaultLocale: 'en', // the locale that will run during development on live reload.
            locales: ['en', 'es', 'zh', 'ar', 'pt', 'de', ...] // the locales to generate bundles for.
        })
    ]
};

The plugin currently lives in https://github.com/rxaviers/globalize-webpack-plugin/tree/b0.0.1 (very bad documented so far). The best I can show for now is a React.js example that uses it: https://github.com/rxaviers/yarsk/tree/react-globalize. It's all work in progress, but I wanted to share fast to receive your early feedback.

I invite everyone to join the brainstorm and to contribute to a proper solution.

[sompylasar]

@rxaviers Looks great so far!

What do you think of building an app that depends on a library that in turn depends on Globalize?
What do you think of making a globalize-webpack-loader instead?

[jquense]

I will dig into this a bit more when I get some free time. Thanks a ton for really exploring and looking into this. For now here are some quick thoughts.

For the first one, I am not comfortable making changes motivated by its side-effect solving another completely different thing.

Changing the order fixes most of the issue, and doesn't seem like it would be a big deal, there is no reason why the AMD defs need to come first in the umd wrapper, and plenty of umd variations order differently. I do totally get the yuck factor of that solution tho. A more robust solution, and a fairly common one, would be to build just commonJS modules for the NPM package. If it makes a difference i'd be happy to help contribute that build step.

In terms of the second issue of dynamic json loads, I haven't run into this personally (or haven't realized I did), since I usually just require the json files directly instead of using cldr.load. From a lib auther perspective I don't usually need to mess with it, bc my library isn't loading culture files, the consumer would be, in which case a plugin would probably be nice for making that all simpler.

[rxaviers]

A more robust solution, and a fairly common one, would be to build just commonJS modules for the NPM package. If it makes a difference i'd be happy to help contribute that build step.

Your contribution is very welcome and I'm open for such change. But, first let's get this better defined. Let's move this discussion into: #467.

PS:

Changing the order fixes most of the issue, and doesn't seem like it would be a big deal, there is no reason why the AMD defs need to come first in the umd wrapper, and plenty of umd variations order differently. I do totally get the yuck factor of that solution tho.

Across various jquery projects, we use UMD/returnExports. So, if changes are made upstream, I won't object updating it in here as well.

[jzaefferer]

Can we go back to the inconsistency of cldr and cldrjs between the AMD and require dependencies?

    if ( typeof define === "function" && define.amd ) {
        define(["cldr","cldr/event"], factory );
    } else if ( typeof exports === "object" ) {
        module.exports = factory( require( "cldrjs" ) );
    } else {
          // global one omitted
    }

This is somewhat related to the cldrjs "export name" discussion: rxaviers/cldrjs#7

Maybe a better solution here is to actually rename cldrjs to cldr-traverse and use that consistently to specify the dependency? Would that resolve the webpack issue?

[rxaviers]

We could, but that alone would not solve this problem given AMD paths would
still need to be configured. I have a comment about it above

On Wednesday, August 5, 2015, Jörn Zaefferer [email protected]
wrote:

Can we go back to the inconsistency of cldr and cldrjs between the AMD
and require dependencies?

if ( typeof define === "function" && define.amd ) {
    define(["cldr","cldr/event"], factory );
} else if ( typeof exports === "object" ) {
    module.exports = factory( require( "cldrjs" ) );
} else {
      // global one omitted
}

This is somewhat related to the cldrjs "export name" discussion:
rxaviers/cldrjs#7 rxaviers/cldrjs#7

Maybe a better solution here is to actually rename cldrjs to cldr-traverse
and use that consistently to specify the dependency? Would that resolve the
webpack issue?

—
Reply to this email directly or view it on GitHub
#441 (comment).

+55 (16) 98138-1583, +1 (415) 568-5854, skype: rxaviers
http://rafael.xavier.blog.br

[rxaviers]

All,

#481 implements an example and it's supposed to fix all problems raised here. Therefore, it's going to be considered as a fix for this issue. It's almost complete and by now one should be able to run the example.

It would be great if you could check it and provide your feedback.

[rxaviers]

This has been fixed with webpack plugins, which are pretty solid now (Twitter mobile uses it). More details in the demos.

[nishantkagrawal]

@rxaviers I am having issues with bower-resolve-webpack-plugin and globalize-plugin to work together.

If I install gloablize in node_modules it works fine. But globalize-webpack-plugin doesn't pickup the globalize module from bower_components.

Note that all my other modules are loaded from bower_components using bower-resolve-webpack-plugin.

bower-resolve-plugin is using existing-directory plugin to process all the stuff. When webpack hits globalize, it ignores that plugin somehow.

The following is my configuration.
`
resolve: {
plugins: [
new BowerResolvePlugin(),

        new webpack.optimize.CommonsChunkPlugin({
            name: ['app', 'vendor']
        }),
    ],
    extensions: ['.js'],
    modules: ['node_modules', 'bower_components'],
    descriptionFiles: ['package.json', 'bower.json', '.bower.json'],
    mainFields: ['main', 'browser']
},

plugins: [
    new globalizePlugin({
        production: true, // true: production, false: development
        developmentLocale: 'en', // locale to be used for development.
        supportedLocales: ['en'], // locales that should be built support for.
        messages: 'messages/[locale].json', // messages (optional)
        output: 'globalize-compiled-data-[locale].[hash].js', // build output.                
    }),
],` 

The following is the error:
ERROR in ./generated/typescript/webpack-vendor.js Module parse failed: \generated\typescript\webpack-vendor.js Cannot find module 'globalize' You may need an appropriate loader to handle this file type. Error: Cannot find module 'globalize' at Function.Module._resolveFilename (module.js:469:15) at Function.Module._load (module.js:417:25) at Module.require (module.js:497:17) at require (internal/module.js:20:19) at Object.compileExtracts (some_path\node_modules\globalize-compiler\lib\compile-extracts.js:9:18) at GlobalizeCompilerHelper.compile (some_path\node_modules\globalize-webpack-plugin\GlobalizeCompilerHelper.js:63:33) at GlobalizeCompilerHelper.createCompiledDataModule (some_path\node_modules\globalize-webpack-plugin\GlobalizeCompilerHelper.js:37:35) at Parser.<anonymous> (some_path\node_modules\globalize-webpack-plugin\ProductionModePlugin.js:89:60) at Parser.applyPluginsBailResult (some_path\node_modules\tapable\lib\Tapable.js:109:27) at Parser.parser.plugin (some_path\node_modules\webpack\lib\dependencies\CommonJsRequireDependencyParserPlugin.js:59:27) at Parser.applyPluginsBailResult1 (some_path\node_modules\tapable\lib\Tapable.js:120:27) at Parser.walkCallExpression (some_path\node_modules\webpack\lib\Parser.js:896:18) at Parser.walkExpression (some_path\node_modules\webpack\lib\Parser.js:693:40) at Parser.walkExpressionStatement (some_path\node_modules\webpack\lib\Parser.js:443:7) at Parser.walkStatement (some_path\node_modules\webpack\lib\Parser.js:434:32) at Parser.<anonymous> (some_path\node_modules\webpack\lib\Parser.js:417:9) at Array.forEach (native) at Parser.walkStatements (some_path\node_modules\webpack\lib\Parser.js:415:13) at Parser.parse (some_path\node_modules\webpack\lib\Parser.js:1145:8) at some_path\node_modules\webpack\lib\NormalModule.js:200:17 at some_path\node_modules\webpack\lib\NormalModule.js:162:10 at some_path\node_modules\loader-runner\lib\LoaderRunner.js:370:3 at iterateNormalLoaders (some_path\node_modules\loader-runner\lib\LoaderRunner.js:211:10) at Array.<anonymous> (some_path\node_modules\loader-runner\lib\LoaderRunner.js:202:4) at Storage.finished (some_path\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:38:15) at some_path\node_modules\enhanced-resolve\lib\CachedInputFileSystem.js:69:9 at some_path\node_modules\graceful-fs\graceful-fs.js:78:16 at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:445:3) @ multi ./generated/typescript/webpack-vendor.js
Am I missing a sequence in which the plugins should be setup or anything else?

[rxaviers]

@nishantkagrawal please file an issue against rxaviers/globalize-webpack-plugin preferably with a reduced reproducible demo.

[russellborja]

Hi @rxaviers , I'm currently writing a library that depends on globalize but I have no control over consumers' webpack configs, i.e. globalize-webpack-plugin is not an option. What are my options here?

[compulim]

FYI, for newer Webpack:

{
  module: {
    rules: [
      { test: /globalize/, parser: { amd: false } }
    ]
  }
}