Dependency scanning does not support import statements on more than one line

[rahbari]

I just updated to 20170626, and I get this error while compiling:

ERROR - required "...." namespace never provided

20170521 works just fine.

[ChadKillingsworth]

I thought we got this fixed ...

[MatrixFrog]

If you can give us a small repro case that would be quite helpful.

[kranich]

I have encountered the same problem. As far as I can tell this happens whenever there is a circular dependency, e.g. two modules importing each other.

[brad4d]

Probably @ChadKillingsworth is the best person to fix this, but we can't really do anything without an example to reproduce the problem.
@rahbari and @kranich , could either of you provide one?

[ChadKillingsworth]

I may have an example - I can test it on https://github.com/PolymerLabs/polymer-closure-sample

[kranich]

Another (non-minimal) example would be https://github.com/kranich/ComplexCurves/tree/closure-compiler-20170626. I will see if I can come up with a minimal failing example.

[kranich]

Now I have a minimal (?) failing example at https://github.com/kranich/closure-compiler-issue-2559.

[kranich]

In my case, the issue seems to be that v20170626 no longer allows to spread import statements across multiple lines.

[ChadKillingsworth]

@concavelenz I believe had a plan for supporting that. @blickly have any suggestions in the mean time?

[blickly]

Looks like this is a known limitation of the regexp-based dependency parsing that we do in JsFileParser. It seems like the best fix would be to make the regexp-parser smarter, like maybe by processing an entire statement at a time (up to a ';') instead of a line at a time (up to a '\n').

In the meantime, the best workaround I can see is either:

• Adding an extra side-effect only import, which js-beautify doesn't wrap: import './Baz';
  or
• Manually specify the file order to Closure Compiler so that it doesn't have to do the reordering

[ChadKillingsworth]

A semi-colon isn't guaranteed though. You almost need a mini-parser there.

So externally we either need to go back to parsing every file for dependency info, or fix the regex parsing. This will break multiple projects.

[blickly]

Yeah, it's complicated. I think that the main reason for needing to do the regex based dependency "parsing" is that we have a few really huge projects that use manage_closure_dependencies and will blow up if all their input files are actually parsed with a real parser. For the ordering of files (as opposed to pruning), I'd think we could always use the more accurate full-parse based information without causing issues there.

Adding @shicks in case he has other ideas.

[blickly]

The other option, of course, is to do what we were (unintentionally) doing before and do a full parse for any file that contains an ES6 module before doing the pruning. I doubt that there are enough ES6 module uses yet for this to cause performance problems, but it seems less than ideal.

[ChadKillingsworth]

There is another option (for strict dependency modes at least): Parse the entry points and follow the path. Discard any files not reachable. This is very easy with ES6 and CommonJS modules as they are path based. Not sure how you'd approach goog.require/goog.provide code. I suppose you could do regex matching for those.

[ChadKillingsworth]

After thinking about this for a couple of days, here's what I propose:

If dependency management is enabled:

1. In order, do a full parse of the first entry point and get correct dependencies.
2. Parse each dependency discovered from the entry point - ordering the files during the parsing. This should be done depth first.
3. Move back to step one for the next entry point.

After traversal, discard any unreached files.

As I mentioned during the last meeting, we'll need a full parse to recognize the new dynamic import statement as it can occur anywhere. And yes, we wish to recognize this at build time.

[blickly]

This sounds like a good strategy to me for --dependency_mode=STRICT.

I guess that supporting goog.module/goog.require/goog.provide and --dependency_mode=LOOSE will still require us to keep using the regex-based "parser" for at least some of the cases, but it would mean a big savings for projects with STRICT dependency pruning and no live goog.module/goog.provide files.

[ChadKillingsworth]

Closed by #2641