Need relative paths in source map #1676

mk0x9 · 2015-01-31T03:39:27Z

Hi, is there anyway to generate source map without the aboslute path in the file field and without the sourceRoot field?

$ traceur --source-maps file --out project.js main.js 
$ cat project.map | jq .
{
  "sourcesContent": [
    "System.registerModule($__placeholder__0, [], $__placeholder__1);",
    "function() {\n        $__placeholder__0\n      }",
    "var __moduleName = $__placeholder__0;",
    "var $__placeholder__0 = $__placeholder__1",
    "class Foo {\n  static hello() {\n    return 'Oh, hi!';\n  }\n}\n\nexport default Foo;\n",
    "($traceurRuntime.createClass)($__placeholder__0, $__placeholder__1, $__placeholder__2)",
    "var $__default = $__placeholder__0",
    "return $__placeholder__0",
    "get $__placeholder__0() { return $__placeholder__1; }",
    "import Foo from 'lib/foo';\n\nconsole.log(Foo.hello());\n",
    "System.get($__placeholder__0)",
    "System.get($__placeholder__0 +'')"
  ],
  "sourceRoot": "/tmp/project/",
  "file": "/tmp/project/project.js",
  "mappings": "AAAA,KAAK,eAAe,AAAC,WAAoB,GAAC,CCA1C,UAAS,AAAD;;ACAR,AAAI,IAAA,CAAA,YAAW,YAAoB,CAAC;ACApC,AAAI,IAAA,MCAJ,SAAM,IAAE,KAIR,ADJwC,CAAA;AEAxC,EAAC,eAAc,YAAY,CAAC,AAAC,WDCpB,KAAI,CAAX,UAAa,AAAD,CAAG;AACb,WAAO,UAAQ,CAAC;IAClB,ECHmF;ACArF,AAAI,IAAA,CAAA,UAAS,EFME,IAAE,AENgB,CAAA;ACAjC,SCAA,aAAwB;AAAE,uBAAwB;IAAE,EDA7B;ANEjB,CDFwD,CAAC;AAA/D,KAAK,eAAe,AAAC,WAAoB,GAAC,CCA1C,UAAS,AAAD;;ACAR,AAAI,IAAA,CAAA,YAAW,YAAoB,CAAC;IOA7B,IAAE,ECAT,CAAA,MAAK,IAAI,AAAC,WAAkB;ADE5B,QAAM,IAAI,AAAC,CAAC,GAAE,MAAM,AAAC,EAAC,CAAC,CAAC;AFFxB,WAAuB;ANEjB,CDFwD,CAAC;AWA/D,KAAK,IAAI,AAAC,CAAC,WAAmB,GAAC,CAAC,CAAA",
  "names": [],
  "sources": [
    "@traceur/generated/TemplateParser/6",
    "@traceur/generated/TemplateParser/5",
    "@traceur/generated/TemplateParser/4",
    "@traceur/generated/TemplateParser/7",
    "lib/foo.js",
    "@traceur/generated/TemplateParser/8",
    "@traceur/generated/TemplateParser/1",
    "@traceur/generated/TemplateParser/3",
    "@traceur/generated/TemplateParser/2",
    "main.js",
    "@traceur/generated/TemplateParser/9",
    "@traceur/generated/TemplateParser/10"
  ],
  "version": 3
}

Are there any cases when full path to project's directory is needed inside sourceRoot?

I've tried some hacking, but no luck so far except breaking tests and sourcesContent field :)

The text was updated successfully, but these errors were encountered:

johnjbarton · 2015-01-31T17:21:52Z

What is your use case for relative sourceRoot? The whole point of sourceRoot is to set the root for the relative paths in the 'sources' property.

mk0x9 · 2015-01-31T19:09:10Z

According to source map spec sourceRoot is used to relocate sources on the server. Also traceur populates sourceContents source names are used only to resolve original file names used during compilation. IMO there are really no reason to use this excessive information, especially for the web facing scripts.

For the same reason sourceMappingURL in the compiled script should be referencing source map with relative path, not the /home/user/projects/project/main.js because browser will look for it, but definitely not where developer intends browser to look for it.

Also sourceRoot somehow breaks uglify-js2 intermediate sourcemap transformation (I'll fill issue there later), but that's not really related here.

Now --source-root <false|string|default>. The default option is as before. Fixes google#1676

johnjbarton · 2015-02-03T01:02:32Z

This is all the spec says about sourceroot:

Resolving Sources
If the sources are not absolute URLs after prepending of the “sourceRoot”, the sources are resolved relative to the SourceMap (like resolving script src in a html document).

jmm · 2015-02-03T16:37:05Z

@johnjbarton First of all, in the spec when they say "absolute URL" do they really mean that? E.g. http://example.com/something/. Or do they mean an absolute path, e.g. /something/?

Here's a summary of the issues (overlaps significantly if not completely with @tensai-cirno):

I don't want my absolute local file system paths exposed in the source map, period.
Where there are relative paths in the source map I want them to be relative to either (depending on the situation) the source map's URL, the document root (host), or perhaps even some absolute URL if that's indeed supported by the spec and browsers.

I think part of the problem is that traceur is currently munging 2 different things into the sourceRoot param of the compile functions, isn't it? To achieve the goals I listed, traceur would need 2 pieces of information I think:

An absolute local file system path to use as the base for making relative paths for sources.
A URI to output in the map as sourceRoot.

johnjbarton · 2015-02-03T16:48:32Z

PR #1686 is what I propose to do here.

jmm · 2015-02-03T18:18:51Z

Please forgive me if I've missed something, but how does #1686 address the issues raised here? It still doesn't appear to be possible to achieve all of these things at once:

Keep absolute local file system paths out of sourceRoot and sources.
Populate sources with paths relative to a specified base path.
Specify the desired sourceRoot output (since it's currently used as a local path to generate relative paths for sources).

Say I have this dir structure:

+-- somedir/
    +-- src/
        +-- js/
            +-- entry.js
            +-- sub/
                +-- b.js

And I want to compile b.js and get a source map like this:

{
  sourceRoot: '/some/uri',
  sources: ['js/sub/b.js]
}

How could I achieve that? Currently I can only seem to get either the output I want for sourceRoot or sources, not both. sourceRoot pertains to the output paths and I think it's incorrectly being used here to resolve input paths. I think there should be either a separate parameter for what sourceRoot is being used for currently (resolving local paths), or turn the sourceRoot input param into something like this:

{in: '/some/local/path', out: '/some/uri'}

johnjbarton · 2015-02-03T18:24:10Z

sourceRoot pertains to the output paths and I think it's incorrectly being used here to resolve input paths.

Please point to the source where this is happening. I can't imagine how that can work, the input paths have to resolve to actual files.

jmm · 2015-02-03T18:28:32Z

I mean in the spec it pertains to the paths in sources:

If the sources are not absolute URLs after prepending of the “sourceRoot”

Or do you mean that I'm wrong that the sourceRoot parameter to the compiler is being used to ~~resolve local paths~~ generate the relative paths for sources?

johnjbarton · 2015-02-03T18:38:19Z

I mean that as far as I can tell the sourceRoot parameter is not used to resolve input paths. It has no effect on compilation.

The sourceRoot parameter is used to convert absolute paths of input source files to relatives paths in the sources array. As far as I am aware that is the complete function of the sourceRoot, a way to abbreviate the content of the sources array.

In PR #1686 we allow the sourceRoot to be undefined, and thus it has no impact on the sources array.

jmm · 2015-02-03T19:20:54Z

I mean that as far as I can tell the sourceRoot parameter is not used to resolve input paths. It has no effect on compilation.

"Resolve paths" was a poor choice of words on my part.

The sourceRoot parameter is used to convert absolute paths of input source files to relatives paths in the sources array.

Yes, this is what I was referring to (which was better worded when I described it as "it's currently used as a local path to generate relative paths for sources").

As far as I am aware that is the complete function of the sourceRoot, a way to abbreviate the content of the sources array.

The spec only references it in terms of applying it to the sources output though. Applying it to input paths confuses its purpose I believe. It should be an output value, not treated as a local path.

In PR #1686 we allow the sourceRoot to be undefined, and thus it has no impact on the sources array.

That's my point -- you can choose to either generate the sources paths you want or the sourceRoot output you want, not both. Again, how would I acheive the desired source map output illustrated in my previous comment?

I believe that what the sourceRoot input param is currently being used for, and the sourceRoot output need to be split into separate values.

johnjbarton · 2015-02-04T01:09:45Z

Again, how would I acheive the desired source map output illustrated in my previous comment?

I can't see how that sourcemap could be useful. The combination of sourceRoot and sources does not exist in the file system.

I think we should land PR #1686 and have you try the --source-root=false and then tell me errors you get and what tool you see them in. What do you think?

Now --source-root <false|string|default>. The default option is as before. Fixes google#1676

jmm · 2015-02-04T14:52:06Z

The combination of sourceRoot and sources does not exist in the file system.

Exactly. How would absolute local file system paths from where traceur compiles be useful to a web browser receiving a source map from a server? My understanding is that the sources values are for identifying (and possibly retrieving) the source files, and sourceRoot is a URL for prepending to sources values to request them from the server.

According to the spec:

Resolving Sources

If the sources are not absolute URLs after prepending of the “sourceRoot”

Both values, sourceRoot and sources elements, are components of a URL for the browser to request a file, are they not? The files requested at these URLs may have existed anywhere, or not at all, when traceur compiled and may exist anywhere or not at all when requested. Two totally separate concepts.

I can't see how that sourcemap could be useful.

It would be useful if I have a source map at http://example.com/whatever/map.js and I want the browser to be able to request a source file at http://example.com/some/uri/js/sub/b.js.

If I have a local file system /a/b/c/d/e/whatever.js and I set up a web host like:

<VirtualHost *:80>
  Server example.com
  DocumentRoot /a/b/c/d
</VirtualHost

Then /e/whatever.js is a path relevant to the web browser. /a/b/c/d/e/whatever.js certainly isn't.

I think we should land PR #1686 and have you try the --source-root=false and then tell me errors you get and what tool you see them in. What do you think?

I can tell you right now: if sourceName is an absolute local file system path, it's exposed in sources. What sort of value is sourceName supposed to be?

johnjbarton · 2015-02-04T15:54:13Z

It would be useful if I have a source map at http://example.com/whatever/map.js and I want the browser to be able to request a source file at http://example.com/some/uri/js/sub/b.js.

Couldn't you set --source-root=false --out=whatever/out.js and compile from the directory containing /some?

jmm · 2015-02-04T16:05:21Z

Perhaps in some situation. I'm talking about making this a general purpose tool that allows the desired output to be generated and doesn't marry the output to a file structure that may or may not exist where traceur compiles.

mnieper · 2015-02-04T16:12:33Z

I have to agree with @jmm. In general it is not possible to duplicate the virtual path structure of a web application on the local file system of the machine used to build the images - nor is it desirable.

domenic · 2015-02-04T16:12:37Z

Perhaps someone can help me understand why Traceur does such complicated (and, from my perspective both as a consumer and maintainer of a few tools built on top Traceur, nonsensical) things with source map filenames. Have you looked at how other tools such as 6to5 do it? Absolute file paths have no place in source map output...

johnjbarton · 2015-02-04T16:28:08Z

If you want something different just send a pull request.

domenic · 2015-02-04T16:29:02Z

I'd like to understand the motivation behind the current setup first.

johnjbarton · 2015-02-04T16:37:39Z

The motivation is someone wanted sourcemaps then someone wanted sourceroot. And options setting in Traceur was such a nightmare, minimal changes were best. I think I've mostly solved this side problem. But work on sourcemaps is a bottomless pit in my experience.

I think the best solution is relative sources and no sourceroot (my original solution). Note that the sources are included in the map so we do not need any path in sourceroot to allow debugging. The only extra benefit I see from sourceroot is support for Chrome devtools workspaces and in that case the virtual path issues don't apply.

I would like to return to the question of whether #1686 is a net improvement or should be dropped.

jmm · 2015-02-04T16:45:52Z

I would like my question about sourceName to be answered before I comment on that.

johnjbarton · 2015-02-04T16:59:53Z

What sort of value is sourceName supposed to be?

There are several things called sourceName, all of them paths. Only one is a input, an argument to compile() which is used to set the source.name in the parse tree.

Maybe there are scenarios where sources ends up being absolute paths?

johnjbarton · 2015-02-04T17:10:51Z

It would be useful if I have a source map at http://example.com/whatever/map.js and I want the browser to be able to request a source file at http://example.com/some/uri/js/sub/b.js.

Or set --source-root=http://example.com/path-to-output --out=path-to-output. Then the sources should be relative to path-to-output and the resulting urls would be http://example.com/some/uri/js/sub/b.js. Would that work?

mnieper · 2015-02-04T17:29:40Z

Wouldn't this mean that I have to create a file path-to-output on my build machine?

jmm · 2015-02-04T17:51:31Z

Only one is a input, an argument to compile() which is used to set the source.name in the parse tree.

That's the one I'm referring to. What is source.name used for or what does it affect? Does it ~~name~~ make a difference if sourceName is an absolute or relative path? Does it correspond to any file system access?

Maybe there are scenarios where sources ends up being absolute paths?

What do you mean? Scenarios where you want them to be absolute paths?

Then the sources should be relative to path-to-output and the resulting urls would be http://example.com/some/uri/js/sub/b.js.

I don't understand the example. Can you please provide an example API call illustrating the input parameters, and the resulting output for sourceRoot and sources.

Traceur misappropriates sourceRoot to apply to local input paths and that shouldn't be propagated to consumers of this package. See google/traceur-compiler#1676

johnjbarton · 2015-02-04T19:07:52Z

What is source.name used for or what does it affect?

sources in the sourcemap

Does it name a difference if sourceName is an absolute or relative path?

As far as I know the only effect is if the sourceRoot matches the first part of the sourceName. Then the sources entry is relative.

Does it correspond to any file system access?
Not in Traceur.

Scenarious where you want them to be absolute paths?

No, I wonder it happens but is not desired.

jmm · 2015-02-04T19:14:46Z

No, I wonder it happens but is not desired.

You wonder if it happens? Yes, that's a big part of what this discussion has been about. I quote myself:

I can tell you right now: if sourceName is an absolute local file system path, it's exposed in sources

jmm · 2015-02-04T19:24:45Z

sourceRoot can not be treated as a local path (e.g. used as the base to generate relative paths for sources) and output in the source map. If outputting sourceRoot makes sense at all then there should be a sourceRoot input argument or property to traceur that is passed through to output and used for nothing else, at least by default. It sounds like to get relative paths in sources, the desired path should be passed as sourceName. This is easily enough accomplished when using the API, using the sourceRoot value or another path:

compile(
  src,
  path.relative('/whatever', sourceName),
  outputName,
  sourceRoot
)

I'm not as familiar with the CLI, but perhaps it needs an additional flag like --basedir to use for half of what --source-root is being used for right now, and / or perhaps there could be some convenience flag to allow opt-in to use sourceRoot in that capacity.

johnjbarton · 2015-02-05T00:13:32Z

Yep, sourceName ends up in sources. But this bug and #1686 are about sourceRoot.

johnjbarton · 2015-02-05T05:06:27Z

I changed PR #1686 to ensure that the 'sources' are relative and that they are not related to the sourceRoot input. I added a test including a sourceroot set to a URL.

jmm · 2015-02-05T14:15:53Z

Yep, sourceName ends up in sources. But this bug and #1686 are about sourceRoot.

sourceName and sourceRoot were yoked by sourceRoot being misapplied to alter sourceName values.

I changed PR #1686 to ensure that the 'sources' are relative

What do you mean by that? The sourceName input should just be passed through unchanged, right? Is that what you mean -- that they're just not being altered anymore?

and that they are not related to the sourceRoot input.

Good, thank you.

While checking out these changes I discovered a different problem: sourceName input is also altered based on outputName input. I'm not crystal clear on how the file property of source maps is intended to be used, but what's the rationale for altering sourceName based on that value? Or is it just an accident?

(new traceur.NodeCompiler({sourceMaps: 'inline'}))
  .compile(';', '/a/b/c.js', '/x/y/z.js')

Results in "../../a/b/c.js" in the sources output.

jmm · 2015-02-05T14:29:12Z

What's the rationale for automatically generating a sourceRoot value from the outputName value?

johnjbarton · 2015-02-05T16:31:12Z

I think I have addressed this issue raised here. If not we need to start fresh anyway. We may decide to default sourceRoot to false.

johnjbarton mentioned this issue Feb 2, 2015

Source Maps Url Problem #1685

Closed

johnjbarton added a commit to johnjbarton/traceur-compiler that referenced this issue Feb 2, 2015

Change --source-root option to allow omitting sourceRoot

70b54cf

Now --source-root <false|string|default>. The default option is as before. Fixes google#1676

johnjbarton mentioned this issue Feb 2, 2015

Change --source-root option to allow omitting sourceRoot #1686

Closed

arv assigned johnjbarton Feb 2, 2015

arv added the in progress label Feb 2, 2015

johnjbarton added a commit to johnjbarton/traceur-compiler that referenced this issue Feb 4, 2015

Change --source-root option to allow omitting sourceRoot

d92f4bd

Now --source-root <false|string|default>. The default option is as before. Fixes google#1676

jmm mentioned this issue Feb 4, 2015

Bump Traceur to 0.0.82 thlorenz/es6ify#80

Open

jmm added a commit to jmm/es6ify that referenced this issue Feb 4, 2015

Adjust to take basedir opt instead of sourceRoot.

3e6d526

Traceur misappropriates sourceRoot to apply to local input paths and that shouldn't be propagated to consumers of this package. See google/traceur-compiler#1676

mciparelli mentioned this issue Feb 4, 2015

Having problems with source maps & gruntfile config aaronfrost/grunt-traceur#58

Open

johnjbarton closed this as completed in ce902a6 Feb 5, 2015

arv removed the in progress label Feb 5, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need relative paths in source map #1676

Need relative paths in source map #1676

mk0x9 commented Jan 31, 2015

johnjbarton commented Jan 31, 2015

mk0x9 commented Jan 31, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

Resolving Sources

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

mnieper commented Feb 4, 2015

domenic commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

domenic commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

mnieper commented Feb 4, 2015

jmm commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

jmm commented Feb 4, 2015

johnjbarton commented Feb 5, 2015

johnjbarton commented Feb 5, 2015

jmm commented Feb 5, 2015

jmm commented Feb 5, 2015

johnjbarton commented Feb 5, 2015

Need relative paths in source map #1676

Need relative paths in source map #1676

Comments

mk0x9 commented Jan 31, 2015

johnjbarton commented Jan 31, 2015

mk0x9 commented Jan 31, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 3, 2015

jmm commented Feb 3, 2015

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

Resolving Sources

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

mnieper commented Feb 4, 2015

domenic commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

domenic commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

mnieper commented Feb 4, 2015

jmm commented Feb 4, 2015

johnjbarton commented Feb 4, 2015

jmm commented Feb 4, 2015

jmm commented Feb 4, 2015

johnjbarton commented Feb 5, 2015

johnjbarton commented Feb 5, 2015

jmm commented Feb 5, 2015

jmm commented Feb 5, 2015

johnjbarton commented Feb 5, 2015