Option to produce source maps (src-dest line mapping)

[Zearin]

Since one of CSON’s main selling points is that of a readable alternative to JSON, many people will be writing in CSON, and converting to JSON for use with other code. If something happens in that generated code, it’s desirable to have some way to find where the JSON’s relevant counterpart is in the CSON.

I know that JSON doesn’t actually allow comments. Maybe…

• an option to output a line-mapping from CSON ► JSON
• offer event hooks for people to customize however they like
• …?

[Zearin]

@bevry Ping

[Zearin]

Oddly enough, I never noticed that the coffee command line tool has an option --map. For normal coffeescript, this generates a *.map file, and adds a directive at the end of the compiled Javascript with the location of the *.map file.

Obviously, this ability is invaluable for debugging.

I would love this for CSON!

But JSON doesn’t allow comments (…wtf…). And the directive at the end of the compiled JS output? Yeah, it’s a comment.

So, here’s what I propose…

Usage: cson2json [options] filePath

Options:
    -m, --map=<key>          generate source map and save as .map files;
                             adds <key> to the output with a reference to
                             the source map

This accomplishes the following:

• Builds on existing code in the coffeescript compiler. (That should make it easy on you, @bevry!)
• Allows users to develop with CSON fearlessly. (That should make it easy on your users; given JSON’s refusal to allow comments, the user simply picks a <key>—whatever works for them—and VOILA! They can develop in CSON, test the compiled JSON, and trace errors back to the original CSON files.)

I’m pushing for this because I’m actually in this exact situation. I bet it would make CSON a more attractive alternative to YAML (yuck!). 😉

@bevry ? What do you think? (Tell me you still love this repo! :D)

[Zearin]

@bevry ping

[Zearin]

@bevry ping ping ping ping ping :)

[Zearin]

@bevry I’m dying, here! Please tell me this can be done…

[RobLoach]

Source maps for CSON files?

[Zearin]

@RobLoach Yep. I’m working on a project where this is sorely needed.

We’re using CSON to write human-friendly tests, and the compiled JSON for test runners. When a test fails, we need the line number of the CSON, because that’s what people will be using.

[Zearin]

(Forgot to explain…Why are we using JSON at all? Because these tests are going to be run by many different languages. They can all read JSON, but the tests needs to be human-friendly in ways that JSON is incapable of; linebreaks, multiline strings, and so forth.

Therefore, CSON is perfect for humans, JSON is perfect for multiple languages to parse and run tests, and the only thing that’s missing is a line map back to the original source when a test breaks.)

[RobLoach]

Sounds pretty neat. I'm sure there could be a way to get source maps working. Have you investigated at all? Would be a great feature to have.

[Zearin]

Sounds pretty neat. I'm sure there could be a way to get source maps working. Have you investigated at all? Would be a great feature to have.

Unfortunately, I’m amazingly bad at {Java|Coffee}Script. :) I’m trying to learn, but it’s coming slowly. So, it’s not within my current skillset to investigate.

The best I could do is spot CoffeeScript’s ability to generate maps between *.coffee files and *.js files. I did suggest a way to allow this feature through CSON’s CLI; and the ability to do so programmatically would also be awesome.

But alas! I can only ask for a guide to the promised land. I do not know the way. :-/

[Zearin]

That said…if you check out the CoffeeScript mapping feature, see what needs to happen, and can walk me through it, I’d be happy to help you code through any nitty-gritty. I just need your guidance. :)

[Zearin]

(Forgot…last comment was for @RobLoach .)

[Zearin]

@RobLoach, @balupton:

It took a bit of stumbling, but I’ve almost got something working for this. Slightly messy, but it’s passing about half the tests.

The tests fail with 5.coffee:

module.exports = {
  filename: require('path').basename(__filename)
  abc: [ "a", "b", "c" ]
  a:
    b: "c"
}

For whatever reason, I’m getting the following error:

/« … »/cson/test/src/5.coffee:3
  abc: [ "a", "b", "c" ]
  ^^^
async ➞  5.coffee ➞  parse source file ✘   
async ➞  5.coffee ✘  
async ✘  

FAILURE: 20/21 tests ran successfully; 1 failed, 0 incomplete, 1 errors

Error #1:
async ➞  5.coffee ➞  parse source file
SyntaxError: Unexpected identifier
  at Module._compile (module.js:439:25)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.app.requireFresh (/« … »/cson/node_modules/requirefresh/out/lib/requirefresh.js:30:14)
  at /« … »/cson/node_modules/requirefresh/out/lib/requirefresh.js:16:22
  at b (domain.js:183:18)
  at Domain.run (domain.js:123:23)
  at app.requireFreshSafe (/« … »/cson/node_modules/requirefresh/out/lib/requirefresh.js:13:7)
  at Object.CSON.parseFile (/« … »/cson/out/lib/cson.js:31:9)
  at _Class.<anonymous> (/« … »/cson/out/test/cson-async-test.js:34:23)
  at ambi (/« … »/cson/node_modules/joe/node_modules/taskgroup/node_modules/ambi/out/lib/ambi.js:23:18)
  at fire (/« … »/cson/node_modules/joe/node_modules/taskgroup/out/lib/taskgroup.js:163:25)
  at b (domain.js:183:18)
  at Domain.run (domain.js:123:23)
  at _Class.Task.fire (/« … »/cson/node_modules/joe/node_modules/taskgroup/out/lib/taskgroup.js:173:25)
  at processImmediate [as _immediateCallback] (timers.js:330:15)


npm ERR! Test failed.  See above for more details.
npm ERR! not ok code 0

/« … »/cson/Cakefile:99
      throw err;
            ^
Error: Process exited with error status code
  at ChildProcess.<anonymous> (/« … »/cson/Cakefile:88:15)
  at ChildProcess.EventEmitter.emit (events.js:98:17)
  at maybeClose (child_process.js:743:16)
  at Process.ChildProcess._handle.onexit (child_process.js:810:5)

I’ll be publishing a branch with my changes shortly, so you can see what’s happening under the hood.

[Zearin]

(On a side note, it would be nifty if the test files could explain what they are testing. Since adding comments into the test files themselves would alter the tests—test 3.cson tests the handling of comments, for instance—perhaps adding a README.md in src/test/ with descriptions of the test files would do.)

[Zearin]

(Opened #39 as a standalone issue. 😃)

[Zearin]

My embarrassing attempt at this is now published. :)

Zearin/cson@b590c17

[Zearin]

(EDIT: D’oh! I originally linked to the wrong SHA in the last comment. Fixed now.)

[Zearin]

@balupton @RobLoach:

I’m still stuck. Can I get a little help? Anybody?

[Zearin]

So, CSON 2.0 is out! And its parser has been abstracted to groupon/cson-parser. Yay!

I’m rusty on the code for this. Does anyone know if it’s possible to get source maps of the original CSON to the generated JSON? (Maybe the refactoring of groupon/cson-parser made it easier…?)

[balupton]

This is now an issue for groupon/cson-parser rather than for us. When they add sourcemap support to their cson parser, we'll add support for their API in bevry/cson.