tools: introduce `make mdlint` command and tools #12756

watilde · 2017-04-30T06:07:08Z

Summary

I'm trying to put this PR as a replacement for #11610 and #8551.

cc @ChALkeR @thefourtheye

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

Affected core subsystem(s)

tools, doc

wooorm

Awesome! Just one question and one suggestion!

wooorm · 2017-04-30T09:11:39Z

doc/guides/writing-tests.md

 'sources': [
  'test/cctest/test_env.cc',
  ...
 ],
 ```
 The test can be executed by running the `cctest` target:
-```
+```text


Shouldn’t this be sh or bash or something instead of text?

console is the correct language for those that start with a $.

I updated it to console. It's handy :)

wooorm · 2017-04-30T09:14:38Z

doc/api/v8.md

@@ -1,6 +1,6 @@
 # V8


Why not use references and definitions in this file?

Oh yeah, why not! I've updated it.

ChALkeR · 2017-04-30T10:06:47Z

@watilde Thanks so much for doing this, I will take a look =).

ChALkeR · 2017-04-30T10:10:15Z

doc/api/v8.md

-[`vm.Script`]: vm.html#vm_new_vm_script_code_options
-[here]: https://github.com/thlorenz/v8-flags/blob/master/flags-0.11.md
-[`GetHeapSpaceStatistics`]: https://v8docs.nodesource.com/node-5.0/d5/dda/classv8_1_1_isolate.html#ac673576f24fdc7a33378f8f57e1d13a4
-


Is there a reason for inlining those?

No, there wasn't. I've updated it!

thefourtheye · 2017-04-30T11:24:35Z

Makefile

@@ -960,6 +966,7 @@ endif
  lint \
  lint-ci \
  list-gtests \
+	mdlint \


Nit: Extra indentation.

thefourtheye · 2017-04-30T11:27:09Z

Makefile

@@ -853,6 +853,11 @@ bench: bench-net bench-http bench-fs bench-tls

 bench-ci: bench



You should probably edit the $(TARBALL) target as well, to exclude the tools/remark*

That's right! I added it on L717.

ChALkeR · 2017-04-30T11:34:46Z

This currently fails CI at the linter stage.

refack · 2017-04-30T12:56:13Z

~~Before I dive in, one question, what are the benefits over #12563?~~ Answered.
👍

vsemozhetbyt · 2017-04-30T12:58:44Z

@refack If I get it right, eslint-plugin-markdown checks js code in md files, while this PR aims to check markdown syntax itself, so they do not intersect.

refack · 2017-04-30T13:04:28Z

We need the switch in vcbuild.bat as well🤷, sorry...

Also for the reasons I mention in https://github.com/nodejs/node/pull/12744/files#diff-6a3371457528722a734f3c51d9238c13R54 Could you split this into two PRs:

adding the tools
activating the tool, and fixing.

Since I want to review, and I usually use the webUI for that, and 2181 files kills my browser :(

P.S. does anyone have a better review tool?

vsemozhetbyt · 2017-04-30T13:15:53Z

Linter errors:

doc\api\fs.md:
226:7 error Parsing error: Unexpected token {

The js tag was removed to conform with eslint-plugin-markdown. Maybe you can use console or add an ESlint disabling comment.
doc\guides\writing-tests.md:
323:10 error Parsing error: Unexpected token :
tools\remark-preset-lint-node\index.js:
31:1 error Line 31 exceeds the maximum line length of 80 max-len
39:2 error Missing semicolon semi

addaleax · 2017-04-30T14:16:13Z

Could you split this into two PRs:

There should be no need for separate PRs. But yeah, it would be nice to have the changes that only add dependencies in their own commit.

watilde · 2017-04-30T19:32:45Z

Thanks all for your review comments! I just divided the commit into several commits for the review, and coped with the review comments.

For the vcbuild.bat, unfortunately I cannot run vcbuild.bat on my environment. I'd like to pass the implementation to another contributor who has the execution environment after this patch gets merge if it's fine to you all.

watilde · 2017-04-30T19:41:51Z

ci: https://ci.nodejs.org/job/node-test-pull-request/7763/

ChALkeR · 2017-05-01T00:08:01Z

.eslintignore should include tools/remark-cli — we don't want that to be checked. It currently passes, but nevertheless.

remark-preset-lint-node should not be ignored, though (it currently isn't, mentioning just in case).

ChALkeR · 2017-05-01T00:10:38Z

Makefile

+mdlint:
+	@echo "Running Markdown linter..."
+	$(NODE) tools/remark-cli/cli.js -q \
+	  doc/api doc/api_assets doc/guides src lib benchmark tools/doc/ tools/icu/


Is there a reason why are other stuff from doc is ignored (e.g. onboarding.md and changelogs)?

Changelogs should be excluded since they have commit messages that make some lint errors and shouldn't be changed. The other docs can be included, then I will update it with --ignore-path=doc/changelogs.

Hm. There are two lint errors in the whole doc dir currently:

doc/changelogs/CHANGELOG_V6.md 1325:95-1325:107 warning Don’t pad `emphasis` with inner spaces no-inline-padding remark-lint doc/releases.md 298:1-302:4 warning Missing code-language flag fenced-code-flag remark-lint

I don't agree that the commit messages in the changelog should not be touched — those could be actual errors that made changelog look bad. E.g. if a changelog includes __proto__, we don't want it to render as «proto», we want to render it as «__proto__» or «__proto__» — so amending the changelog to escape those things sounds reasonable to me, and I would want the linter to catch those errors. See commit a58b48bc3bc, for example.

So I would prefer if this just replaced handle__ with either `handle__` or handle\_\_ in the changelogs (and similarly for handle_).

Looking at this comment, I thought that anyone should not touch the commit message in the changelog anyway, but it's fine if we've polished it in the past because it's absolutely better for users who read the rendered docs. Will update it :)

Yeah, that was about typos, which is pretty much the actual content — it makes sense not to change that.

And the fact that commit messages (initially in text) are converted to changelogs (which are Markdown) without proper escaping is mostly a tooling issue here (which is likely not worth fixing, though — those are very rare and the linter would catch that).

I guess that whatever generates the changelogs should have escaped some symbols like `_*~\, prefixing those with \. But for the cases that are already there and that require manual fixing, perhaps using inline code blocks makes more sense where appropriate.

I may also include the root directory as well to check them(e.g. README.md, CONTRIBUTING.md).

Rebased with your suggestion🐱

@ChALkeR remark is pretty good at escaping and generating markdown btw ;)

@wooorm, I filed an issue to nodejs/changelog-maker#47.

watilde · 2017-05-01T11:34:29Z

~~ci: https://ci.nodejs.org/job/node-test-pull-request/7771/~~
(I didn't add the tool sources into .eslintignore)

new ci: https://ci.nodejs.org/job/node-test-pull-request/7772/

watilde · 2017-05-02T07:01:12Z

The failed test on FreeBSD seems unrelated: https://ci.nodejs.org/job/node-test-commit-freebsd/nodes=freebsd10-64/8681/console

watilde · 2017-05-02T07:28:15Z

I'm thinking it would be easier to manage the rule set if I add the preset plugin as just a directory with a package.json which has private: true prop.

lpinca · 2017-05-02T08:40:17Z

It's a bit impractical to review the first 2 commits. Last 3 LGTM.

ChALkeR · 2017-05-02T14:42:03Z

The remark-preset-lint-node package that contains the remark settings for Node.js documentation and that was created for this purpose (introduced in the second commit here) has upstream location in https://github.com/watilde/remark-preset-lint-node and is copyright by @watilde.

I personally don't have anything against that (so no action required atm), but just to be sure: @nodejs/ctc — is that something that we better want in the foundation or not?

Edit: /ping @nodejs/tsc

PR-URL: #12756 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Sakthipriyan Vairamani <[email protected]>

MylesBorins · 2017-10-31T07:32:33Z

when this landed it should have come with a license update, without it we would be in violation of the remark-cli MIT license. I've opened #16637 to fix this.

gibfahn · 2017-10-31T13:55:28Z

Should land with #16637 if it lands on v6.x.

MylesBorins · 2017-11-16T21:10:43Z

I'd like to figure out what the story is with dependencies before landing this.

I think it is safe to add tools in semver-patch, but I don't want to break testing the tarballs

CONTRIBUTING.md + L857: Unused definition + L861: Unused definition + L863: Unused definition doc/api/assert.md + L719: Unused definition doc/api/async_hooks.md + L460: Missing code-language flag doc/api/child_process.md + L1362: Unused definition doc/api/dns.md + L674: Unused definition doc/api/esm.md + L178: Missing code-language flag doc/api/http.md + L1868: Unused definition + L1887: Unused definition + L1888: Unused definition + L1889: Unused definition + L1916: Unused definition + L1917: Unused definition doc/api/https.md + L260: Unused definition doc/api/os.md + L1226: Unused definition doc/api/process.md + L1888: Unused definition doc/api/stream.md + L2227: Definitions with the same identifier doc/guides/writing-and-running-benchmarks.md + L1: Missing newline character at end of file Refs: nodejs/node#12756 PR-URL: nodejs/node#16385 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Anatoli Papirovski <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Gibson Fahnestock <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>

PR-URL: nodejs/node#12756 Reviewed-By: Refael Ackermann <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Sakthipriyan Vairamani <[email protected]>

PR-URL: nodejs#14707 Refs: nodejs#12756 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> Reviewed-By: Sakthipriyan Vairamani <[email protected]>

Backport-PR-URL: #19447 PR-URL: #14707 Refs: #12756 Reviewed-By: Luigi Pinca <[email protected]> Reviewed-By: Daijiro Wachi <[email protected]> Reviewed-By: Sakthipriyan Vairamani <[email protected]>

nodejs-github-bot added build Issues and PRs related to build files or the CI. tools Issues and PRs related to the tools directory. labels Apr 30, 2017

watilde added the doc Issues and PRs related to the documentations. label Apr 30, 2017

wooorm reviewed Apr 30, 2017

View reviewed changes

ChALkeR self-assigned this Apr 30, 2017

ChALkeR reviewed Apr 30, 2017

View reviewed changes

thefourtheye mentioned this pull request Apr 30, 2017

introduce doc linter #11610

Closed

2 tasks

thefourtheye reviewed Apr 30, 2017

View reviewed changes

watilde force-pushed the feature/make-mdlint branch 2 times, most recently from 248a8bb to 3d6a5c0 Compare April 30, 2017 19:20

ChALkeR reviewed May 1, 2017

View reviewed changes

ChALkeR mentioned this pull request May 1, 2017

markdown special symbols in commit description should be escaped nodejs/changelog-maker#47

Closed

watilde force-pushed the feature/make-mdlint branch from 3d6a5c0 to 7498d01 Compare May 1, 2017 09:25

watilde force-pushed the feature/make-mdlint branch from 7498d01 to fd939da Compare May 1, 2017 11:36

gibfahn mentioned this pull request Oct 31, 2017

v8.9.0 proposal #16630

Merged

watilde mentioned this pull request Oct 31, 2017

node-test-linter npm installs remark-cli for each build #16628

Closed

gibfahn added the lts-watch-v6.x label Oct 31, 2017

richardlau mentioned this pull request Nov 20, 2017

doc: require CI status indicator in PRs #17151

Closed

1 task

MylesBorins added the baking-for-lts PRs that need to wait before landing in a LTS release. label Dec 20, 2017

vsemozhetbyt mentioned this pull request Jan 18, 2018

doc: add missing link references #18222

Closed

3 tasks

MylesBorins removed the baking-for-lts PRs that need to wait before landing in a LTS release. label Aug 17, 2018

		@@ -853,6 +853,11 @@ bench: bench-net bench-http bench-fs bench-tls

		bench-ci: bench

tools: introduce make mdlint command and tools #12756

tools: introduce make mdlint command and tools #12756

Conversation

watilde commented Apr 30, 2017 • edited Loading

Summary

Checklist

Affected core subsystem(s)

wooorm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChALkeR Apr 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChALkeR commented Apr 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChALkeR commented Apr 30, 2017 • edited Loading

refack commented Apr 30, 2017 • edited Loading

vsemozhetbyt commented Apr 30, 2017 • edited Loading

refack commented Apr 30, 2017

vsemozhetbyt commented Apr 30, 2017 • edited Loading

addaleax commented Apr 30, 2017

watilde commented Apr 30, 2017

watilde commented Apr 30, 2017

ChALkeR commented May 1, 2017 • edited Loading

ChALkeR May 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChALkeR May 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

watilde commented May 1, 2017 • edited Loading

watilde commented May 2, 2017

watilde commented May 2, 2017

lpinca commented May 2, 2017

ChALkeR commented May 2, 2017 • edited Loading

MylesBorins commented Oct 31, 2017

gibfahn commented Oct 31, 2017

MylesBorins commented Nov 16, 2017

tools: introduce `make mdlint` command and tools #12756

tools: introduce `make mdlint` command and tools #12756

watilde commented Apr 30, 2017 •

edited

Loading

ChALkeR Apr 30, 2017 •

edited

Loading

ChALkeR commented Apr 30, 2017 •

edited

Loading

refack commented Apr 30, 2017 •

edited

Loading

vsemozhetbyt commented Apr 30, 2017 •

edited

Loading

vsemozhetbyt commented Apr 30, 2017 •

edited

Loading

ChALkeR commented May 1, 2017 •

edited

Loading

ChALkeR May 1, 2017 •

edited

Loading

ChALkeR May 1, 2017 •

edited

Loading

watilde commented May 1, 2017 •

edited

Loading

ChALkeR commented May 2, 2017 •

edited

Loading