Add US English spell checker/linter for markdown

[hamishwillee]

We should have a US English spell checker.

#373 changed all know UK English spellings to US English, as per MDN policy. However this is likely to rot. As discussed in #10787 we should also have an automated spell checker system in place for English content.

[nschonni]

FYI, here are the config files I've been using with cspell to look at the diffs on the custom dictionary to find new misspellings https://gist.github.com/nschonni/4d271853c2af1612bcf9f5c739620ce5

[bsmth]

We may consider checking in a project dictionary under ./vscode that can be updated incrementally using CSpell? This would work well with https://github.com/streetsidesoftware/cspell-action which can also run incrementally (on changed files only).

In my experience, content is looking okay (not overwhelmed with errors) with config dictionaries like this when working on PRs that edit a few markdown files only:

{
  "$schema": "https://raw.githubusercontent.com/streetsidesoftware/cspell/main/cspell.schema.json",
  "version": "0.2",
  "language": "en-US",
  "languageId": "*",
  "dictionaries": [
    "bash",
    "css",
    "cpp",
    "django",
    "filetypes",
    "fonts",
    "fullstack",
    "html",
    "latex",
    "lorem-ipsum",
    "markdown",
    "node",
    "npm",
    "project-words",
    "python",
    "softwareTerms",
    "svelte",
    "typescript"
  ],
  "ignorePaths": [
    ".vscode/cspell.json",
    ".vscode/extensions.json",
    ".markdownlint-cli2.jsonc"
  ],
  "allowCompoundWords": true,
  "dictionaryDefinitions": [
    {
      "name": "project-words",
      "path": "./project-words.txt",
      "addWords": true
    }
  ]
}

If we extend using the ignorelist from @nschonni, I think we're in a good place.

[nschonni]

I believe there is/was a performance difference with a large custom dictionary inside the config file vs the external TXT

[nschonni]

I normally use cspell --no-progress --words-only --unique '**' > project-words.txt || sort project-words.txt -o project-words.txt to generate the missing words dictionary. I would previously stage this file and git, then review the diffs to find new misspellings too

[bsmth]

I believe there is/was a performance difference with a large custom dictionary inside the config file vs the external TXT

I see, so having a large external text file (project-words.txt) incurs a performance knock, does it?

[nschonni]

No, I believe the external txt is more performant, as it gets compiled once, where the JSON file gets repeatedly parsed, so smaller is better for that file

[bsmth]

No, I believe the external txt is more performant, as it gets compiled one, where the JSON file gets repeatedly parsed

Very good, I'm all for trying this out in parallel with our usual day-to-day and evaluating how it fits into our workflow after some time.

[bsmth]

Minor update that the config is checked for local use:

• feat: check in a cspell config file #28487

[OnkarRuikar]

I've gathered 5424 word list for this repo so far. If you wish we can merge this or use personally in local environment. Here is the config I used: https://github.com/OnkarRuikar/temp/blob/main/cspell.json.

I think we can start using cspell in our automations.

[Josh-Cena]

Is everyone onboard with doing this in CI? Do we need a discussion?

[OnkarRuikar]

Is everyone onboard with doing this in CI? Do we need a discussion?

We can't fail a PR if someone adds a new acronym(like HDCP) or uses a word like unzoomed. We can make it a nightly/weekly check and file an issue if new/wrong words are found.

[Josh-Cena]

Sure, that also sounds fine to me, like how you are regularly sending typo-fix PRs.

[OnkarRuikar]

I can simply move the automation from my temp repo to mdn/content. This won't affect anything but let's discuss this in the next weekly meeting.

[bsmth]

I've been using this action on different repositories and it's been quite good, something to consider:

jobs:
  spellcheck:
    name: "Check spelling"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: streetsidesoftware/cspell-action@v6
        with:
          inline: error
          config: ".vscode/cspell.json"
          verbose: true
          # Limit the files checked to the ones in the pull request or push.
          incremental_files_only: true

[OnkarRuikar]

I've been using this action on different repositories and it's been quite good, something to consider:

Could you share an example of it catching errors? How does it handle new words?

[bsmth]

I've been using this action on different repositories and it's been quite good, something to consider:

Could you share an example of it catching errors? How does it handle new words?

It leaves annotations for spelling errors on the job, like:

spellcheck: content/posts/thing#L25
Unknown word (argaman)
spellcheck: files/thing.css#L115
Unknown word (diabled)
spellcheck
2 spelling issues found in 2 of the 44 files checked.

All of the failure logs from this workflow are expired, so I don't have an example available, but I can recommend trying it out as there's a lot of configuration possible that we won't have to re-roll: https://github.com/streetsidesoftware/cspell-action