Infra spell bot #35224

OnkarRuikar · 2024-07-29T13:22:17Z

fixes Add US English spell checker/linter for markdown #12522

I've been running a spell-checker bot in my temp repo for over 1.5 years. The bot runs every Monday start and files an issue giving details about found typos/spelling mistakes in the mdn/content repo. The issue is resolved by fixing typos in mdn/content and adding new words in the project-words.txt file.

In the last weekly meeting, we got the green signal to move the bot to the content repo. The PR moves the bot to this repo.

The following changes have been made to existing files:

The project-words.txt file contains 5k+ ignored words gathered over the period.
The RegExs have been updated to handle umlauts in fragments, to allow valid=true case
There are multiple cases of favourite-colour. As the article is named in such a way, we have to ignore all the occurrences.

github-actions · 2024-07-29T13:23:36Z

Preview URLs

External URLs (1)

URL: /en-US/docs/Web/HTTP/Status/204
Title: 204 No Content

Parsing content after a 204 response httpwg/http-core#26 (1 time) (Note! This may be a new URL 👀)

(comment last updated: 2024-08-13 07:12:42)

rebloor

For the corrections to favourite-colour

Josh-Cena · 2024-08-01T19:15:10Z

.github/workflows/spelling-check-bot.yml

+      - name: Create an issue
+        if: env.OUTPUT != ''
+        uses: dacbd/create-issue-action@main
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          title: Weekly spelling check
+          body: |
+            ## Found typos and unknown words:
+            ${{ env.OUTPUT }}
+
+            > [!TIP]
+            > Add valid words to the https://github.com/mdn/content/blob/main/.vscode/project-words.txt file, and fix the typos.


Instead of creating one issue every week, is it possible to keep one dashboard issue that gets updated every week? No strong feelings, just because our issues tend to stack up and get forgotten—not everyone here has the habit of going through past issues.

I hope the typo issues will not linger more than a day. A maintainer won't even have to wait for a reviewer.

There are many things to handle. We'll have to reopen the issue every week and update it. We might need a custom implementation for this. If you know of any third-party action that does this out of the box, let me know.
How about we merge this as it is and get this bot going, then work on the recurring issue implementation?

.vscode/project-words.txt

Josh-Cena · 2024-08-11T15:55:51Z

@OnkarRuikar Perhaps you can split out the content change into a separate PR so it can be quickly merged? #35404

Josh-Cena · 2024-08-13T05:48:10Z

@OnkarRuikar Could you split all content changes into separate PRs? We don't want to loop more and more people into this.

Josh-Cena · 2024-08-13T07:33:16Z

~~We already have project-words.txt right? Why two?~~

I see that they won't be added to editor suggestions. However I think in ignored-words.txt some of them are real words like "granularities". This is going to be tricky ;)

OnkarRuikar · 2024-08-13T08:06:57Z

However I think in ignored-words.txt some of them are real words like "granularities". This is going to be tricky ;)

Because the bot was working standalone and the editor consideration was not there so all the words were accumulated in one file. There are 5578 words in the file, and classifying them is a huge time-consuming task. But we can do it later after the bot goes live.

If we could force contributors to use camel case variable names then a ton of words will get removed. 🙄 myvar vs myVar

.github/workflows/spelling-check-bot.yml

Co-authored-by: Brian Thomas Smith <[email protected]>

.github/workflows/spelling-check-bot.yml

Co-authored-by: Brian Thomas Smith <[email protected]>

OnkarRuikar · 2024-08-28T06:27:20Z

I've successfully tested the workflow.
Created Issue: OnkarRuikar#29
The workflow run: https://github.com/OnkarRuikar/content/actions/runs/10591216755/job/29348284353#step:4:45

bsmth · 2024-08-29T09:57:56Z

I've successfully tested the workflow. Created Issue: OnkarRuikar#29 The workflow run: https://github.com/OnkarRuikar/content/actions/runs/10591216755/job/29348284353#step:4:45

It looks good, tnx. I think we can merge this shortly. I would also echo the sentiment from Josh that we should try to reduce the 5k+ size dictionary by some means in a follow-up. Like:

moving actual technology terms from ignore-words to project-words
camelCasing variable names (requires content updates)
Some project documentation about the spellchecker, how to add words, what kinds of words go where (CONTRIBUTING.md?)

bsmth

One last thing I noticed; we have dupes in the ignore-words.txt file, could you uniq it first?

edit: Also if you think renaming the dictionaries will help with their purpose, we can do that in this PR, for instance ignored-words -> ignore-list?

OnkarRuikar · 2024-08-29T10:37:44Z

I ran sort file | uniq file > file on the word list files. How about names project-words-list.txt and ignored-words-list.txt for the files?

bsmth · 2024-08-29T12:06:55Z

I ran sort file | uniq file > file on the word list files.

nice, thank you

How about names project-words-list.txt and ignored-words-list.txt for the files?

IMO "words" is the part to get rid of in the ignore file, so I'd prefer to keep the original or do something like:

terms-abbreviations.txt
ignore-list.txt

What do you reckon?

OnkarRuikar · 2024-08-29T14:06:05Z

@bsmth terms-abbreviations.txt and ignore-list.txt sound good I've renamed the files and updated rest of the content.

bsmth

Looking good, let's give it a shot! Thanks, Onkar

OnkarRuikar · 2024-08-29T15:28:42Z

Don't forget to trigger the workflow manually.

OnkarRuikar requested review from a team as code owners July 29, 2024 13:22

OnkarRuikar requested review from rebloor and pepelsbey and removed request for a team July 29, 2024 13:22

github-actions bot added Content:WebExt WebExtensions docs system [PR only] Infrastructure and configuration for the project size/xl [PR only] >1000 LoC changed labels Jul 29, 2024

rebloor approved these changes Jul 30, 2024

View reviewed changes

Josh-Cena reviewed Aug 1, 2024

View reviewed changes

bsmth reviewed Aug 2, 2024

View reviewed changes

.vscode/project-words.txt Outdated Show resolved Hide resolved

OnkarRuikar marked this pull request as draft August 10, 2024 05:12

github-actions bot added Content:HTML Hypertext Markup Language docs Content:HTTP HTTP docs and removed Content:WebExt WebExtensions docs labels Aug 13, 2024

github-actions bot added size/m [PR only] 51-500 LoC changed and removed size/xl [PR only] >1000 LoC changed labels Aug 13, 2024

OnkarRuikar force-pushed the infra_spell_bot branch from cc99af8 to 33dda95 Compare August 13, 2024 07:20

github-actions bot added size/xl [PR only] >1000 LoC changed and removed Content:HTML Hypertext Markup Language docs Content:HTTP HTTP docs size/m [PR only] 51-500 LoC changed labels Aug 13, 2024

OnkarRuikar marked this pull request as ready for review August 13, 2024 07:28

OnkarRuikar commented Aug 27, 2024

View reviewed changes

.github/workflows/spelling-check-bot.yml Outdated Show resolved Hide resolved

OnkarRuikar commented Aug 27, 2024

View reviewed changes

.github/workflows/spelling-check-bot.yml Outdated Show resolved Hide resolved

OnkarRuikar and others added 5 commits August 27, 2024 10:21

feat(bot): spelling checking bot

2fcee3b

remove a duplicate pattern

581b46b

add HTTPWG

9d58e11

Apply suggestions from code review

ef798af

Co-authored-by: Brian Thomas Smith <[email protected]>

Update .github/workflows/spelling-check-bot.yml

40c444f

OnkarRuikar force-pushed the infra_spell_bot branch from 94c1034 to 40c444f Compare August 27, 2024 04:51

OnkarRuikar requested a review from bsmth August 27, 2024 04:51

OnkarRuikar marked this pull request as draft August 27, 2024 05:02

bsmth reviewed Aug 27, 2024

View reviewed changes

.github/workflows/spelling-check-bot.yml Outdated Show resolved Hide resolved

bsmth reviewed Aug 27, 2024

View reviewed changes

.github/workflows/spelling-check-bot.yml Outdated Show resolved Hide resolved

OnkarRuikar and others added 2 commits August 27, 2024 14:56

Apply suggestions from code review

4164d8c

Co-authored-by: Brian Thomas Smith <[email protected]>

update workflow

0887305

OnkarRuikar marked this pull request as ready for review August 27, 2024 13:33

OnkarRuikar requested a review from bsmth August 27, 2024 13:33

bsmth requested changes Aug 29, 2024

View reviewed changes

run sort-uniq on word files and rename them

c576166

OnkarRuikar requested a review from bsmth August 29, 2024 10:38

rename files

152ae79

bsmth approved these changes Aug 29, 2024

View reviewed changes

Merge branch 'main' into infra_spell_bot

fab57cf

bsmth merged commit 3f4f116 into mdn:main Aug 29, 2024
8 of 9 checks passed

OnkarRuikar deleted the infra_spell_bot branch August 29, 2024 15:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Infra spell bot #35224

Infra spell bot #35224

OnkarRuikar commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

rebloor left a comment

Josh-Cena Aug 1, 2024

OnkarRuikar Aug 2, 2024 •

edited

Loading

Josh-Cena commented Aug 11, 2024

Josh-Cena commented Aug 13, 2024

Josh-Cena commented Aug 13, 2024 •

edited

Loading

OnkarRuikar commented Aug 13, 2024

OnkarRuikar commented Aug 28, 2024

bsmth commented Aug 29, 2024

bsmth left a comment •

edited

Loading

OnkarRuikar commented Aug 29, 2024

bsmth commented Aug 29, 2024

OnkarRuikar commented Aug 29, 2024

bsmth left a comment

OnkarRuikar commented Aug 29, 2024

Infra spell bot #35224

Infra spell bot #35224

Conversation

OnkarRuikar commented Jul 29, 2024 • edited Loading

github-actions bot commented Jul 29, 2024 • edited Loading

rebloor left a comment

Choose a reason for hiding this comment

Josh-Cena Aug 1, 2024

Choose a reason for hiding this comment

OnkarRuikar Aug 2, 2024 • edited Loading

Choose a reason for hiding this comment

Josh-Cena commented Aug 11, 2024

Josh-Cena commented Aug 13, 2024

Josh-Cena commented Aug 13, 2024 • edited Loading

OnkarRuikar commented Aug 13, 2024

OnkarRuikar commented Aug 28, 2024

bsmth commented Aug 29, 2024

bsmth left a comment • edited Loading

Choose a reason for hiding this comment

OnkarRuikar commented Aug 29, 2024

bsmth commented Aug 29, 2024

OnkarRuikar commented Aug 29, 2024

bsmth left a comment

Choose a reason for hiding this comment

OnkarRuikar commented Aug 29, 2024

OnkarRuikar commented Jul 29, 2024 •

edited

Loading

github-actions bot commented Jul 29, 2024 •

edited

Loading

OnkarRuikar Aug 2, 2024 •

edited

Loading

Josh-Cena commented Aug 13, 2024 •

edited

Loading

bsmth left a comment •

edited

Loading