Local build error for translated-content (macro parsing / h2m macro calls conversion incl. JSON / arrays)

[SphinxKnight]

Steps to reproduce

1. Fetch yari or mdn/content
2. yarn install / yarn start
3. Try to navigate to localhost:5000/fr/docs/Web/HTML/Element

Actual result

Build error / cf https://github.com/mdn/translated-content/runs/3787142211

Expected result

Build is OK as per production: https://developer.mozilla.org/fr/docs/Web/HTML/Element

Notes

• I've tried to apply current French (or even English) to my local translated-content: same error.
• Blocking Prepare HTML section for Markdown conversion translated-content#2628 so far
• I'll submit a PR when time allows / I'll dig deeper

[SphinxKnight]

When trying to minimize the case, I can see that this is not due to the French/locale content on the side but rather the presence of

{{HTMLRefTable({"include":\["HTML embedded content"], "exclude":\["multimedia"]})}}

cf. https://github.com/mdn/content/blob/main/files/en-us/web/html/element/index.md?plain=1#L63

[SphinxKnight]

When looking for this form of macro call, I'm seeing it only for this doc (over the whole docset which is mdn/content). I'm going to submit a PR to remove the escape which breaks the parser.

Note to self: I used this regexp \{\{.*?\\\[.*?\}\}

[escattone]

@SphinxKnight Thank you so much for digging into this, much appreciated! ❤️

[SphinxKnight]

Well, now I think the parser did its job and it's more a "html2md" conversion side effect. I'll try to figure out a PR here to fix the root cause. I would that the probability of this happening again is quite small given the major sections of mdn/content have been converted to markdown and I could find only one occurrence.

💀 to old KS macros

[schalkneethling]

Well, now I think the parser did its job and it's more a "html2md" conversion side effect. I'll try to figure out a PR here to fix the root cause. I would that the probability of this happening again is quite small given the major sections of mdn/content have been converted to markdown and I could find only one occurrence.

💀 to old KS macros

I am wondering though, would this be worth digging into for other localization communities? We are probably going to activate es soon and if there is a bug in the html2md tool then they are bound to run into it. Perhaps I misunderstand though so, let me know if this is actually not a problem anymore. Thanks!

[SphinxKnight]

@schalkneethling as far as I can tell, this issue is still valid. I'll (try to) reproduce in the day and will update here.

[schalkneethling]

@schalkneethling as far as I can tell, this issue is still valid. I'll (try to) reproduce in the day and will update here.

Great! Thanks, @SphinxKnight

[SphinxKnight]

@schalkneethling here is the status of local checks with fresh repos (2cb8c53 for mdn/content and 8748d8c for mdn/translated-content, ae33b02 for mdn/yari)

• Build errors

  • (reproduced) Build error for localized content based on English content, the latter having incorrectly escaped content : When adding {{HTMLRefTable({"include":\["HTML embedded content"], "exclude":\["multimedia"]})}} in ./content/files/en-us/web/html/element/index.md and after running node build/cli.js -l fr /web/html/element/index.md, I get an error (namely error: Expected "-", "0", "[", "\"", "false", "null", "true", "{", [ \t\n\r;], or [1-9] but "\\" found.).

  • (reproduced) Build error for localized content when incl. bad escape sequences: when introducing {{HTMLRefTable({"include":\["HTML embedded content"], "exclude":\["multimedia"]})}} in ./translated-content/files/fr/web/html/element/index.md it fails as well

• Converter behavior

  • (reproduced) The h2m converter introduces "bad" escape sequences when the HTML content has a macro where an argument is an object having an array as a property (ex. <p>{{HTMLRefTable({"include": ["HTML forms"], "exclude":["Deprecated"]})}}</p>}. For instance, one can take https://github.com/mdn/translated-content/blob/2727d3c76fd54476ecc29fc01b6ce3fe8e33b582/files/fr/web/html/element/index.html and run
    1. yarn run md h2m web/html/element --locale fr --mode replace
    2. node build/cli.js -l fr /web/html/element/index.md
       which causes an error. The newly created markdown file contains {{HTMLRefTable({"include": \["HTML forms"], "exclude":\["Deprecated"]})}}

Hope this helps :)

[schalkneethling]

Hey @SphinxKnight,

Thank you again for all of the information you provided here. This is indeed an issue that needs to be addressed but, we do not currently have the availability to dissect and address this properly. Seeing that so much of the content has already been converted, this always seems like something that is not going to come up a lot. As such, we are going to close this for now but, should someone in the community want to take on this issue, please let us know and we will happily reopen it.