[WEBLATE] Prevent code error within Weblate

[Mark-Giesen]

Translator usually know not to touch strings within {} or ```. But every now and then this goes wrong. The time it takes to find this error is usually some days (merge and test of the Weblate Pull Request). We have to fix the code and risk merging conflicts.
Could it be possible to write an add-on for Weblate to prevent {} blocks to be translated and ``` blocks to be transpiled. Errors could be provided to the translators at the moment of translation. This shortens the loop with days and gives translators much needed feedback.

[Mark-Giesen]

A first step has been made: all {} code is now shown grayed in Weblate. This brings extra attention to translators not to translate these strings. It is still possible though. Some more investigation wil go into this issue to find more ways to prevent errors due to translation.

[Mark-Giesen]

A second step has been made. Now inline code is shown grayed as well. Usually this is of a form like `{print}`. This is now completely greyed, not only the {print} part. In code blocks (with three ticks) the first step (code in {}) is still very usefull.
Some complex inline code examples exist, like {for} counter {is} 1 {to} 5. This is now greyed, but can be translated. f.e. in dutch: {for} i [is} 1 {to} 5. Translating counter into i seems reasonable, but Weblate now misses the first pattern and finds a new one.

We can dismiss this per string, so not a big problem, but is this the best way to deal with code?

[Felienne]

Ok I want to bump this issue cause it is making my life kinda miserable not having this!!

It is a lot of work to fix code errors in Weblate PRs and there are always a few, misspelled keywords, spaces in weird places and I end up spending a few hours each week fixes these small things which distracts a lot from other work but yeah we do want to gather more translations...!

What we would like (but I do not even know if this is possible!) is to do a call from Weblate to our parser (accessing /parse/) to see if a code snippet works and if not, stop Weblate from saving it. This might require us to run Weblate ourselves rather than use the hosted solution but that might be a price we are willing to pay?

[Felienne]

Maybe @jpelay could do a bit of Googling to see if there is something like a hook we can use? Or maybe @Mark-Giesen still wants to explore this?

[Mark-Giesen]

I had a lot of different communications with Weblate about this, finally resulting in what we have now as being "normally enough". But having runnable code to translate is far from normal, even for Weblate. I'll have to retrace my steps in this communication. I believe it is possible, but...? In the end; Weblate itself is open source, so if it's not in there, we can add it.
I might get to this tomorrow.

[Mark-Giesen]

I couldn't wait :-). One of the things that looked promissing was AUTOFIX_LIST. These are fixes Weblate silently does see here some examples. We could build one of our own like this.
This might be a quick way to prevent errors by simply checking if it compiles and if not, replacing it with the original again. We'd have to investigate a little further to see if we can get a message to the translator, maybe even with a smart error message and preventing the save. Normal behaviour for weblate is saving it anyway, the translators are in charge there.

[Felienne]

Ah thanks for looking into this further @Mark-Giesen! The https://docs.weblate.org/en/latest/admin/checks.html#custom-autofix looks promising! Can we dive into whether that would allow us to do a call to the (online) parser, or would we copy-paste the transpiler code in? And can we do this hosted too?

[Mark-Giesen]

I just posted a new summary question on my weblate helpdesk site. I believe this sums up the problem and options we have and the results I had sofar with Weblate. See message below:

As one of my previous requests allready states, we need a way to check a translation and prevent is from saving when not correct. Parts of our text is Python code. These blocks are recognized by three ticks (```). This could be something like:

{print} 'Hello world!'
{print} 'I'm learning Hedy with the tutorial!'

The parts between curlies should not be translated and therefore these are grayed with a regex. Usually this goes just fine, but every week we get code blocks that do not compile. For instance this Chinese translation, because it's missing a space between the print command and the text:

{print}'你好世界！'
{print}'我正在通过教程学习海迪！'

We can't (and won't) blame translators for writing failing code, but you can imagine dealing with these errors is a lot of work an almost impossible in some situations (we can't read chinese or hindi or arabic etc.).
In a previous question we discussed options and preventing translations seems to be only possible with "readonly", which we clearly don't want. Checks seemed an other option, but those don't prevent, just alarm the translator. What we need is something in between.
Wat we would like is a way to have these snippets parsed by our parser (it's online, there is a URL we can call) and if it fails, the save should be prevented. Best would be to give a error to the translator helping to fix the problem, but changing back to original (English) would also be feasable.

I believe I can build the last option with an AUTOFIX. I could even copy our parser code into the fix if calling an URL is not possible from your website.
My question now is, is this indeed possible, would you accept an PR with this. And could we call an URL as well, making the code more robust for changes on our side. And finally, could we create the fix so that it would give an error message to the translator and prevent the save?

[Mark-Giesen]

First reaction from Weblate is in, they wanted some example strings in our database to check on the configuration. Since we're on the open source free edition, the support is done on a little lower priority, but done! I expect further progress tomorrow.

[Felienne]

How cool that this is moving forward!!

[Felienne]

Hi @Mark-Giesen! Do you have any updates? @ToniSkulj wants to explore whether he can put something together in a fork or Weblate.

And @rix0rrr had a nice idea, we could also consider making a GreaseMonkey script? It would require people to install a plugin but it would be relatively easy and enthusiastic translators might be willing to do that. Something to think about if we don't succeed with Weblate.

[Mark-Giesen]

No, not yet. I'll asked for an update. Trying is a good idea. If we have something working, they might be able to react to that specific.

[Mark-Giesen]

Well, just received an answer:

If you want a string with a failing check not to be saveable, Enforce the check as you have it with Python brace format with this string:
https://hosted.weblate.org/translate/hedy/adventures/zh_Hans/?q=check:placeholders.
You can also write your own check:
https://docs.weblate.org/en/latest/admin/checks.html#writing-own-checks

Apparently the python brace check should prevent saving, thought this wasn't true, have to reinvestigate that. Anyway, we can write one ourselves (with the same restriction of not saving to work). If someone wants to check this out, be my guest. If not, I can have a look in my Christmas holidays, maybe a little sooner.

[Mark-Giesen]

Well, I had 5 minutes, so did a quick check to find out if my memory or this answer was the truth. The example string has 3 errors, of which one is enforced. The enforced one should have prevented this text to exist in de DB, my opinion. Changing the text is still possible. Add a full stop at the end and save, this works and one of the errors is fixed. This is what we don't want.
So one option left, writing one of our own and see if it can be done like we want it to.

[Mark-Giesen]

At FOSDEM I met the Creator of Weblate and discussed this issue. I finally got the confirmation that an enforced check does not prevent saving, but it prevents the "needs editing" to be cleared. A real "prevent saving" check might be pretty easy to build for them, but they don't have much time to spare on this item, but would like this to become part of the tool.
An PR in their repo would have a big chance to get merged. This would be a great next step, but needs some discussing.
The last step to create an external check (like our parser) would be some more work but also worth it.

[Mark-Giesen]

What needs to be discussed is whether or not we take the "stop saving" check approach first. It's easy (Weblate thinks) because all parts are available somewhere in Weblate. The last step (external check of code by our parser) is the final and best approach, but might take more effort and therefore more time.
The first step has one disadvantage, all languages need the same commands in the code. That is not always easy. Take this dutch translation for instance. The example uses a childs song and they differ per culture. The choice made here is to pick a song with an input and this is not like the original. If we create a "don't save" check, this choice is no longer valid. In dutch we could change this, we have more fitting songs about "visjes" or "kleutertjes", but some other languages might not.

So is it worthwhile to try the first option or shoud we skip that and go for the more complex solution at once?

[Felienne]

Hi @Mark-Giesen!

Pasting your text from Discord here so we can discuss it:

I think we now have 3 options to explore.

1. An Add-On that, when saving a translation, has code compiled by us and, in the event of an error, gives a message and NOT saves it. A translator without programming experience will sometimes not be able to do much with this other than inserting the old code again.
2. I think method 2 is faster and simpler: the AutoFix. Also an option in Weblate that does the same compile check when saving, but restores the old code in the event of an error. Faster, useful for translators who cannot fix any code, as they no longer have to revert anything themselves.
3. GreaseMonkey, no idea what that entails.

(3 by the way is a browser plugin, I think we can declare that out of scope for now)

As for 2, if we are going to do autofix, then it would be a lot less work to do the auto-fix on our side, than we do not need a plugin at all. Yesterday I have worked on a script in our tester that reverts broken code back to the original English. It is not ideal, but it is more or less ready to go! We will discuss tomorrow in the meeting also what other people think about the reverting (in our code or in Weblate)

[Mark-Giesen]

What's the status of this? Did the "revert broken code to EN" make it into production? And if so, is this issue closed then? Other issues make take precedence now, or do we want to investigate further options on Weblate side?

[Felienne]

What's the status of this? Did the "revert broken code to EN" make it into production? And if so, is this issue closed then? Other issues make take precedence now, or do we want to investigate further options on Weblate side?

Yes, see: #4544

My pressing time-sink problem of fixing Weblate is solved, so my life is noticeably better now! I think there is enough other work on Weblate for now, so even though this would still be nice to have, the amount of work it would be for you would be better spent elsewhere I think.