Support / samples on 'equivalent' languages (2-step process)

[fvet]

Related to #74

Hi @rvanbekkum,

Looking for some guidelines on how to achieve the following:

• Our main development language is English.
• After packaging, the g.xliff is created with English captions (this file is never touched)

English (en-US)

• We don't ship a separate en-US.xliff file (which would be a copy of the g.xliff files), since BC falls back on the captions included in the AL source files.

French (fr-BE / fr-FR)

• NOTE: We make no distinction between fr-BE and fr-FR
• Part of the release process is synching any new transunits from g.xliff to the fr-FR xliff files and maintain / review this fr-FR file
• fr-BE uses the caption of its primary language (fr-FR)

Dutch (nl-BE / nl-NL)

• NOTE: We do make a distinction between nl-BE and nl-NL
• Part of the release process is synching any new transunits from g.xliff to the nl-BE xliff files.
• Once the nl-BE xliff files have been updated / reviewed, we would like to sync these new / updated transunits to the nl-NL files, so we can benefit from the nl-BE effort and review/update the new / update translations coming from nl-BE

How can we adopt the settings below to achieve a similar proces?

[fvet]

Just did some tests but don't get the expected result:

• Workspace with repo with g.xlf / nl-BE.xlf / nl-NL.xlif
• Project settings

"xliffSync.equivalentLanguagesEnabled": true,
    "xliffSync.equivalentLanguages": {
        "nl-BE": "nl-NL"
    },

• Add a label in code
• AL: Package
• g.xlf is updated
• select g.xlf
• Synchronize translation units
• New trans-unit element is added to nl-BE.xlf and nl-NL.xlf

<target state="needs-translation"/>

• Open nl-BE.xlf and add the translation
• Trans-unit target state is updated to 'translated'

<target state="translated">value</target>

- Now I'd like to apply the nl-BE translations to the nl-NL file and mark those as 'Needs translation'

• Any hints on how to achieve this?

<target state="needs-l10n">value</target>

Using the equivalentLanguages seems not to make any difference.

[rvanbekkum]

Hi @fvet,
There seemed to be an issue with xliffSync.equivalentLanguages: it used workspace.getConfiguration, but I just saw/learned that this method merges the default value with the values at different scopes (see https://stackoverflow.com/questions/65192859/for-workspace-getconfiguration-how-do-i-get-a-setting-from-the-multi-root-works).
I will make adjustments for this (already have them ready) and ship them in a future update.

The equivalentLanguages feature simply copies everything from master to slave languages. There is currently no configurations of marking them with a different state. What do you think might be good for this?

[fvet]

@rvanbekkum
For the Powershell variant - which we will use for our pipeline - it would be good if the sync commandlets would support an optional parameter to set a 'state' that is assigned when (just 'translated' ?) trans-units are synced from source/parent to target/slave.

We would not use the parameter from en > nl-BE, but could use the parameter to sync from nl-BE to nl-NL.

For the VSCode extension, it should be possible to set this 'state' parameter on the parent/slave config.

"xliffSync.equivalentLanguages": {
    {
        parent: "nl-BE",
       slave:  "nl-NL",
      state: "To translate"
   }
}

or add a new setting ?

 "xliffSync.equivalentLanguageStates": {
        {
            "nl-BE": "To translate"
       }
    },

[fvet]

Thinking on some scenarios:

---

1. Happy flow (g > nl-BE > nl-NL)

Add label

• Added new label : g.xlif gets a new trans-unit
• Sync to nl-BE : new trans-unit is added without translation / state 'needs-translation'
• Update translation in nl-BE
• Sync to nl-NL with state 'needs review' : new trans-unit is added with translation / with state 'needs-review'

Update label

• Update label : g.xlif gets an update trans-unit
• Sync to nl-BE : current trans-unit is updated with state 'needs-review' (since source is updated)
• Update translation in nl-BE
• Sync to nl-NL with state 'needs review' : current trans-unit is updated with state 'needs-review' (since source is updated)

---

2. Specific flow (g > nl-NL / nl-BE) : nl-NL is not waiting for nl-BE to be ready and already started adding translations (by force syncing g > nl-NL)

Add label

• Added new label : g.xlif gets a new trans-unit
• Force sync to nl-NL: new trans-unit is added without translation / with state 'needs-translation'
• Sync to nl-BE : new trans-unit is added without translation / with state 'needs-translation'
• Update translation in nl-NL
• Update translation in nl-BE
• Sync to nl-NL with state 'needs review' : trans-unit is already available in state translated (and source is same) > no change
  / keep

Update label

• Update label : g.xlif gets an update trans-unit
• Force sync to nl-NL: current trans-unit is added with state 'needs-review' (since source is updated)
• Sync to nl-BE : current trans-unit is added with state 'needs-review' (since source is updated)
• Update translation in nl-NL
• Update translation in nl-BE
• Sync to nl-NL with state 'needs review' : trans-unit is already available in state translated (and source is same) > no change
  / keep

So I think when using the new 'state' parameter (or a more rigid 'needs translation' parameter) for syncing, we should only apply the state for new trans-units added to the target language, or if source has changed.

[rvanbekkum]

A fix for the issue of not being able to fully override the xliffSync.equivalentLanguages setting has been shipped in release 1.2.0.

I will come back at the other part of this issue later on. 😉

[fvet]

Hi @rvanbekkum, any chance of having the above included - on short term (mid september?) as part of xliff-sync?
If not, we'll see internally if we can build something around it ...

[rvanbekkum]

Hi @fvet,
Yes!

[rvanbekkum]

Ok, my plan was to do the following. Could you let me know if that makes sense to you as well? I haven't gone much further yet on this as I think it would be good to have some thought together on the feature, with consideration of how the equivalentLanguages feature is currently implemented.

So, what equivalentLanguages now actually does, is copying(!!) the contents of the master language's file to the slave languages' files.
To allow for alternative behaviour we could add a new setting:

  "xliffSync.equivalentLanguagesStrategy": {
      "type": "string",
      "default": "copy",
      "description": "Specifies which strategy to follow for updating slave languages' files from master languages' files.",
      "enum": [
          "copy",
          "sync"
      ],
      "enumDescriptions": [
          "Copy contents of master language's file to slave languages' files.",
          "Synchronize from master language's file to slave languages' files, filling in non-translated entries only, i.e., without overwriting translations."
      ],
      "scope": "resource"
  },

The default being the current behaviour, "copy", just make the contents exactly the same.
And the "sync" option passing on translations without overwriting, adding a needs-adaptation state to the newly added translations (if it was empty, but now filled through the master file) or if the source text changed.

We could also change the naming, e.g., calling the options "force" and "sync", or something like that.

[fvet]

This new 'sync' option is indeed what we're looking for. I'm pro the new strategy setting ;)

[rvanbekkum]

Hi @fvet,
Haven't forgotten about this request, but should have mentioned I set it on hold until I got the time to continue on it. Will try to find some time for it soon! 😉

[fvet]

Hi @rvanbekkum , this issue is no longer relevant to me. If no one feels the need for this feature, feel free to close the issue.

[es-mh]

Hi Rob,
is the equivalentLanguagesStrategy Feature still on the roadmap?
Thanks,
Michael

[rvanbekkum]

Hi @es-mh,
Yes, it is still on the planning.