[5.0] Publish to multiple channels #7626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

lukas-lansky wants to merge 38 commits into dotnet:main from lukas-lansky:multiple-channels-for-5.0

Documentation/OneLocBuild.md

-Original file line number
+Diff line change
@@ -0,0 +1,82 @@
+    # Localization with OneLocBuild in Arcade
+    As of April 1, 2021, all .NET repositories will be using OneLocBuild for localization. Documentation on this system can
+    be found [here](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task).
+    This system is **not a replacement for Xliff-Tasks**; rather, it is replacing the localization team's old system called
+    Simple Loc. Xliff-Tasks will continue to be used in addition to OneLocBuild.
+    To make OneLocBuild easier to use, we have integrated the task into Arcade. This integration is a job template
+    ([here](/eng/common/templates/job/onelocbuild.yml)) that is described in this document.
+    ## LocProject.json Index File
+    The core component of OneLocBuild is the LocProject.json file. This file is an index file containing references to
+    all of the files to localize in your repository. In order to reduce overhead for repo owners, we are auto-generating
+    the LocProject.json file (PowerShell script [here](/eng/common/generate-locproject.ps1)). This script and the
+    OneLocBuild Azure DevOps task are both wrapped in the onelocbuild.yml job template.
+    The script searches your checked-in code for all localized XLF files and template JSON files. Files can be excluded
+    using a checked in file (/Localize/LocExclusions.json). The LocExclusions file excludes files based on simple matching.
+    For example, the LocExclusions.json file below will exclude everything in a directory called `tests` and any file
+    which include `test.xlf` in its name.
+    ```json
+    {
+        "Exclusions": [
+            "\\tests\\",
+            "test.xlf"
+        ]
+    }
+    ```
+    The selected files are then added to a generated LocProject.json file. At this point, template currently provides two options
+    for how to proceed.
+    ### Build-Time Generation
+    **The recommended path** is to have the script pass the generated LocProject.json directly to the OneLocBuild task.
+    This is the simpler of the two methods and removes the overhead of needing to maintain a checked in
+    LocProject.json file. The LocProject.json file is emitted in build logs and as a build artifact for examination.
+    ### Build-Time Validation
+    While it is **not the recommended path**, repos can instead opt to check in a static LocProject.json and have the
+    script compare it against the generated one. If they differ, the script will break the build so that a dev can
+    update either the LocProject.json or the LocExclusions.json file accordingly.
+    Because the script can be run locally, devs can also do this validation prior to pushing their changes.
+    ### Custom LocProject.json Files
+    Currently, the LocProject.json generation script only creates fairly uniform LocProject.json files. If your repository
+    requires the use of any of the more complex LocProject.json features as described in the OneLocBuild docs linked above,
+    the OneLocBuild template in this doc will not work and you will need to check in and maintain the LocProject.json file
+    manually.
+    ## OneLocBuild Template Parameters
+    The most basic structure for calling the OneLocBuild template is:
+    ```yaml
+    jobs:
+    - template: /eng/common/templates/job/onelocbuild.yml
+      parameters:
+        LclSource: lclFilesfromPackage
+        LclPackageId: 'LCL-PACKAGE-ID'
+    ```
+    The parameters that can be passed to the template are as follows:
+    | **Parameter** | **Default Value** | **Notes** |
+    |:-:|:-:|-|
+    | `RepoType` | `'gitHub'` | Should be set to `'gitHub'` for GitHub-based repositories and `'azureDevOps'` for Azure DevOps-based ones. |
+    | `SourcesDirectory` | `$(Build.SourcesDirectory)` | This is the root directory for your repository source code. |
+    | `CreatePr` | `true` | When set to `true`, instructs the OneLocBuild task to make a PR back to the source repository containing the localized files. |
+    | `AutoCompletePr` | `false` | When set to `true`, instructs the OneLocBuild task to autocomplete the created PR. Requires permissions to bypass any checks on the main branch. |
+    | `UseCheckedInLocProjectJson` | `false` | When set to `true`, instructs the LocProject.json generation script to use build-time validation rather than build-time generation, as described above. |
+    | `LanguageSet` | `VS_Main_Languages` | This defines the `LanguageSet` of the LocProject.json as described in the [OneLocBuild task documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=languageset%2C-languages-(required)). |
+    | `LclSource` | `LclFilesInRepo` | This passes the `LclSource` input to the OneLocBuild task as described in [its documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=languageset%2C-languages-(required)). For most repos, this should be set to `LclFilesfromPackage`. |
+    | `LclPackageId` | `''` | When `LclSource` is set to `LclFilesfromPackage`, this passes in the package ID as described in the [OneLocBuild task documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=scenario-2%3A-lcl-files-from-a-package). |
+    | `condition` | `''` | Allows for conditionalizing the template's steps on build-time variables. |
+    It is recommended that you set `LclSource` and `LclPackageId` as shown in the example above.

NuGet.config

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -5,7 +5,7 @@
  
      </solution>

      <packageSources>

        <clear />

        <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />

        <add key="dotnet-public" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" />

        <add key="dotnet-tools" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" />

        <add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />

        <add key="vs-buildservices" value="https://pkgs.dev.azure.com/azure-public/vside/_packaging/vs-buildservices/nuget/v3/index.json" />

azure-pipelines.yml

-Original file line number
+Diff line change
@@ Expand Up / @@ -4,12 +4,14 @@ trigger: @@
         include:
         - master
         - release/3.x
+        - release/5.0
     pr:
       branches:
         include:
         - master
         - release/3.x
+        - release/5.0
         - templates
     variables:
@@ Expand Down @@

eng/Version.Details.xml

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -7,25 +7,25 @@
  
        </Dependency>

      </ProductDependencies>

      <ToolsetDependencies>

        <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.20478.3">

        <Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="5.0.0-beta.21328.3">

          <Uri>https://github.com/dotnet/arcade</Uri>

          <Sha>c8a7611e15656f45b039d46ec4e09f9f2a7edf99</Sha>

          <Sha>5266aa9856457785b84739fda2616f21da7ee6b4</Sha>

        </Dependency>

        <Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="5.0.0-beta.20478.3">

        <Dependency Name="Microsoft.DotNet.Build.Tasks.Feed" Version="5.0.0-beta.21328.3">

          <Uri>https://github.com/dotnet/arcade</Uri>

          <Sha>c8a7611e15656f45b039d46ec4e09f9f2a7edf99</Sha>

          <Sha>5266aa9856457785b84739fda2616f21da7ee6b4</Sha>

        </Dependency>

        <Dependency Name="Microsoft.DotNet.SignTool" Version="5.0.0-beta.20478.3">

        <Dependency Name="Microsoft.DotNet.SignTool" Version="5.0.0-beta.21328.3">

          <Uri>https://github.com/dotnet/arcade</Uri>

          <Sha>c8a7611e15656f45b039d46ec4e09f9f2a7edf99</Sha>

          <Sha>5266aa9856457785b84739fda2616f21da7ee6b4</Sha>

        </Dependency>

        <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.20478.3">

        <Dependency Name="Microsoft.DotNet.Helix.Sdk" Version="5.0.0-beta.21328.3">

          <Uri>https://github.com/dotnet/arcade</Uri>

          <Sha>c8a7611e15656f45b039d46ec4e09f9f2a7edf99</Sha>

          <Sha>5266aa9856457785b84739fda2616f21da7ee6b4</Sha>

        </Dependency>

        <Dependency Name="Microsoft.DotNet.SwaggerGenerator.MSBuild" Version="5.0.0-beta.20478.3">

        <Dependency Name="Microsoft.DotNet.SwaggerGenerator.MSBuild" Version="5.0.0-beta.21328.3">

          <Uri>https://github.com/dotnet/arcade</Uri>

          <Sha>c8a7611e15656f45b039d46ec4e09f9f2a7edf99</Sha>

          <Sha>5266aa9856457785b84739fda2616f21da7ee6b4</Sha>

        </Dependency>

        <Dependency Name="Microsoft.DotNet.Maestro.Client" Version="1.1.0-beta.20258.6">

          <Uri>https://github.com/dotnet/arcade-services</Uri>

eng/Versions.props

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -60,8 +60,8 @@
  
        <XUnitVersion>2.4.1</XUnitVersion>

        <XUnitAbstractionsVersion>2.0.3</XUnitAbstractionsVersion>

        <XUnitVSRunnerVersion>2.4.1</XUnitVSRunnerVersion>

        <MicrosoftDotNetBuildTasksFeedVersion>5.0.0-beta.20478.3</MicrosoftDotNetBuildTasksFeedVersion>

        <MicrosoftDotNetSignToolVersion>5.0.0-beta.20478.3</MicrosoftDotNetSignToolVersion>

        <MicrosoftDotNetBuildTasksFeedVersion>5.0.0-beta.21328.3</MicrosoftDotNetBuildTasksFeedVersion>

        <MicrosoftDotNetSignToolVersion>5.0.0-beta.21328.3</MicrosoftDotNetSignToolVersion>

        <MicrosoftAzureDocumentDBVersion>1.22.0</MicrosoftAzureDocumentDBVersion>

        <MicrosoftAzureCosmosDBTableVersion>1.1.2</MicrosoftAzureCosmosDBTableVersion>

        <MicrosoftAspNetCoreAllVersion>2.0.0</MicrosoftAspNetCoreAllVersion>

    @@ -74,9 +74,9 @@
  
        <MicrosoftDotNetMaestroClientVersion>1.1.0-beta.20258.6</MicrosoftDotNetMaestroClientVersion>

        <MicrosoftSourceLinkGitHubVersion>1.1.0-beta-20476-01</MicrosoftSourceLinkGitHubVersion>

        <MicrosoftSourceLinkAzureReposGitVersion>1.1.0-beta-20476-01</MicrosoftSourceLinkAzureReposGitVersion>

        <MicrosoftDotNetSwaggerGeneratorMSBuildVersion>5.0.0-beta.20478.3</MicrosoftDotNetSwaggerGeneratorMSBuildVersion>

        <MicrosoftDotNetSwaggerGeneratorMSBuildVersion>5.0.0-beta.21328.3</MicrosoftDotNetSwaggerGeneratorMSBuildVersion>

        <XliffTasksVersion>1.0.0-beta.20502.2</XliffTasksVersion>

        <MicrosoftDotNetMaestroTasksVersion>1.1.0-beta.20461.2</MicrosoftDotNetMaestroTasksVersion>

        <MicrosoftDotNetMaestroTasksVersion>1.1.0-beta.21228.1</MicrosoftDotNetMaestroTasksVersion>

        <MicrosoftDotNetXHarnessCLIVersion>1.0.0-prerelease.20505.1</MicrosoftDotNetXHarnessCLIVersion>

        <MicrosoftSymbolUploaderBuildTaskVersion>1.1.145102</MicrosoftSymbolUploaderBuildTaskVersion>

      </PropertyGroup>

eng/common/SetupNugetSources.ps1

-Original file line number
+Diff line change
@@ Expand Up @@
     function EnablePrivatePackageSources($DisabledPackageSources) {
         $maestroPrivateSources = $DisabledPackageSources.SelectNodes("add[contains(@key,'darc-int')]")
         ForEach ($DisabledPackageSource in $maestroPrivateSources) {
-            Write-Host "`tEnsuring private source '$($DisabledPackageSource.key)' is enabled"
-            $DisabledPackageSource.SetAttribute("value", "false")
+            Write-Host "`tEnsuring private source '$($DisabledPackageSource.key)' is enabled by deleting it from disabledPackageSource"
+            # Due to https://github.com/NuGet/Home/issues/10291, we must actually remove the disabled entries
+            $DisabledPackageSources.RemoveChild($DisabledPackageSource)
         }
     }
@@ Expand Down @@

eng/common/SetupNugetSources.sh

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -158,8 +158,8 @@ if [ "$?" == "0" ]; then
  
        for DisabledSourceName in ${DisabledDarcIntSources[@]} ; do

            if [[ $DisabledSourceName == darc-int* ]]

                then

                    OldDisableValue="add key=\"$DisabledSourceName\" value=\"true\""

                    NewDisableValue="add key=\"$DisabledSourceName\" value=\"false\""

                    OldDisableValue="<add key=\"$DisabledSourceName\" value=\"true\" />"

                    NewDisableValue="<!-- Reenabled for build : $DisabledSourceName -->"

                    sed -i.bak "s|$OldDisableValue|$NewDisableValue|" $ConfigFile

                    echo "Neutralized disablePackageSources entry for '$DisabledSourceName'"

            fi

eng/common/build.ps1

-Original file line number
+Diff line change
@@ Expand Up / @@ -7,7 +7,6 @@ Param( @@
       [string] $msbuildEngine = $null,
       [bool] $warnAsError = $true,
       [bool] $nodeReuse = $true,
-      [bool] $useDefaultDotnetInstall = $false,
       [switch][Alias('r')]$restore,
       [switch] $deployDeps,
       [switch][Alias('b')]$build,
@@ Expand Down Expand Up / @@ -66,7 +65,6 @@ function Print-Usage() { @@
       Write-Host "  -prepareMachine         Prepare machine for CI run, clean up processes after build"
       Write-Host "  -warnAsError <value>    Sets warnaserror msbuild parameter ('true' or 'false')"
       Write-Host "  -msbuildEngine <value>  Msbuild engine to use to run build ('dotnet', 'vs', or unspecified)."
-      Write-Host "  -useDefaultDotnetInstall <value> Use dotnet-install.* scripts from public location as opposed to from eng common folder"
       Write-Host ""
       Write-Host "Command line arguments not listed above are passed thru to msbuild."
@@ Expand Down @@

eng/common/build.sh

-Original file line number
+Diff line change
@@ Expand Up / @@ -36,8 +36,6 @@ usage() @@
       echo "  --prepareMachine         Prepare machine for CI run, clean up processes after build"
       echo "  --nodeReuse <value>      Sets nodereuse msbuild parameter ('true' or 'false')"
       echo "  --warnAsError <value>    Sets warnaserror msbuild parameter ('true' or 'false')"
-      echo "  --useDefaultDotnetInstall <value> Use dotnet-install.* scripts from public location as opposed to from eng common folder"
       echo ""
       echo "Command line arguments not listed above are passed thru to msbuild."
       echo "Arguments can also be passed in with a single hyphen."
@@ Expand Down Expand Up / @@ -80,7 +78,6 @@ prepare_machine=false @@
     verbosity='minimal'
     runtime_source_feed=''
     runtime_source_feed_key=''
-    use_default_dotnet_install=false
     properties=''
     while [[ $# > 0 ]]; do
@@ Expand Down Expand Up / @@ -159,14 +156,10 @@ while [[ $# > 0 ]]; do @@
           runtime_source_feed=$2
           shift
           ;;
-        -runtimesourcefeedkey)
+         -runtimesourcefeedkey)
           runtime_source_feed_key=$2
           shift
           ;;
-        -usedefaultdotnetinstall)
-          use_default_dotnet_install=$2
-          shift
-          ;;
         *)
           properties="$properties $1"
           ;;
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[5.0] Publish to multiple channels #7626

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!