Skip to content

Commit

Permalink
Merge pull request #29 from Azure-Samples/zheyushen/2019update3
Browse files Browse the repository at this point in the history
Revamp sample code and documentation for .NET Standard 2.0 and Linux support
  • Loading branch information
arsdragonfly authored Nov 14, 2024
2 parents 5859a7a + 11f4a7c commit 1a6c447
Show file tree
Hide file tree
Showing 62 changed files with 416 additions and 2,362 deletions.
47 changes: 41 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,51 @@ languages:
- vb
products:
- azure
description: "Sample code of Microsoft HPC Pack 2016 RTM / Update 1 / Update 2"
description: "Sample code of Microsoft HPC Pack 2019"
urlFragment: hpcpack-samples
---

# Microsoft HPC Pack 2016 Sample Code
Sample code of Microsoft HPC Pack 2016 RTM / Update 1 / Update 2
# Microsoft HPC Pack 2019 Sample Code
Sample code of Microsoft HPC Pack 2019

Please choose SDK version (_in VisualStudio or by altering packages.config_) according to your need
- Version 5.0.5931: When connecting to __Microsoft HPC Pack 2016 RTM__
- Version 5.1.6088 or later (default): When connecting to __Microsoft HPC Pack 2012__ or __Microsoft HPC Pack 2016 Update 1__ and later versions
Please choose SDK version (_in VisualStudio or by altering .csproj) according to your need:
- Version 6.3.8187-beta: preview SDK with Linux support
- Version 6.2.7756: for __Microsoft HPC Pack 2019 Update 2__
- Version 6.1.7531: for __Microsoft HPC Pack 2019 Update 1__
- Version 6.0.7121 or 6.0.7205: for __Microsoft HPC Pack 2019 RTM__

## .NET Standard 2.0 and Linux support

### Registry
The Linux SDK supports specifying configuration originally taken care of by Windows registry keys by environment variables prefixed with `CCP_CONFIG_`. For example, the `CertificateValidationType` registry key under `HKLM\SOFTWARE\Microsoft\HPC` can be specified on Linux by passing in the environment variable `CCP_CONFIG_CertificateValidationType`.

In addition, configuration can be specified via the `/etc/hpcpack/config.json` configuration file. For example, to configure `CertificateValidationType` to `1` (Skip CN check), use the following JSON config:
```json
{
"CertificateValidationType": 1
}
```

### Certificates
- On Linux, users need to manually add their certificate into the appropriate [X.509 certificate store](https://learn.microsoft.com/en-us/dotnet/standard/security/cross-platform-cryptography#x509store) corresponding to `CurrentUser\My` and `LocalMachine\Root` for Linux. For `CurrentUser\My`, user would need to import their certificate using code similar to the following:
```
using (var store = new X509Store(StoreName.My, StoreLocation.CurrentUser, OpenFlags.ReadWrite))
{
store.Add(new X509Certificate2(
"./thePathToTheCert.pfx", "passwordOfTheCert",
X509KeyStorageFlags.PersistKeySet));
}
```
For `LocalMachine\Root`, user would need to import the certificate into the default OpenSSL CA bundle using the appropriate command for your Linux distribution. See [here](https://ubuntu.com/server/docs/install-a-root-ca-certificate-in-the-trust-store) and [here](https://www.redhat.com/sysadmin/configure-ca-trust-list) for examples of how to do it on Ubuntu and RHEL.

### Logging
Logging can be configured via `appsettings.json`. See [here](https://learn.microsoft.com/en-us/dotnet/core/extensions/logging?tabs=command-line#configure-logging-without-code) for more information.

## Known issues with .NET Standard 2.0 SDK and Linux support
- When getting certificates from certificate stores on Windows and Linux, there is no UI pop-up when more than one certificate is available, resulting in no certificate being chosen and failure downstream.
- Connecting to cluster via .NET Remoting is not supported
- Entering credentials interactively is not supported. Pass username and password explicitly or use `CCP_USERNAME` and `CCP_PASSWORD` environment variables instead.
- Excel isn't supported

## Contributing
This project welcomes contributions and suggestions. Most contributions require you to
Expand Down
18 changes: 9 additions & 9 deletions Scheduler/DynamicNodeGroups/DynamicNodeGroups.sln
Original file line number Diff line number Diff line change
@@ -1,25 +1,25 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.7.34031.279
VisualStudioVersion = 17.10.35004.147
MinimumVisualStudioVersion = 10.0.40219.1
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DynamicNodeGroups", "DynamicNodeGroups\DynamicNodeGroups.csproj", "{D81F78F7-D5EC-4AAF-87FF-C2A5BCFD4148}"
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "DynamicNodeGroups", "DynamicNodeGroups\DynamicNodeGroups.csproj", "{E35EE4C4-B6A1-444B-B9FA-EDC3C2E53302}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|x64 = Debug|x64
Release|x64 = Release|x64
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{D81F78F7-D5EC-4AAF-87FF-C2A5BCFD4148}.Debug|x64.ActiveCfg = Debug|x64
{D81F78F7-D5EC-4AAF-87FF-C2A5BCFD4148}.Debug|x64.Build.0 = Debug|x64
{D81F78F7-D5EC-4AAF-87FF-C2A5BCFD4148}.Release|x64.ActiveCfg = Release|x64
{D81F78F7-D5EC-4AAF-87FF-C2A5BCFD4148}.Release|x64.Build.0 = Release|x64
{E35EE4C4-B6A1-444B-B9FA-EDC3C2E53302}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{E35EE4C4-B6A1-444B-B9FA-EDC3C2E53302}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E35EE4C4-B6A1-444B-B9FA-EDC3C2E53302}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E35EE4C4-B6A1-444B-B9FA-EDC3C2E53302}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {54E44C0A-7362-44E6-BF2C-F8D5BE520D50}
SolutionGuid = {058073F2-3899-43BB-8F70-9ACF51375A5A}
EndGlobalSection
EndGlobal
149 changes: 13 additions & 136 deletions Scheduler/DynamicNodeGroups/DynamicNodeGroups/DynamicNodeGroups.csproj
Original file line number Diff line number Diff line change
@@ -1,143 +1,20 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{D81F78F7-D5EC-4AAF-87FF-C2A5BCFD4148}</ProjectGuid>
<OutputType>Exe</OutputType>
<RootNamespace>DynamicNodeGroups</RootNamespace>
<AssemblyName>DynamicNodeGroups</AssemblyName>
<TargetFrameworkVersion>v4.7.2</TargetFrameworkVersion>
<FileAlignment>512</FileAlignment>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<Deterministic>true</Deterministic>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Debug|x64'">
<DebugSymbols>true</DebugSymbols>
<OutputPath>bin\x64\Debug\</OutputPath>
<DefineConstants>DEBUG;TRACE</DefineConstants>
<DebugType>full</DebugType>
<PlatformTarget>x64</PlatformTarget>
<LangVersion>7.3</LangVersion>
<ErrorReport>prompt</ErrorReport>
<Prefer32Bit>true</Prefer32Bit>
<PropertyGroup Condition=" '$(OS)' != 'Windows_NT' ">
<TargetFrameworks>net8.0</TargetFrameworks>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == 'Release|x64'">
<OutputPath>bin\x64\Release\</OutputPath>
<DefineConstants>TRACE</DefineConstants>
<Optimize>true</Optimize>
<DebugType>pdbonly</DebugType>
<PlatformTarget>x64</PlatformTarget>
<LangVersion>7.3</LangVersion>
<ErrorReport>prompt</ErrorReport>
<Prefer32Bit>true</Prefer32Bit>
<PropertyGroup Condition=" '$(OS)' == 'Windows_NT' ">
<TargetFrameworks>net472;net8.0</TargetFrameworks>
</PropertyGroup>

<ItemGroup>
<Reference Include="BrokerBase, Version=5.0.0.0, Culture=neutral, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\BrokerBase.dll</HintPath>
</Reference>
<Reference Include="CcpApi, Version=1.0.2.3, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\CcpApi.dll</HintPath>
</Reference>
<Reference Include="HpcCommon, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\HpcCommon.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Azure.KeyVault.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.Azure.KeyVault.Core.1.0.0\lib\net40\Microsoft.Azure.KeyVault.Core.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Data.Edm, Version=5.8.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.Data.Edm.5.8.2\lib\net40\Microsoft.Data.Edm.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Data.OData, Version=5.8.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.Data.OData.5.8.2\lib\net40\Microsoft.Data.OData.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Data.Services.Client, Version=5.8.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.Data.Services.Client.5.8.2\lib\net40\Microsoft.Data.Services.Client.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.AADAuthUtil, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.AADAuthUtil.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Azure.DataMovement, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Azure.DataMovement.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Azure.FileStaging.Client, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Azure.FileStaging.Client.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Diagnostics.Helpers, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Diagnostics.Helpers.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Excel, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Excel.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Excel.Internal, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Excel.Internal.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Scheduler, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Scheduler.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Scheduler.Properties, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Scheduler.Properties.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Scheduler.Session, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Scheduler.Session.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Hpc.Scheduler.Store, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.HPC.SDK.6.2.7756\lib\net472\Microsoft.Hpc.Scheduler.Store.dll</HintPath>
</Reference>
<Reference Include="Microsoft.IdentityModel.Clients.ActiveDirectory, Version=3.16.0.14, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.IdentityModel.Clients.ActiveDirectory.3.16.0\lib\net45\Microsoft.IdentityModel.Clients.ActiveDirectory.dll</HintPath>
</Reference>
<Reference Include="Microsoft.IdentityModel.Clients.ActiveDirectory.Platform, Version=3.16.0.14, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.IdentityModel.Clients.ActiveDirectory.3.16.0\lib\net45\Microsoft.IdentityModel.Clients.ActiveDirectory.Platform.dll</HintPath>
</Reference>
<Reference Include="Microsoft.IdentityModel.Protocol.Extensions, Version=1.0.40306.1554, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.IdentityModel.Protocol.Extensions.1.0.4.403061554\lib\net45\Microsoft.IdentityModel.Protocol.Extensions.dll</HintPath>
</Reference>
<Reference Include="Microsoft.WindowsAzure.Storage, Version=8.4.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\WindowsAzure.Storage.8.4.0\lib\net45\Microsoft.WindowsAzure.Storage.dll</HintPath>
</Reference>
<Reference Include="Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL">
<HintPath>..\packages\Newtonsoft.Json.6.0.8\lib\net45\Newtonsoft.Json.dll</HintPath>
</Reference>
<Reference Include="System" />
<Reference Include="System.Management.Automation, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>C:\Program Files (x86)\Reference Assemblies\Microsoft\WindowsPowerShell\3.0\System.Management.Automation.dll</HintPath>
</Reference>
<Reference Include="System.CodeDom, Version=7.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
<HintPath>..\packages\System.CodeDom.7.0.0\lib\net462\System.CodeDom.dll</HintPath>
</Reference>
<Reference Include="System.Core" />
<Reference Include="System.IdentityModel.Tokens.Jwt, Version=4.0.40306.1554, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\System.IdentityModel.Tokens.Jwt.4.0.4.403061554\lib\net45\System.IdentityModel.Tokens.Jwt.dll</HintPath>
</Reference>
<Reference Include="System.Management" />
<Reference Include="System.Net.Http.Formatting, Version=5.2.9.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.AspNet.WebApi.Client.5.2.9\lib\net45\System.Net.Http.Formatting.dll</HintPath>
</Reference>
<Reference Include="System.Spatial, Version=5.8.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
<HintPath>..\packages\System.Spatial.5.8.2\lib\net40\System.Spatial.dll</HintPath>
</Reference>
<Reference Include="System.ValueTuple, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51, processorArchitecture=MSIL">
<HintPath>..\packages\System.ValueTuple.4.3.0\lib\netstandard1.0\System.ValueTuple.dll</HintPath>
<Private>True</Private>
<Private>True</Private>
</Reference>
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="Microsoft.CSharp" />
<Reference Include="System.Data" />
<Reference Include="System.Net.Http" />
<Reference Include="System.Xml" />
<PackageReference Include="Microsoft.HPC.SDK" Version="6.3.8187-beta" />
</ItemGroup>
<ItemGroup>
<Compile Include="Program.cs" />
<Compile Include="Properties\AssemblyInfo.cs" />
</ItemGroup>
<ItemGroup>
<None Include="App.config" />
<None Include="packages.config" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
</Project>

</Project>
9 changes: 3 additions & 6 deletions Scheduler/DynamicNodeGroups/DynamicNodeGroups/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,9 @@
//
//Copyright (C) Microsoft Corporation. All rights reserved.

using System;
using System.Threading;
using Microsoft.Hpc.Scheduler;
using Microsoft.Hpc.Scheduler.Properties;
using System.Diagnostics;
using System.Threading.Tasks;

namespace DynamicNodeGroups
{
Expand All @@ -20,12 +17,12 @@ class Program
//for best results, run this sample code in queued scheduling mode
static async Task Main(string[] args)
{
string clusterName = Environment.GetEnvironmentVariable("CCP_SCHEDULER");
string? clusterName = Environment.GetEnvironmentVariable("CCP_SCHEDULER");
using (IScheduler scheduler = new Scheduler())
{
Console.WriteLine("Connecting to {0}", clusterName);
scheduler.Connect(clusterName);

Console.WriteLine("Connected");
//assume you have two nodegroups, NodeGroup1 and NodeGroup2
IStringCollection nodeGroup1 = scheduler.GetNodesInNodeGroup("NodeGroup1");
IStringCollection nodeGroup2 = scheduler.GetNodesInNodeGroup("NodeGroup2");
Expand Down Expand Up @@ -125,7 +122,7 @@ static async Task Main(string[] args)
}
}

static void Job_OnTaskState(object sender, TaskStateEventArg e)
static void Job_OnTaskState(object? sender, TaskStateEventArg e)
{
if (e.NewState == TaskState.Running)
{
Expand Down

This file was deleted.

18 changes: 0 additions & 18 deletions Scheduler/DynamicNodeGroups/DynamicNodeGroups/app.config

This file was deleted.

Loading

0 comments on commit 1a6c447

Please sign in to comment.