Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a managed Git implementation #521

Merged
merged 125 commits into from
Nov 24, 2020
Merged
Show file tree
Hide file tree
Changes from 123 commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
1bb16df
Add Benchmark project
qmfrederik Oct 18, 2020
9e537e9
Target netstandard2.1/netcoreapp3.1 to get native Span<T> support
qmfrederik Oct 18, 2020
9de6fb4
Make VersionOracle an abstract base class
qmfrederik Oct 18, 2020
fe0ba90
Bulk-import managed Git implementation
qmfrederik Oct 18, 2020
69f2673
Prototype ManagedVersionOracle
qmfrederik Oct 18, 2020
cfb093a
Compat fixes
qmfrederik Oct 18, 2020
7bf2f56
Actually calculate version height
qmfrederik Oct 18, 2020
aa8af44
Add debug statements
qmfrederik Oct 19, 2020
b6f0d39
Fix version calculation in scenarios where the first commit contains …
qmfrederik Oct 19, 2020
64f6c45
GitCommit: Parse author information
qmfrederik Oct 19, 2020
168ce42
Fix buffer handling
qmfrederik Oct 19, 2020
eff6450
WorkingDirectory is the top-level directory of the checkout out repos…
qmfrederik Oct 19, 2020
b202975
Fix an issue where a version change in the second commit in the repos…
qmfrederik Oct 19, 2020
63b3b0f
Properly handle scenarios where there's:
qmfrederik Oct 19, 2020
0ef14c2
Provide access HEAD as a reference
qmfrederik Oct 19, 2020
8ee8260
GitObjectId: Add unit tests
qmfrederik Oct 19, 2020
82f4317
GitCommit: Add unit tests
qmfrederik Oct 19, 2020
fc71605
GitCommitReader: Add unit tests
qmfrederik Oct 19, 2020
eae9d27
GitTreeStreamingReader: Add unit tests.
qmfrederik Oct 19, 2020
3b461e2
GitPackIndexReader: Add unit tests
qmfrederik Oct 20, 2020
083a994
GitObjectStream: Add unit tests, split off ZLibStream base class
qmfrederik Oct 20, 2020
a9c7f0b
DeltaStreamReader: Add unit tests
qmfrederik Oct 20, 2020
4e96bb0
DeltaStreamreaderTests: fixup
qmfrederik Oct 20, 2020
35e67a4
GitPack: Add unit tests
qmfrederik Oct 20, 2020
9144c2e
GitPackDeltafiedStream: Add unit tests
qmfrederik Oct 21, 2020
8cd0bae
Don't store commits as .txt files to avoid line ending differences
qmfrederik Oct 21, 2020
b3ede9a
GitRepository: Add unit tests
qmfrederik Oct 21, 2020
310903e
Add Git alternate support
qmfrederik Oct 21, 2020
e84725a
Fix IndexPath and PackPath being inverted
qmfrederik Oct 21, 2020
ed246fb
Update Benchmark project
qmfrederik Oct 21, 2020
f69109c
Cover additional scenarios:
qmfrederik Oct 23, 2020
b145612
Support reading Git trees
qmfrederik Oct 23, 2020
7b41b48
Initial support for path filters
qmfrederik Oct 23, 2020
1eb35ec
Target netstandard2.0, netcoreapp2.1, netcoreapp3.1
qmfrederik Oct 26, 2020
7c0d366
Linux: Trim the trailing \0 from path
qmfrederik Oct 26, 2020
ff165f6
Use embedded resources for unit test assets
qmfrederik Oct 26, 2020
52446d5
Relax unit test
qmfrederik Oct 26, 2020
6ee7080
Embed System.Text.Json in NuGet package
qmfrederik Oct 26, 2020
89984ff
Add System.Runtime.CompilerServices.Unsafe.dll
qmfrederik Oct 26, 2020
8e39558
Nullable annotate managed Git API
qmfrederik Oct 27, 2020
0c34296
Span -> ReadOnlySpan
qmfrederik Oct 27, 2020
6948e89
Unicode -> Utf16
qmfrederik Oct 27, 2020
236dd40
Add perf test job
qmfrederik Oct 27, 2020
745d0d5
Simplify GitObjectId use of fixed arrays
AArnott Oct 27, 2020
103bce7
Update filter
qmfrederik Oct 27, 2020
8328e84
Make net461 conditional
qmfrederik Oct 27, 2020
552155d
Update benchmark project
qmfrederik Oct 27, 2020
0608e45
Fix path
qmfrederik Oct 27, 2020
29bc0f4
Add NotNullWhen attributes to TryOpen methods
AArnott Oct 27, 2020
dd7e2ed
Use PowerShell to unpack objects
qmfrederik Oct 27, 2020
96a0a34
Try with bash
qmfrederik Oct 27, 2020
aae92be
Build everything in release mode before running benchmarks
qmfrederik Oct 27, 2020
f8257be
Attempt to get benchmarks building and running correctly
qmfrederik Oct 27, 2020
f83df9e
Avoid empty paths
qmfrederik Oct 27, 2020
d3e27c1
Properly detect benchmarks running on Windows
qmfrederik Oct 27, 2020
006c997
Upload benchmark artifacts
qmfrederik Oct 27, 2020
8ed7f57
Fix artifacts
qmfrederik Oct 27, 2020
ad64eb8
Simplify array rental
AArnott Oct 27, 2020
2e0be45
Fix `StreamExtensions` to not malfunction when `Stream.Read` returns …
AArnott Oct 27, 2020
9da12fe
Avoid initializing locals unnecessarily
AArnott Oct 27, 2020
297206e
Use `char` instead of `byte` when dealing with UTF-16 strings
AArnott Oct 27, 2020
f12d5ae
Fix nullable ref annotation warning
AArnott Oct 27, 2020
646e5fd
Generate Cobertura code coverage
qmfrederik Oct 27, 2020
e3bd782
Skip test run attachments
qmfrederik Oct 27, 2020
21522ea
Bump SourceLink to 1.0.0
qmfrederik Oct 27, 2020
4f31f47
Use PInvoke.Kernel32
qmfrederik Oct 28, 2020
c470b2c
Only read author info for head commit
qmfrederik Oct 28, 2020
02a70f1
Always read the entire commit
qmfrederik Oct 28, 2020
23a3f83
Use memory-mapped files to read from the git pack index
qmfrederik Oct 29, 2020
2b3baa3
GitPack: Use a single stream to open all objects (in read-only mode).
qmfrederik Oct 29, 2020
040aa4e
Workaround issue in unit tests
qmfrederik Oct 29, 2020
35edb99
DeltaStreamReader: Add ReadOnlyMemory<byte>-based overloads
qmfrederik Oct 29, 2020
ae28491
Search pack files before trying loose objects.
qmfrederik Oct 29, 2020
0c5a573
Use built-in MemoryMappedViewStream
qmfrederik Oct 29, 2020
313b899
Avoid allocating a List<GitCommitId> for parents in most cases
AArnott Oct 31, 2020
ec6482f
Re-route calls GitExtensions.GetVersionHeight to VersionOracle
qmfrederik Oct 27, 2020
9e9d165
Reroute GitExtensions.GetIdAsVersion calls to VersionOracle
qmfrederik Oct 27, 2020
31fb870
Remove test for bare repositories
qmfrederik Oct 27, 2020
469eb1d
Fixes after rebase
qmfrederik Oct 31, 2020
300c2de
Merge branch 'master' into features/managed-git
qmfrederik Oct 31, 2020
61e877a
Revert "Use built-in MemoryMappedViewStream"
qmfrederik Oct 31, 2020
20f4055
Update GetVersionHeight to use the managed version oracle.
qmfrederik Nov 2, 2020
36f2496
Better suport for inheritance
qmfrederik Nov 2, 2020
180489b
Update the managed implementation to closely mimic the LibGit2 implem…
qmfrederik Nov 2, 2020
f53841a
Merge branch 'master'
qmfrederik Nov 2, 2020
8e400c1
Syntax fix
qmfrederik Nov 2, 2020
3b403df
PR touch-ups
AArnott Nov 7, 2020
5a9e10a
Fix namespace casing
AArnott Nov 8, 2020
150611d
Make it clearer when product or test code uses libgit2 vs. managed
AArnott Nov 8, 2020
486103e
Stop packing benchmarks
AArnott Nov 8, 2020
b32589b
Split out managed git implementation from version computation
AArnott Nov 8, 2020
c4126df
Remove dead code
AArnott Nov 8, 2020
2192ff9
Isolate Managed from LibGit2 implementations
AArnott Nov 8, 2020
3608036
Fix RootNamespace
AArnott Nov 8, 2020
78f27aa
Support for packfiles larger than 2 GB
qmfrederik Nov 13, 2020
2ae0b8f
Add abstraction around git engine
AArnott Nov 18, 2020
96ca53a
Merge remote-tracking branch 'refs/remotes/qmfrederik/features/manage…
AArnott Nov 18, 2020
a462fb8
Finished fixing the build breaks
AArnott Nov 19, 2020
39d677f
Got a few more tests passing
AArnott Nov 19, 2020
051ba0f
More fixes
AArnott Nov 19, 2020
8eaf3bb
All libgit2 tests now pass
AArnott Nov 21, 2020
aca448c
Fix managed tests
AArnott Nov 21, 2020
168e9d0
Fix compiler warnings
AArnott Nov 21, 2020
4c4a311
Fix build integration test failures when they run concurrently
AArnott Nov 21, 2020
fb1a76b
Remove ps1 scripts
AArnott Nov 21, 2020
1f1ca7e
Implement `ManagedGitContext.GetShortUniqueCommitId`
AArnott Nov 21, 2020
052735a
Add shallow clone detection
AArnott Nov 21, 2020
18af50e
Add shallow clone recognition with helpful error message
AArnott Nov 21, 2020
d70601e
Add environment variable to force `nbgv` to use LibGit2
AArnott Nov 21, 2020
9e488a5
Add tests for selecting commits by refs
AArnott Nov 21, 2020
d7ec20b
Add uppercase hex test and fix it
AArnott Nov 21, 2020
df90653
Fix up ref matching
AArnott Nov 21, 2020
209e53e
Resolve remote refs too
AArnott Nov 21, 2020
366d2c2
Add partial commit id from pack test
AArnott Nov 22, 2020
38d5df3
Remove unused GitPackIndexStreamReader class.
qmfrederik Nov 23, 2020
7685c32
GitRepository: Implement Lookup() for partial Git object IDs
qmfrederik Nov 23, 2020
7c08676
Add .NET Core 5.0 to benchmark
qmfrederik Nov 23, 2020
74e617b
Add short commit id test
AArnott Nov 23, 2020
14822fd
GitRepository: Implement ShortenObjectId
qmfrederik Nov 23, 2020
912e129
Merge remote-tracking branch 'refs/remotes/qmfrederik/features/manage…
AArnott Nov 23, 2020
1074e57
Add tests of various lengths of partial commit matching
AArnott Nov 23, 2020
b5f1fc5
Test for odd length commit lookups
AArnott Nov 23, 2020
77fe64b
Fix handling of odd-length partial commit IDs
AArnott Nov 24, 2020
de13412
Do more with .NET 5
AArnott Nov 24, 2020
670cbe6
Run functional tests in parallel with perf analysis
AArnott Nov 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 74 additions & 2 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ stages:
--filter "TestCategory!=FailsOnAzurePipelines"
--logger "trx;LogFileName=$(Build.ArtifactStagingDirectory)/TestLogs/TestResults.trx"
--results-directory $(Build.ArtifactStagingDirectory)/CodeCoverage/
--collect:"XPlat Code Coverage"
--
--collect:"XPlat Code Coverage"
--
RunConfiguration.DisableAppDomain=true
displayName: Run tests
workingDirectory: src
Expand Down Expand Up @@ -196,6 +196,78 @@ stages:
displayName: npm publish to PublicCI feed
condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'), eq(variables['System.TeamFoundationCollectionUri'], 'https://dev.azure.com/andrewarnott/'), ne(variables['Build.Reason'], 'PullRequest'))

- job: PerfTest
strategy:
matrix:
ubuntu:
imageName: ubuntu-18.04
repoDir: '~/git'
windows:
imageName: windows-2019
repoDir: '${USERPROFILE}/source/repos'
macOS:
imageName: macOS-10.15
repoDir: '~/git'
pool:
vmImage: $(imageName)
steps:
- task: UseDotNet@2
displayName: Install .NET Core SDK 2.1.811
inputs:
packageType: sdk
version: 2.1.811
- task: UseDotNet@2
displayName: Install .NET Core SDK 3.1.100
inputs:
packageType: sdk
version: 3.1.100
- task: UseDotNet@2
displayName: Install .NET Core SDK 5.0.100
inputs:
packageType: sdk
version: 5.0.100
- script: dotnet --info
displayName: Show dotnet SDK info
- bash: |
mkdir -p $(repoDir)
git clone https://github.com/xunit/xunit $(repoDir)/xunit
git clone https://github.com/gimlichael/Cuemon $(repoDir)/Cuemon
git clone https://github.com/kerryjiang/SuperSocket $(repoDir)/SuperSocket
git clone https://github.com/dotnet/NerdBank.GitVersioning $(repoDir)/NerdBank.GitVersioning
displayName: Clone test repositories
- script: |
dotnet build -c Release
workingDirectory: src/
displayName: Build in Release mode
- script: |
dotnet run -c Release -f netcoreapp3.1 -- --filter GetVersionBenchmarks --artifacts $(Build.ArtifactStagingDirectory)/benchmarks/packed/$(imageName)
workingDirectory: src/NerdBank.GitVersioning.Benchmarks
displayName: Run benchmarks (packed)
- bash: |
cd $(repoDir)/xunit
git unpack-objects < .git/objects/pack/*.pack

cd $(repoDir)/Cuemon
git unpack-objects < .git/objects/pack/*.pack

cd $(repoDir)/SuperSocket
git unpack-objects < .git/objects/pack/*.pack

cd $(repoDir)/NerdBank.GitVersioning
git unpack-objects < .git/objects/pack/*.pack
displayName: Unpack Git repositories
- script: |
dotnet run -c Release -f netcoreapp3.1 -- --filter GetVersionBenchmarks --artifacts $(Build.ArtifactStagingDirectory)/benchmarks/unpacked/$(imageName)
workingDirectory: src/NerdBank.GitVersioning.Benchmarks
displayName: Run benchmarks (unpacked)
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: $(Build.ArtifactStagingDirectory)/benchmarks
ArtifactName: benchmarks
ArtifactType: Container
displayName: Publish benchmarks artifacts
condition: succeededOrFailed()

- stage: Test
jobs:
- job: linux
Expand Down
7 changes: 4 additions & 3 deletions doc/nuget-acquisition.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ If in a project that uses PackageReference for this package reference, you shoul
After installing this NuGet package, you may need to configure the version generation logic
in order for it to work properly.

When using packages.config, the configuration is handled automatically via the tools\Install.ps1 script.
When using project.json or PackageReference, you can run the script tools\Create-VersionFile.ps1 to help
you create the version.json file and remove the old assembly attributes.
We recommend installing the `nbgv` tool using `dotnet tool install -g nbgv`.
Then use `nbgv install` to add the package reference and `version.json` file to your repo.
But you can simply add the package reference yourself, and create the `version.json` in your repo
with content conforming to [this doc](versionJson.md).

The scripts will look for the presence of a version.json or version.txt file.
If one already exists, nothing happens. If the version file does not exist,
Expand Down
30 changes: 4 additions & 26 deletions src/Cake.GitVersioning/GitVersioningAliases.cs
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ public static class GitVersioningAliases
/// Information(GetVersioningGetVersion().SemVer2)
/// });
/// </example>
/// <param name="context">The context.</param>
/// <param name="cakeContext">The context.</param>
/// <param name="projectDirectory">Directory to start the search for version.json.</param>
/// <returns>The version information from Git Versioning.</returns>
[CakeMethodAlias]
public static VersionOracle GitVersioningGetVersion(this ICakeContext context, string projectDirectory = ".")
public static VersionOracle GitVersioningGetVersion(this ICakeContext cakeContext, string projectDirectory = ".")
{
var fullProjectDirectory = (new DirectoryInfo(projectDirectory)).FullName;

Expand All @@ -42,30 +42,8 @@ public static VersionOracle GitVersioningGetVersion(this ICakeContext context, s
throw new InvalidOperationException("Could not locate the Cake.GitVersioning library");
}

// Even after adding the folder containing the native libgit2 DLL to the PATH, DllNotFoundException is still thrown
// Workaround this by copying the contents of the found folder to the current directory
GitExtensions.HelpFindLibGit2NativeBinaries(directoryName, out var attemptedDirectory);

// The HelpFindLibGit2NativeBinaries method throws if the directory does not exist
var directoryInfo = new DirectoryInfo(attemptedDirectory);

// There should only be a single file in the directory, but we do not know its extension
// So, we will just get a list of all files rather than trying to determine the correct name and extension
// If there are other files there for some reason, it should not matter as long as we don't overwrite anything in the current directory
var fileInfos = directoryInfo.GetFiles();

foreach (var fileInfo in fileInfos)
{
// Copy the file to the Cake.GitVersioning DLL directory, without overwriting anything
string destFileName = Path.Combine(directoryName, fileInfo.Name);

if (!File.Exists(destFileName))
{
File.Copy(fileInfo.FullName, destFileName);
}
}

return VersionOracle.Create(fullProjectDirectory, null, CloudBuild.Active);
var gitContext = GitContext.Create(fullProjectDirectory);
return new VersionOracle(gitContext, cloudBuild: CloudBuild.Active);
}
}
}
3 changes: 2 additions & 1 deletion src/Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<BaseIntermediateOutputPath>$(MSBuildThisFileDirectory)..\obj\$(MSBuildProjectName)\</BaseIntermediateOutputPath>
<OutputPath>$(MSBuildThisFileDirectory)..\bin\$(MSBuildProjectName)\$(Configuration)\</OutputPath>
<DocumentationRootFolder>$(MSBuildThisFileDirectory)..\wiki\api</DocumentationRootFolder>
<LangVersion>7.3</LangVersion>
<LangVersion>8.0</LangVersion>

<SignAssembly>true</SignAssembly>
<AssemblyOriginatorKeyFile>$(MSBuildThisFileDirectory)strongname.snk</AssemblyOriginatorKeyFile>
Expand All @@ -24,6 +24,7 @@
<LibGit2SharpNativeVersion>2.0.306</LibGit2SharpNativeVersion>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Net.Compilers.Toolset" Version="3.8.0-5.final" PrivateAssets="all" />
<PackageReference Include="Microsoft.NETFramework.ReferenceAssemblies" Version="1.0.0" PrivateAssets="All" />
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0" PrivateAssets="All" />
</ItemGroup>
Expand Down
70 changes: 70 additions & 0 deletions src/NerdBank.GitVersioning.Benchmarks/GetVersionBenchmarks.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
using System;
using System.IO;
using System.Runtime.InteropServices;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs;

namespace Nerdbank.GitVersioning.Benchmarks
{
[SimpleJob(RuntimeMoniker.NetCoreApp31, baseline: true)]
[SimpleJob(RuntimeMoniker.NetCoreApp21)]
[SimpleJob(RuntimeMoniker.NetCoreApp50)]
[SimpleJob(RuntimeMoniker.Net461)]
public class GetVersionBenchmarks
{
// You must manually clone these repositories:
// - On Windows, to %USERPROFILE%\Source\Repose
// - On Unix, to ~/git/
[Params(
"xunit",
"Cuemon",
"SuperSocket",
"NerdBank.GitVersioning")]
public string ProjectDirectory;

public Version Version { get; set; }

[Benchmark(Baseline = true)]
public void GetVersionLibGit2()
{
using var context = GitContext.Create(GetPath(this.ProjectDirectory), writable: true);
var oracle = new VersionOracle(context, cloudBuild: null);
this.Version = oracle.Version;
}

[Benchmark]
public void GetVersionManaged()
{
using var context = GitContext.Create(GetPath(this.ProjectDirectory), writable: false);
var oracle = new VersionOracle(context, cloudBuild: null);
this.Version = oracle.Version;
}

private static string GetPath(string repositoryName)
{
string path = null;

if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
{
path = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.UserProfile),
@"Source\Repos",
repositoryName);
}
else
{
path = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.UserProfile),
@"git",
repositoryName);
}

if (!Directory.Exists(path))
{
throw new DirectoryNotFoundException($"The directory '{path}' could not be found");
}

return path;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFrameworks>netcoreapp3.1;netcoreapp2.1</TargetFrameworks>
<TargetFrameworks Condition="'$(OS)' == 'Windows_NT'">$(TargetFrameworks);net461</TargetFrameworks>
<OutputType>Exe</OutputType>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<PlatformTarget>AnyCPU</PlatformTarget>
<IsPackable>false</IsPackable>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.12.1" />
<PackageReference Include="BenchmarkDotNet.Diagnostics.Windows" Version="0.12.0" />
<PackageReference Include="Nerdbank.GitVersioning.LKG" Version="3.1.93" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="../NerdBank.GitVersioning/NerdBank.GitVersioning.csproj" />
</ItemGroup>

</Project>
11 changes: 11 additions & 0 deletions src/NerdBank.GitVersioning.Benchmarks/Program.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
using System;
using BenchmarkDotNet.Running;

namespace Nerdbank.GitVersioning.Benchmarks
{
class Program
{
static void Main(string[] args) =>
BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
}
}
Loading