Received map file

docs/mdsource/naming.source.md

@@ -347,6 +347,46 @@ TheTest.TheMethod.DotNet.verified.txt
 ```
 
 
+## Received map file
+
+The verified name cannot be reliably derived from the received name. A multi targeted project adds the runtime and version to the received name only, and ignored parameters are kept in the received name but dropped from the verified name. Code running inside the test has both paths available, but tooling that runs after the test run, for example a snapshot review or accept tool, only sees the files on disk. So, whenever a received file is left on disk, Verify records which verified file it belongs to.
+
+The record is written to the intermediate (`obj`) directory of the test project, not next to the snapshot, so it neither clutters the directory holding the code and snapshots nor is picked up by the `*.received.*` globs used to find snapshots:
+
+```
+{IntermediateDirectory}/VerifyReceived/{hash}.txt
+```
+
+Each file holds two lines, the received path and the verified path:
+
+```
+C:\code\MyProject\Tests\TheTest.TheMethod.DotNet11_0.received.txt
+C:\code\MyProject\Tests\TheTest.TheMethod.verified.txt
+```
+
+The [Verify.ExceptionParsing](https://nuget.org/packages/Verify.ExceptionParsing) NuGet package provides a reader for these:
+
+```cs
+var maps = ReceivedMaps.Read(directory);
+if (maps.TryGetVerified(receivedPath, out var verifiedPath))
+{
+    // accept by moving receivedPath to verifiedPath
+}
+```
+
+`ReceivedMaps.Pairs` enumerates every pair instead, for accepting a whole run at once.
+
+It scans the directory recursively, so it can be pointed at a project or a repository root. `.git` and `node_modules` are skipped.
+
+Notes:
+
+ * A record is only written when a received file is left on disk, so passing tests produce none, and none are written for [AutoVerify](autoverify.md).
+ * No records are written on a [build server](build-server.md), since nothing there consumes them, and the paths recorded do not apply off the agent.
+ * The file name is derived from the received path, so re running a test overwrites the same file rather than accumulating.
+ * Records outlive the received files they describe, for example once a snapshot has been accepted, or the test is fixed or deleted. The reader drops those, so only pairs that can be acted on are returned. They are removed from disk whenever `obj` is cleaned.
+ * The intermediate directory is resolved at build time via Verify's MSBuild props. A project that does not consume those props gets no records.
+
+
 ## Orphaned verified files
 
 One problem with Verify is there is currently no way to track or clean up orphaned verified files.

docs/naming.md

@@ -922,6 +922,46 @@ TheTest.TheMethod.DotNet.verified.txt
 ```
 
 
+## Received map file
+
+The verified name cannot be reliably derived from the received name. A multi targeted project adds the runtime and version to the received name only, and ignored parameters are kept in the received name but dropped from the verified name. Code running inside the test has both paths available, but tooling that runs after the test run, for example a snapshot review or accept tool, only sees the files on disk. So, whenever a received file is left on disk, Verify records which verified file it belongs to.
+
+The record is written to the intermediate (`obj`) directory of the test project, not next to the snapshot, so it neither clutters the directory holding the code and snapshots nor is picked up by the `*.received.*` globs used to find snapshots:
+
+```
+{IntermediateDirectory}/VerifyReceived/{hash}.txt
+```
+
+Each file holds two lines, the received path and the verified path:
+
+```
+C:\code\MyProject\Tests\TheTest.TheMethod.DotNet11_0.received.txt
+C:\code\MyProject\Tests\TheTest.TheMethod.verified.txt
+```
+
+The [Verify.ExceptionParsing](https://nuget.org/packages/Verify.ExceptionParsing) NuGet package provides a reader for these:
+
+```cs
+var maps = ReceivedMaps.Read(directory);
+if (maps.TryGetVerified(receivedPath, out var verifiedPath))
+{
+    // accept by moving receivedPath to verifiedPath
+}
+```
+
+`ReceivedMaps.Pairs` enumerates every pair instead, for accepting a whole run at once.
+
+It scans the directory recursively, so it can be pointed at a project or a repository root. `.git` and `node_modules` are skipped.
+
+Notes:
+
+ * A record is only written when a received file is left on disk, so passing tests produce none, and none are written for [AutoVerify](autoverify.md).
+ * No records are written on a [build server](build-server.md), since nothing there consumes them, and the paths recorded do not apply off the agent.
+ * The file name is derived from the received path, so re running a test overwrites the same file rather than accumulating.
+ * Records outlive the received files they describe, for example once a snapshot has been accepted, or the test is fixed or deleted. The reader drops those, so only pairs that can be acted on are returned. They are removed from disk whenever `obj` is cleaned.
+ * The intermediate directory is resolved at build time via Verify's MSBuild props. A project that does not consume those props gets no records.
+
+
 ## Orphaned verified files
 
 One problem with Verify is there is currently no way to track or clean up orphaned verified files.

src/StaticSettingsTests/ReceivedMapTests.cs

@@ -0,0 +1,151 @@
+// Lives here, rather than in Verify.Tests, since these toggle the build server detection, and this
+// project runs serially with BaseTest restoring that state between tests.
+public class ReceivedMapTests :
+    BaseTest
+{
+    [Fact]
+    public async Task WritesMapToIntermediateDirectory()
+    {
+        DiffEngine.BuildServerDetector.Detected = false;
+
+        using var temp = new TempDirectory();
+        var settings = new VerifySettings();
+        settings.UseDirectory(temp);
+        settings.DisableDiff();
+
+        await Assert.ThrowsAsync<VerifyException>(() => Verify("value", settings));
+
+        var received = Received(temp);
+
+        // The map lives in obj, not beside the snapshot, so it neither clutters the snapshot
+        // directory nor gets picked up by the `*.received.*` globs used to find snapshots.
+        Assert.Equal(received, Assert.Single(Directory.EnumerateFiles(temp)));
+
+        Assert.Equal(
+            Path.Combine(temp.Path, "ReceivedMapTests.WritesMapToIntermediateDirectory.verified.txt"),
+            FindMap(received));
+    }
+
+    [Fact]
+    public async Task NoMapOnBuildServer()
+    {
+        DiffEngine.BuildServerDetector.Detected = true;
+
+        using var temp = new TempDirectory();
+        var settings = new VerifySettings();
+        settings.UseDirectory(temp);
+        settings.DisableDiff();
+
+        await Assert.ThrowsAsync<VerifyException>(() => Verify("value", settings));
+
+        // A received file is still produced, but nothing on a build server consumes a map.
+        Assert.NotNull(Received(temp));
+        Assert.Null(FindMap(Received(temp)));
+    }
+
+    [Fact]
+    public async Task NoMapWhenAutoVerified()
+    {
+        DiffEngine.BuildServerDetector.Detected = false;
+
+        using var temp = new TempDirectory();
+        var settings = new VerifySettings();
+        settings.UseDirectory(temp);
+        settings.DisableDiff();
+        settings.AutoVerify();
+
+        await Verify("value", settings);
+
+        // The received file is moved to verified, so there is nothing for a map to describe. Asserted
+        // against the directory rather than a constructed received name, so that a name that never
+        // matches cannot pass this for the wrong reason.
+        Assert.DoesNotContain(
+            MapRecords(),
+            _ => _.Received.StartsWith(temp.Path, StringComparison.OrdinalIgnoreCase));
+    }
+
+    [Fact]
+    public async Task MapIsOverwrittenNotDuplicated()
+    {
+        DiffEngine.BuildServerDetector.Detected = false;
+
+        using var temp = new TempDirectory();
+
+        VerifySettings Settings()
+        {
+            var settings = new VerifySettings();
+            settings.UseDirectory(temp);
+            settings.DisableDiff();
+            settings.DisableRequireUniquePrefix();
+            return settings;
+        }
+
+        await Assert.ThrowsAsync<VerifyException>(() => Verify("value", Settings()));
+        var first = MapFiles().Count;
+
+        await Assert.ThrowsAsync<VerifyException>(() => Verify("value", Settings()));
+
+        // The map name is derived from the received path, so a re run overwrites it rather than
+        // accumulating a new file each time.
+        Assert.Equal(first, MapFiles().Count);
+    }
+
+    [Fact]
+    public async Task MapEnablesAccept()
+    {
+        DiffEngine.BuildServerDetector.Detected = false;
+
+        using var temp = new TempDirectory();
+
+        VerifySettings Settings()
+        {
+            var settings = new VerifySettings();
+            settings.UseDirectory(temp);
+            settings.DisableDiff();
+            settings.DisableRequireUniquePrefix();
+            return settings;
+        }
+
+        await Assert.ThrowsAsync<VerifyException>(() => Verify("value", Settings()));
+
+        // Accept the way out of process tooling would: read the verified path from the map, rather
+        // than trying to derive it from the received name.
+        var received = Received(temp);
+        File.Move(received, FindMap(received)!, true);
+
+        // The accept landed on the file Verify expects, so the next run passes.
+        await Verify("value", Settings());
+    }
+
+    static string MapDirectory =>
+        Path.Combine(AttributeReader.GetIntermediateDirectory(typeof(ReceivedMapTests).Assembly), "VerifyReceived");
+
+    static List<string> MapFiles() =>
+        Directory.Exists(MapDirectory) ? Directory.EnumerateFiles(MapDirectory).ToList() : [];
+
+    // Maps are named from a hash of the received path, so they are located by content rather than name.
+    static List<(string Received, string Verified)> MapRecords()
+    {
+        var records = new List<(string, string)>();
+        foreach (var file in MapFiles())
+        {
+            var lines = File.ReadAllLines(file);
+            if (lines.Length == 2)
+            {
+                records.Add((lines[0], lines[1]));
+            }
+        }
+
+        return records;
+    }
+
+    static string? FindMap(string receivedPath) =>
+        MapRecords()
+            .Where(_ => _.Received == receivedPath)
+            .Select(_ => _.Verified)
+            .SingleOrDefault();
+
+    static string Received(string directory) =>
+        Directory.EnumerateFiles(directory)
+            .Single(_ => _.EndsWith(".received.txt", StringComparison.Ordinal));
+}

src/Verify.ExceptionParsing.Tests/ReceivedMapsTests.cs

@@ -0,0 +1,153 @@
+using VerifyTests.ExceptionParsing;
+
+// Verify's build server detection is global state, and the contract test below toggles it.
+[assembly: CollectionBehavior(DisableTestParallelization = true)]
+
+public class ReceivedMapsTests
+{
+    // Pins the contract between the writer, in Verify, and the reader here. If either side changes
+    // the location or the format, this fails.
+    [Fact]
+    public async Task ReadsMapWrittenByVerify()
+    {
+        // Maps are not written on a build server, so enable the writer for this test.
+        var detected = DiffEngine.BuildServerDetector.Detected;
+        DiffEngine.BuildServerDetector.Detected = false;
+        try
+        {
+            using var temp = new TempDirectory();
+            var settings = new VerifySettings();
+            settings.UseDirectory(temp);
+            settings.DisableDiff();
+
+            await Assert.ThrowsAsync<VerifyException>(() => VerifyXunit.Verifier.Verify("value", settings));
+
+            var received = Directory.EnumerateFiles(temp)
+                .Single(_ => _.EndsWith(".received.txt", StringComparison.Ordinal));
+
+            var maps = ReceivedMaps.Read(AttributeReader.GetIntermediateDirectory());
+
+            Assert.True(maps.TryGetVerified(received, out var verified));
+
+            // The received name carries the runtime and version, since this project is multi
+            // targeted, while the verified name does not.
+            Assert.Equal(
+                Path.Combine(temp.Path, "ReceivedMapsTests.ReadsMapWrittenByVerify.verified.txt"),
+                verified);
+        }
+        finally
+        {
+            DiffEngine.BuildServerDetector.Detected = detected;
+        }
+    }
+
+    [Fact]
+    public void FindsMapsInNestedDirectories()
+    {
+        using var temp = new TempDirectory();
+        var received = Path.Combine(temp.Path, "Foo.received.txt");
+        var verified = Path.Combine(temp.Path, "Foo.verified.txt");
+        File.WriteAllText(received, "value");
+        WriteMap(Path.Combine(temp.Path, "project", "obj", "Debug", "net10.0"), received, verified);
+
+        var maps = ReceivedMaps.Read(temp);
+
+        Assert.Single(maps.Pairs);
+        Assert.True(maps.TryGetVerified(received, out var found));
+        Assert.Equal(verified, found);
+    }
+
+    [Fact]
+    public void ExcludesRecordWhenReceivedIsGone()
+    {
+        using var temp = new TempDirectory();
+        // No received file on disk, as after the snapshot was accepted, or the test fixed or deleted.
+        var received = Path.Combine(temp.Path, "Foo.received.txt");
+        WriteMap(Path.Combine(temp.Path, "obj"), received, Path.Combine(temp.Path, "Foo.verified.txt"));
+
+        var maps = ReceivedMaps.Read(temp);
+
+        // Enumerating Pairs is how a run is accepted, so a record that cannot be acted on must not
+        // appear there.
+        Assert.Empty(maps.Pairs);
+        Assert.False(maps.TryGetVerified(received, out _));
+    }
+
+    [Fact]
+    public void DeduplicatesReceivedRecordedByMoreThanOneBuild()
+    {
+        using var temp = new TempDirectory();
+        var received = Path.Combine(temp.Path, "Foo.received.txt");
+        var verified = Path.Combine(temp.Path, "Foo.verified.txt");
+        File.WriteAllText(received, "value");
+        WriteMap(Path.Combine(temp.Path, "obj", "Debug"), received, verified);
+        WriteMap(Path.Combine(temp.Path, "obj", "Release"), received, verified);
+
+        var maps = ReceivedMaps.Read(temp);
+
+        var pair = Assert.Single(maps.Pairs);
+        Assert.Equal(received, pair.Received);
+        Assert.Equal(verified, pair.Verified);
+    }
+
+    [Fact]
+    public void SkipsDirectoriesThatCannotHoldMaps()
+    {
+        using var temp = new TempDirectory();
+        var received = Path.Combine(temp.Path, "Foo.received.txt");
+        File.WriteAllText(received, "value");
+        WriteMap(Path.Combine(temp.Path, ".git"), received, Path.Combine(temp.Path, "Foo.verified.txt"));
+
+        var maps = ReceivedMaps.Read(temp);
+
+        Assert.Empty(maps.Pairs);
+    }
+
+    [Fact]
+    public void EmptyWhenDirectoryMissing()
+    {
+        var maps = ReceivedMaps.Read(Path.Combine(Path.GetTempPath(), $"missing-{Guid.NewGuid()}"));
+
+        Assert.Empty(maps.Pairs);
+        Assert.False(maps.TryGetVerified("Foo.received.txt", out _));
+    }
+
+    [Fact]
+    public void IgnoresMalformedMaps()
+    {
+        using var temp = new TempDirectory();
+        var directory = Path.Combine(temp.Path, "obj", "VerifyReceived");
+        Directory.CreateDirectory(directory);
+        File.WriteAllText(Path.Combine(directory, "empty.txt"), "");
+        File.WriteAllText(Path.Combine(directory, "single.txt"), "only-a-received-path");
+        File.WriteAllLines(Path.Combine(directory, "blank.txt"), ["", ""]);
+
+        var maps = ReceivedMaps.Read(temp);
+
+        Assert.Empty(maps.Pairs);
+    }
+
+    [Fact]
+    public void LookupNormalizesPaths()
+    {
+        using var temp = new TempDirectory();
+        // recorded with a redundant segment, then looked up with the resolved path
+        var recorded = Path.Combine(temp.Path, "sub", "..", "Foo.received.txt");
+        var resolved = Path.Combine(temp.Path, "Foo.received.txt");
+        var verified = Path.Combine(temp.Path, "Foo.verified.txt");
+        File.WriteAllText(resolved, "value");
+        WriteMap(Path.Combine(temp.Path, "obj"), recorded, verified);
+
+        var maps = ReceivedMaps.Read(temp);
+
+        Assert.True(maps.TryGetVerified(resolved, out var found));
+        Assert.Equal(verified, found);
+    }
+
+    static void WriteMap(string parent, string received, string verified)
+    {
+        var directory = Path.Combine(parent, "VerifyReceived");
+        Directory.CreateDirectory(directory);
+        File.WriteAllLines(Path.Combine(directory, "map.txt"), [received, verified]);
+    }
+}