Migrate file I/O tasks to multithreadable task API #12914

JanProvaznik · 2025-12-10T16:21:57Z

fixes #12484
This PR migrates several file I/O tasks to use the new multithreadable task API by implementing the \IMultiThreadableTask\ interface and using \TaskEnvironment\ for thread-safe path resolution. The path absolutization is wrapped in try-catches to align error reporting with what exists in the tasks. (absolutization may throw on windows on net framework with invalid characters)

Changes

The following tasks now implement \IMultiThreadableTask\ and use \TaskEnvironment.GetAbsolutePath()\ instead of \Path.GetFullPath():

Copy.cs
Delete.cs
MakeDir.cs
RemoveDir.cs
Touch.cs

Why

\Path.GetFullPath()\ uses process-global state (current working directory) which is not thread-safe when multiple tasks run in parallel. The \TaskEnvironment.GetAbsolutePath()\ method provides thread-safe path resolution by using the task's execution context.

This enables these tasks to be safely executed in parallel when MSBuild's multithreading is enabled.

…n status Co-authored-by: JanProvaznik <[email protected]>

…s section Co-authored-by: JanProvaznik <[email protected]>

- Updated FileState.cs to accept TaskEnvironment parameter - Migrated 7 File I/O tasks to IMultiThreadableTask: * Copy - added attribute, interface, TaskEnvironment property and updated path/environment access * Delete - migrated with proper path resolution * MakeDir - migrated with path deduplication * RemoveDir - migrated with directory operations * Touch - migrated with file timestamp operations * ReadLinesFromFile - migrated for file reading * WriteLinesToFile - migrated for file writing - All tasks use MSBuildMultiThreadableTaskAttribute for routing - All tasks implement IMultiThreadableTask for TaskEnvironment access - Tasks use TaskEnvironment.GetAbsolutePath() for thread-safe path resolution - Tasks use TaskEnvironment.GetEnvironmentVariable() for thread-safe environment access Co-authored-by: JanProvaznik <[email protected]>

Copilot

Pull request overview

This PR converts several file I/O tasks to support multi-threaded execution by implementing IMultiThreadableTask and using TaskEnvironment for thread-safe path resolution. The changes replace static path resolution methods with instance-based TaskEnvironment.GetAbsolutePath() calls to ensure proper path handling in concurrent scenarios.

Key Changes:

Added IMultiThreadableTask interface and [MSBuildMultiThreadableTask] attribute to 7 file I/O task classes
Introduced TaskEnvironment property to all modified tasks for thread-safe operations
Modified FileState constructor to require TaskEnvironment parameter and use it for path resolution

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/Tasks/Touch.cs	Added multithreading support; uses TaskEnvironment for path resolution in TouchFile method
src/Tasks/RemoveDir.cs	Added multithreading support; resolves paths using TaskEnvironment in Execute and RemoveDirectory methods
src/Tasks/MakeDir.cs	Added multithreading support; uses TaskEnvironment for path resolution and deduplication
src/Tasks/FileState.cs	Modified constructor to require TaskEnvironment parameter; uses it for absolute path resolution in Lazy initialization
src/Tasks/FileIO/WriteLinesToFile.cs	Added multithreading support; uses TaskEnvironment for path resolution instead of FileUtilities.NormalizePath
src/Tasks/FileIO/ReadLinesFromFile.cs	Added multithreading support; resolves file paths using TaskEnvironment before file operations
src/Tasks/Delete.cs	Added multithreading support; uses TaskEnvironment for path resolution in delete operations
src/Tasks/Copy.cs	Added multithreading support; uses TaskEnvironment for path resolution, environment variables, and FileState construction; changed PathsAreIdentical from static to instance method

src/Tasks/Copy.cs

src/Tasks/FileState.cs

src/Tasks/FileIO/WriteLinesToFile.cs

src/Tasks/RemoveDir.cs

src/Tasks/Copy.cs

AR-May

LGTM, but I would like one more review to ensure there is no behavior change in multi process mode.

src/Tasks/MakeDir.cs

src/Tasks/Copy.cs

src/Tasks/FileState.cs

src/Tasks/Delete.cs

src/Tasks/Touch.cs

src/Tasks/Copy.cs

JanProvaznik · 2026-01-08T13:26:48Z

added defensiveness: all TaskEnvironment.GetAbsolutePaths happen in trycatches and the errors are logged in line with the specific task's logging

JanProvaznik · 2026-01-08T16:10:42Z

exp insertion for validation passed

src/Tasks/Copy.cs

src/Framework/PathHelpers/AbsolutePath.cs

src/Tasks/Copy.cs

AR-May · 2026-01-09T10:36:09Z

src/Tasks/Copy.cs

-                            MSBuildEventSource.Log.CopyUpToDateStart(destItem.ItemSpec);
+                            MSBuildEventSource.Log.CopyUpToDateStart(destAbsolutePath);
                            bool copyComplete = partitionIndex > 0 &&
-                                                String.Equals(


It looks like some perf optimization for repeated file copy requests. Can't we save some perf and use original file paths for it? Casting to AbsolutePath seems not needed.

src/Tasks/Copy.cs

AR-May · 2026-01-09T13:17:34Z

src/Tasks/FileIO/WriteLinesToFile.cs

+        /// </summary>
+        /// <param name="filePath">The resolved absolute path, if successful.</param>
+        /// <returns>True if the path was resolved successfully; false if an error occurred.</returns>
+        private bool TryGetAbsoluteFilePath(out AbsolutePath filePath)


It looks that we often want to catch exceptions when absolutizing the paths. Should we maybe consider making it the part of the API?

I think it's a good high pri idea and also an answer to Yuliia's comment about repetition. I think it should be factored out of the PR not to bloat it more.
We didn't anticipate this need because we found out too late that .NET framework throws when you concatenate paths...

YuliiaKovalova · 2026-01-09T13:29:30Z

src/Tasks/RemoveDir.cs

+        /// <param name="path">The path to resolve.</param>
+        /// <param name="absolutePath">The resolved absolute path, if successful.</param>
+        /// <returns>True if the path was resolved successfully; false if an error occurred.</returns>
+        private bool TryGetAbsolutePath(string path, out AbsolutePath absolutePath)


you repeat it in other tasks, can we avoid it?

the methods look very similar but are subtly different (different error messages with variant number of params, adding lockcheck message, "fixing" the file path. I think we should provide a nicer way as part of the API as a high pri item I can get started on in which we then use the new API, I think it should be factored out of this PR.

I suggested to consider making it part of the TaskEnvironment API. Then when it returns false you can write whatever message needed. Since we find it useful in a lot of our tasks, it might be useful for customers as well.

I suggest making a helper function that does it for our repo tasks, and we can add the API later. As I mentioned you can check for the return value to be false to write different messages, so there is possibility to deduplicate this code somewhat.

AR-May · 2026-01-09T13:33:47Z

src/Tasks/Copy.cs

            if (destinationFileState.DirectoryExists)
            {
-                Log.LogErrorWithCodeFromResources("Copy.DestinationIsDirectory", sourceFileState.Name, destinationFileState.Name);
+                Log.LogErrorWithCodeFromResources("Copy.DestinationIsDirectory", sourceFileState.Name.Original, destinationFileState.Name);


Suggested change

Log.LogErrorWithCodeFromResources("Copy.DestinationIsDirectory", sourceFileState.Name.Original, destinationFileState.Name);

Log.LogErrorWithCodeFromResources("Copy.DestinationIsDirectory", sourceFileState.Name.Original, destinationFileState.Name).Original;

AR-May · 2026-01-09T13:39:25Z

src/Tasks/FileState.cs

-        /// Not normalized.
        /// </summary>
-        internal string Name => _filename;
+        internal AbsolutePath Name => _filename;


I think we need to review all usages of the Name property after this change. Depending on the scenario, the correct value is either Original or the absolutized path. There are already cases where the new absolutized value is being used incorrectly instead of the original path.

baronfel · 2026-01-09T15:02:36Z

src/Tasks.UnitTests/Copy_Tests.cs

-            var task = new Copy { BuildEngine = new MockEngine(true), };
+            var task = new Copy
+            {
+                TaskEnvironment = TaskEnvironmentHelper.CreateForTest(),


nit: I see that we have to add TaskEnvironment everywhere now - for testing, maybe it makes sense to have a factory function like T Test.NewTask<T>(IBuildEngine engine) where T: Task where we can set the TaskEnvironment to the test-specific one by default? And then for specific tests where we want to test the 'live' logic or w/e you'd still set it manually.

Just thinking about maintainability + patterns that might be easier for the LLMs to follow, etc.

Copilot AI and others added 6 commits December 3, 2025 16:40

Initial plan

4bf3189

Add comprehensive spec document for threading fixes and task migratio…

101bae4

…n status Co-authored-by: JanProvaznik <[email protected]>

Update spec with race condition fix details and concurrent collection…

08bd63a

…s section Co-authored-by: JanProvaznik <[email protected]>

remove document

40f9447

taskenvironment can't be null

7f3f946

Copilot AI review requested due to automatic review settings December 10, 2025 16:21

Copilot started reviewing on behalf of JanProvaznik December 10, 2025 16:22 View session

JanProvaznik changed the title ~~Remove null-conditional operators for TaskEnvironment in file I/O tasks~~ Migrate file I/O tasks to multithreadable task API Dec 10, 2025

JanProvaznik requested a review from AR-May December 10, 2025 16:25

Copilot AI reviewed Dec 10, 2025

View reviewed changes

JanProvaznik added 3 commits December 10, 2025 17:34

Address PR review comments: fix redundant path resolution and comments

efbdd60

Use AbsolutePath struct for type safety in file I/O tasks

9cce81f

refactoring

def225d

AR-May added the Area: Multithreaded label Dec 12, 2025

AR-May self-assigned this Dec 12, 2025

JanProvaznik added 4 commits December 16, 2025 13:09

update tests with taskenvironment

140104f

inject taskenv to all tests

1a6c0bc

fix makedir for invalid directories

7114c55

Merge branch 'main' into copilot/outline-threading-fixes-logic-updates

5fd56aa

AR-May approved these changes Dec 19, 2025

View reviewed changes

src/Tasks/MakeDir.cs Show resolved Hide resolved

src/Tasks/Copy.cs Outdated Show resolved Hide resolved

src/Tasks/FileState.cs Outdated Show resolved Hide resolved

JanProvaznik added 2 commits December 19, 2025 13:05

makedir fix optimization

7dc368c

filestate

e2b230b

JanProvaznik requested review from YuliiaKovalova and rainersigwald December 19, 2025 14:27

YuliiaKovalova reviewed Jan 6, 2026

View reviewed changes

src/Tasks/FileState.cs Outdated Show resolved Hide resolved

src/Tasks/Delete.cs Show resolved Hide resolved

src/Tasks/Touch.cs Outdated Show resolved Hide resolved

YuliiaKovalova reviewed Jan 6, 2026

View reviewed changes

src/Tasks/Copy.cs Outdated Show resolved Hide resolved

JanProvaznik added 3 commits January 7, 2026 12:34

pr feedback 1 - filestate, delete caching

a4a499d

revert wrong change

65f0d01

fix test

7f61524

JanProvaznik added 8 commits January 7, 2026 13:13

simplify filestate

a207afe

absolutize logging

e58fe21

absolutize logging + fix bug Copy

8b3575a

fix compile errors

b708104

original property in AbsolutePath

95febbb

use consistently abstraction

68da521

fix test

31b18c0

catch exception during normalization and log them with appropriate codes

3169478

JanProvaznik added 2 commits January 8, 2026 14:46

nullable value types work this way...

155bc02

Merge branch 'main' into copilot/outline-threading-fixes-logic-updates

9aac1ad

JanProvaznik requested a review from YuliiaKovalova January 8, 2026 15:01

JanProvaznik mentioned this pull request Jan 8, 2026

Should File IO tasks report in logs absolute paths of items they operate on? #12990

Open

JanProvaznik self-assigned this Jan 9, 2026

YuliiaKovalova reviewed Jan 9, 2026

View reviewed changes

src/Tasks/Copy.cs Outdated Show resolved Hide resolved

AR-May reviewed Jan 9, 2026

View reviewed changes

YuliiaKovalova reviewed Jan 9, 2026

View reviewed changes

AR-May reviewed Jan 9, 2026

View reviewed changes

YuliiaKovalova approved these changes Jan 9, 2026

View reviewed changes

rename Original to OriginalValue

d3d459b

baronfel reviewed Jan 9, 2026

View reviewed changes

JanProvaznik added 5 commits January 9, 2026 16:13

clarify naming in helper class

330dcee

nit: indentation

cc674f3

restore relative path tracing

827ea11

more copy consistency

78ad13e

more consistency

ede58b9

	Log.LogErrorWithCodeFromResources("Copy.DestinationIsDirectory", sourceFileState.Name.Original, destinationFileState.Name);
	Log.LogErrorWithCodeFromResources("Copy.DestinationIsDirectory", sourceFileState.Name.Original, destinationFileState.Name).Original;

Migrate file I/O tasks to multithreadable task API #12914

Are you sure you want to change the base?

Migrate file I/O tasks to multithreadable task API #12914

Conversation

JanProvaznik commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AR-May left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JanProvaznik commented Jan 8, 2026

Uh oh!

JanProvaznik commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YuliiaKovalova Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JanProvaznik commented Dec 10, 2025 •

edited

Loading

JanProvaznik commented Jan 8, 2026 •

edited

Loading

YuliiaKovalova Jan 9, 2026 •

edited

Loading