AOT compilation fails when NoWarn spans separate lines

[heaths]

Description

The .rsp file that is generated may contain NoWarn which, as isn't uncommon, may span lines based on project configuration. For example, in a Directory.Build.props, someone may have specified:

<NoWarn>
  $(NoWarn);
  CS1234;
</NoWarn>

In this case, you end up with an error like:

Generating native code
EXEC : error : No files matching ;NU5105;CA1812; [C:\src\myproject.csproj::TargetFramework=net7.0]
  Internal.CommandLine.CommandLineException: No files matching ;NU5105;CA1812;
     at Internal.CommandLine.Helpers.AppendExpandedPaths(Dictionary`2, String, Boolean) in /_/src/coreclr/tools/Common/CommandLine/CommandLineHelpers.cs:line 57
     at ILCompiler.Program.ParseCommandLine(String[]) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 347
     at ILCompiler.Program.Run(String[]) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 427
     at ILCompiler.Program.Main(String[]) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 1150

Reproduction Steps

1. Create a simple console application.
2. Add <PublishAot>true</PublishAot> to a PropertyGroup.
3. Add the following to a PropertyGroup exactly as formatted:

   <NoWarn>
     $(NoWarn>;
     NU5105;
   </NoWarn>

4. Run dotnet publish.

Expected behavior

Publishes native image.

Actual behavior

Fails because .rsp file has line breaks that make the CLI think the NoWarn values are files e.g. partial response file:

--feature:System.Threading.Thread.EnableAutoreleasePool=false
--feature:System.Text.Encoding.EnableUnsafeUTF7Encoding=false
--stacktracedata
--scanreflection
--nowarn:"
      
      ;NU5105;CA1812;
      CA1716; 
      CA1308; 
      CA1819; 
      CA1710; 
      CA1028; 
      CA1032; 
      CA1063; 
      CA1066; 
      CA1815; 
      CA2007; 
      CA2231; 
      CA2225; 
      CA1714; 
      CA1062; 
      CA1031; 
      CA2000; 
      CA2012; ;
      MSB3245; ;IL2121;1701;1702"

Regression?

No response

Known Workarounds

Add something like this to your project or Directory.Build.targets:

  <ItemGroup>
    <_NoWarn Include="$(NoWarn)" />
  </ItemGroup>
  <PropertyGroup>
    <NoWarn>@(_NoWarn)</NoWarn>
  </PropertyGroup>

Configuration

dotnet 7.0 toolset on Windows 11 x64, compiling to net7.0

Other information

No response

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

The .rsp file that is generated may contain NoWarn which, as isn't uncommon, may span lines based on project configuration. For example, in a Directory.Build.props, someone may have specified:

<NoWarn>
  $(NoWarn);
  CS1234;
</NoWarn>

In this case, you end up with an error like:

Generating native code
EXEC : error : No files matching ;NU5105;CA1812; [C:\src\myproject.csproj::TargetFramework=net7.0]
  Internal.CommandLine.CommandLineException: No files matching ;NU5105;CA1812;
     at Internal.CommandLine.Helpers.AppendExpandedPaths(Dictionary`2, String, Boolean) in /_/src/coreclr/tools/Common/CommandLine/CommandLineHelpers.cs:line 57
     at ILCompiler.Program.ParseCommandLine(String[]) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 347
     at ILCompiler.Program.Run(String[]) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 427
     at ILCompiler.Program.Main(String[]) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 1150

Reproduction Steps

1. Create a simple console application.
2. Add <PublishAot>true</PublishAot> to a PropertyGroup.
3. Add the following to a PropertyGroup exactly as formatted:

   <NoWarn>
     $(NoWarn>;
     NU5105;
   </NoWarn>

4. Run dotnet publish.

Expected behavior

Publishes native image.

Actual behavior

Fails because .rsp file has line breaks that make the CLI think the NoWarn values are files e.g. partial response file:

--feature:System.Threading.Thread.EnableAutoreleasePool=false
--feature:System.Text.Encoding.EnableUnsafeUTF7Encoding=false
--stacktracedata
--scanreflection
--nowarn:"
      
      ;NU5105;CA1812;
      CA1716; 
      CA1308; 
      CA1819; 
      CA1710; 
      CA1028; 
      CA1032; 
      CA1063; 
      CA1066; 
      CA1815; 
      CA2007; 
      CA2231; 
      CA2225; 
      CA1714; 
      CA1062; 
      CA1031; 
      CA2000; 
      CA2012; ;
      MSB3245; ;IL2121;1701;1702"

Regression?

No response

Known Workarounds

Add something like this to your project or Directory.Build.targets:

  <ItemGroup>
    <_NoWarn Include="$(NoWarn)" />
  </ItemGroup>
  <PropertyGroup>
    <NoWarn>@(_NoWarn)</NoWarn>
  </PropertyGroup>

Configuration

dotnet 7.0 toolset on Windows 11 x64, compiling to net7.0

Other information

No response

Author:	heaths
Assignees:	-
Labels:	untriaged, area-NativeAOT-coreclr, needs-area-label
Milestone:	-

[am11]

Response file parser does not seem to support multi-line values: https://github.com/dotnet/command-line-api/blob/a045dd54a4c44723c215d992288160eb1401bb7f/src/System.CommandLine/Parsing/StringExtensions.cs#L372, so we should either make sure that the producer does not generate multi-line values or add support for multi-line values in response file parser (cc @adamsitnik @jonsequitur).

[adamsitnik]

Response file parser does not seem to support multi-line values

The default one does not, but it can be customized. Moreover, ILCompiler already provides it's own implementation:

runtime/src/coreclr/tools/aot/ILCompiler/Program.cs

Line 732 in a11628c

ResponseFileTokenReplacer = Helpers.TryReadResponseFile,

runtime/src/coreclr/tools/Common/CommandLineHelpers.cs

Lines 386 to 414 in 519d681

	public static bool TryReadResponseFile(string filePath, out IReadOnlyList<string> newTokens, out string error)
	{
	try
	{
	var tokens = new List<string>();
	foreach (string line in File.ReadAllLines(filePath))
	{
	string token = line.Trim();
	if (token.Length > 0 && token[0] != '#')
	{
	if (token.EndsWith('"'))
	{
	int firstQuotePosition = token.IndexOf('"');
	// strip leading and trailing quotes from value.
	if (firstQuotePosition >= 0 && firstQuotePosition < token.Length - 1 &&
	(firstQuotePosition == 0 || token[firstQuotePosition - 1] != '\\'))
	{
	token = token[..firstQuotePosition] + token[(firstQuotePosition + 1)..^1];
	}
	}
	tokens.Add(token);
	}
	}
	newTokens = tokens;
	error = null;
	return true;

[heaths]

Not sure if you're suggesting that for use instead, but my scenario was merely dotnet publish with no way I'm aware to choose a different parser.

[am11]

We should definitely fix it (either when writing the rsp or while consuming it). Meanwhile, your workaround is the way forward. I'll work on the fix soon. Thanks for the report!

@adamsitnik, thanks for reminding me. I completely forgot about our own implementation of rsp parsing. 😅

[MichalStrehovsky]

This was fixed in #92010 and backported to 8.0 (I think RC2 will have it).