Use managed signer for bundles

[jtschuster]

Updates the single-file bundler to use the managed Mach-O signer. Adds the SymbolTable load command to the MachObjectFile class and adds a method to extend the symbol table to include the bundled data. Adds validation when creating a MachObjectFile from a file on disk.

Now that we don't use codesign for signing, it looks like the executables are sometimes getting killed by antivirus when tests try to run them immediately after they are modified, leading to flaky tests (locally at least). I'm looking into either retrying or running codesign -v before running the apps on Mac.
The issue wasn't AV, it was due to an issue with signature caching on Mac. The way to work around this is to create a new file with a new inode when modifying and resigning a file on disk. Basic host signing didn't run into this issue because it modifies and signs the executable in the same step, so no re-signing happened. For singlefile bundles, we previously would modify the apphost, close the file, then in a separate step, bundle the files and re-sign it. After this PR we create a new file every time we need to re-sign the host, then replace the old host with the new one.

Fixes #110055

[Copilot]

Copilot reviewed 5 out of 12 changed files in this pull request and generated 2 suggestions.

Files not reviewed (7)

• src/installer/managed/Microsoft.NET.HostModel/AppHost/MachOUtils.cs: Evaluated as low risk
• src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs: Evaluated as low risk
• src/installer/tests/TestUtils/TestApp.cs: Evaluated as low risk
• src/installer/managed/Microsoft.NET.HostModel/MachO/MachOFormatError.cs: Evaluated as low risk
• src/installer/tests/Microsoft.NET.HostModel.Tests/AppHost/CreateAppHost.cs: Evaluated as low risk
• src/installer/managed/Microsoft.NET.HostModel/Bundle/Bundler.cs: Evaluated as low risk
• src/installer/tests/TestUtils/SingleFileTestApp.cs: Evaluated as low risk

Comments skipped due to low confidence (2)

src/installer/managed/Microsoft.NET.HostModel/MachO/MachObjectFile.cs:22

• There is an extra semicolon at the end of the line.

uint csSize = CodeSignature.GetCodeSignatureSize(GetSignatureStart(), identifier);;

src/installer/tests/Microsoft.NET.HostModel.Tests/MachObjectSigning/SigningTests.cs:33

• The 'MemoryMappedFileAccess.CopyOnWrite' might not be necessary for a read-only check. Consider using 'MemoryMappedFileAccess.Read' instead.

using (var managedSignedAccessor = memoryMappedFile.CreateViewAccessor(0, 0, MemoryMappedFileAccess.CopyOnWrite))

[elinor-fung]

Now that we don't use codesign for signing, it looks like the executables are sometimes getting killed by antivirus when tests try to run them immediately after they are modified, leading to flaky tests (locally at least). I'm looking into either retrying or running codesign -v before running the apps on Mac.

What was the end result here? Could a product scenario hit the same thing with running immediately after publish with signing?

[jtschuster]

What was the end result here?

It looks like the issue is probably related to modifying the exe in place: https://developer.apple.com/documentation/security/updating-mac-software#Update-Files-that-Include-Signed-Code

I'll update the code to use CopyOnWrite memory-mapped files, write a new file, then replace the old one.

[elinor-fung]

What was the end result here?

It looks like the issue is probably related to modifying the exe in place: https://developer.apple.com/documentation/security/updating-mac-software#Update-Files-that-Include-Signed-Code

Ah, does it cache once the file is created? I had thought it would only cache when it is first run.

I'll update the code to use CopyOnWrite memory-mapped files, write a new file, then replace the old one.

I guess doing the inode check with codesign shows it replaces with an entirely new file? Matching that seems good.

[jtschuster]

What was the end result here?

It looks like the issue is probably related to modifying the exe in place: https://developer.apple.com/documentation/security/updating-mac-software#Update-Files-that-Include-Signed-Code

Ah, does it cache once the file is created? I had thought it would only cache when it is first run.

I'll update the code to use CopyOnWrite memory-mapped files, write a new file, then replace the old one.

I guess doing the inode check with codesign shows it replaces with an entirely new file? Matching that seems good.

It was still broken after changing the behavior to create a new inode. I had forgotten to update the VMSize field of the __LINKEDIT segment when creating the signature. The file size was larger than the VMSize, and that was causing sporadic crashes, but I only noticed it in the larger bundles with larger signatures. I left in the "copy to a temp file, edit, then replace" behavior anyway to make sure we match codesign's behavior.