FreeBSD workflow for SourceBuild

[arrowd]

As requested by @tmds in dotnet/runtime#114285 (comment) here are how we build dotnet in FreeBSD Ports.

A port in FreeBSD terminology is a build recipe that provides a compiled binary package from a source archive called the "distfile". A port may have multiple distfiles and is subject to following restrictions:

• All the distfiles should be explicitly enumerated along with their size and checksums
• No networking is allowed during the build (this is achieved by running the build in a container env with networking shut down)

So, in case of .NET port its distfiles are:

1. A source archive for a given tag from https://github.com/dotnet/dotnet
2. release.json from the same repo
3. Bootstrap distfiles that look like Private.SourceBuilt.Artifacts.9.0.103-servicing.25071.1.freebsd-x64.tar.gz:bootstrap and dotnet-sdk-9.0.103-freebsd-x64.tar.gz.

Bootstrap distfiles are currently obtained from the build of a previous version of the same port and then uploaded to https://github.com/sec/dotnet-core-freebsd-source-build/releases/ (don't mind scripts in that repo, these are old).

The build itself is executed by running build.sh: https://github.com/freebsd/freebsd-ports/blob/5e8551be6b3625e7ae34342268638e9066b48d5b/lang/dotnet/Makefile#L112

Now the important thing is that we want to have a single bootstrap binaries for all supported FreeBSD versions. This is usually achieved by compiling on a oldest FreeBSD release available, but it didn't work for .NET out of the box, because FreeBSD contains OpenSSL in its base and different FreeBSD versions may contain different OpenSSL versions. After some digging I found out that it is possible to pass /p:PortableBuild=true to build.sh, which makes resulting binaries have the simple 'freebsd-x64` RID and dynamically dispatch calls to OpenSSL without depending on a specific library version.

So to sum this up, it'd be great to have an ability to ask the build system to produce a single bootstrap distfile, which can then be fed to the same build system to build a full version of .NET.

[tmds]

Now the important thing is that we want to have a single bootstrap binaries for all supported FreeBSD versions. This is usually achieved by compiling on a oldest FreeBSD release available, but it didn't work for .NET out of the box, because FreeBSD contains OpenSSL in its base and different FreeBSD versions may contain different OpenSSL versions. After some digging I found out that it is possible to pass /p:PortableBuild=true to build.sh, which makes resulting binaries have the simple 'freebsd-x64` RID and dynamically dispatch calls to OpenSSL without depending on a specific library version.

We bootstrap per distro version. When we target an architecture that is not supported by upstream (like ppc64le or s390x), we do build with a cross-compiler and we also like the resulting SDK to work across of distros.

For that reason, our cross-builds also pass the /p:PortableBuild=true argument.

Bootstrap distfiles are currently obtained from the build of a previous version of the same port and then uploaded to https://github.com/sec/dotnet-core-freebsd-source-build/releases/ (don't mind scripts in that repo, these are old).

There are different to go about this, and perhaps some specific FreeBSD requirements.

For our builds, we also package up the artifacts tarball in a distro package. That way, the next build can obtain both the SDK and artifacts tarball from the distro packages.

So to sum this up, it'd be great to have an ability to ask the build system to produce a single bootstrap distfile, which can then be fed to the same build system to build a full version of .NET.

Is this about passing /p:PortableBuild=true to the build? Or what does this mean?

[arrowd]

For our builds, we also package up the artifacts tarball in a distro package. That way, the next build can obtain both the SDK and artifacts tarball from the distro packages.

But these are huge and only needed for people that are doing packaging work, no?

Is this about passing /p:PortableBuild=true to the build? Or what does this mean?

It is about dotnet/runtime#114285 , but generally speaking it is about visibility. So far we had to

• figure out what files make up the bootstrap distribution
• figure out how to create bootstrap distribution out of build products
• figure out to feed bootstrap distribution to the build
• find out that /p:PortableBuild=true flag that made the bootstrap portable enough for our needs.

I'm trying to say that wrapping all of this into --bootstrap and --use-bootstrap machinery is much more packager-friendly that the current state of things.

[tmds]

But these are huge and only needed for people that are doing packaging work, no?

Yes, they are huge. Only for building .NET someone would install these.

It is about dotnet/runtime#114285

I think the title of the PR is misleading you. This is a refactoring of the runtime repo build. It is not about bootstrap as you describe it here.

generally speaking it is about visibility. So far we had to figure out

You can open issues/discussions like this one. Contributors and maintainers will chime in.

If you have some suggestions (tooling/docs/...), maintainers will also consider them.

cc @am11

[am11]

To understand precisely what dotnet/runtime#114285 means for community platform, for which there is no MSFT SDK available just like freebsd-x64 and -arm64, take a look at linux-riscv64:

https://github.com/filipnavara/dotnet-riscv/blob/937be4c41c614a389d3dcb830bac1b6c187b37af/.github/workflows/build.yml#L33-L42

Once that PR is merged, we will remove the Apply patches step which was building .NET 10 runtime in two stages (stage 1: build everything which doesn't require target apphost, stage 2: build the rest). The upcoming --bootstrap in runtime repo's build.sh wraps it under single invocation in a better, cleaner manner (it runs stage 1 as before, backups artifacts/bin as artifacts/bootstrap and runs a full, clean, build pointing to artifacts/bootstrap for prereqs).

I think your post-patch step https://github.com/freebsd/freebsd-ports/blob/5e8551be6b3625e7ae34342268638e9066b48d5b/lang/dotnet/Makefile#L107 is something which should be handled by the SDK in https://github.com/dotnet/sdk/blob/11150c0ec9020625308edeec555a8b78dbfb2aa5/src/Layout/redist/targets/GenerateBundledVersions.targets#L80 or maybe it's "just in case" type patch; not really needed?

[arrowd]

maybe it's "just in case" type patch; not really needed?

Just tried to build without it and got:

    Build FAILED.
    
    /wrkdirs/usr/ports/lang/dotnet/work/bootstrap_sdk/sdk/9.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(158,5): error NETSDK1084: There is no application host available for the specified RuntimeIdentifier 'freebsd-x64'. [/wrkdirs/usr/ports/lang/dotnet/work/dotnet-9.0.4/src/roslyn-analyzers/src/PerformanceTests/Tests/PerformanceTests.csproj]
    /wrkdirs/usr/ports/lang/dotnet/work/bootstrap_sdk/sdk/9.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(158,5): error NETSDK1084: There is no application host available for the specified RuntimeIdentifier 'freebsd-x64'. [/wrkdirs/usr/ports/lang/dotnet/work/dotnet-9.0.4/src/roslyn-analyzers/src/PerformanceTests/Tests/PerformanceTests.csproj]
    /wrkdirs/usr/ports/lang/dotnet/work/bootstrap_sdk/sdk/9.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(158,5): error NETSDK1084: There is no application host available for the specified RuntimeIdentifier 'freebsd-x64'. [/wrkdirs/usr/ports/lang/dotnet/work/dotnet-9.0.4/src/roslyn-analyzers/src/PerformanceTests/Tests/PerformanceTests.csproj]
    /wrkdirs/usr/ports/lang/dotnet/work/bootstrap_sdk/sdk/9.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(158,5): error NETSDK1084: There is no application host available for the specified RuntimeIdentifier 'freebsd-x64'. [/wrkdirs/usr/ports/lang/dotnet/work/dotnet-9.0.4/src/roslyn-analyzers/src/PerformanceTests/Tests/PerformanceTests.csproj]
        0 Warning(s)
        4 Error(s)

[am11]

/wrkdirs/usr/ports/lang/dotnet/work/bootstrap_sdk/sdk/9.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(158,5): error NETSDK1084: There is no application host available for the specified RuntimeIdentifier 'freebsd-x64'. [/wrkdirs/usr/ports/lang/dotnet/work/dotnet-9.0.4/src/roslyn-analyzers/src/PerformanceTests/Tests/PerformanceTests.csproj]

That project is net6.0 https://github.com/dotnet/roslyn-analyzers/blob/f0b5b10c8a5b33a80408d73c2e3423b53acbc9f4/src/PerformanceTests/Tests/PerformanceTests.csproj#L3 and freebsd-x64 RID was added in net8.0 https://github.com/dotnet/sdk/blob/11150c0ec9020625308edeec555a8b78dbfb2aa5/src/Layout/redist/targets/GenerateBundledVersions.targets#L216.

I think it's best to update the outdated projects to net8.0 since net6.0 is anyway out of support at this point. Probably you are in a best position to patch that project sed -i'' 's/TargetFramework>net6/TargetFramework>net8/g' src/roslyn-analyzers/src/PerformanceTests/Tests/PerformanceTests.csproj and continue building to find out if there are other outliers.

Once you know the full scope, those patches could be upstreamed as "Update net6.0 projects to net8.0 (current LTS)". I assume there won't be any objection doing so for net10.0. cc @ViktorHofer

[ViktorHofer]

Just FYI we will be moving the remaining NetAnalyzer from roslyn-analyzer probably to dotnet/sdk and then archive the repo. I expect that as part of that, we will update any stale TFMs.

[am11]

Sounds good. My point was if they continue the source build by patching these project and make a list, we will get full visibility into everything that needs updating (roslyn-analyzer was probably the first one with old TFM where build stopped; it might not be the last).

Right now, we don't see it because linux is supported since .NET Core 1.0 and the hack (to replace all linux- with freebsd-) is papering over the gaps. I am assuming we want to update all projects to .NET 8 or above which participate in source-build.

[arrowd]

This single place was enough. I've submitted a PR to roslyn-analyzers.

[mthalman]

For our builds, we also package up the artifacts tarball in a distro package. That way, the next build can obtain both the SDK and artifacts tarball from the distro packages.

But these are huge and only needed for people that are doing packaging work, no?

Is this about passing /p:PortableBuild=true to the build? Or what does this mean?

It is about dotnet/runtime#114285 , but generally speaking it is about visibility. So far we had to

• figure out what files make up the bootstrap distribution
• figure out how to create bootstrap distribution out of build products
• figure out to feed bootstrap distribution to the build
• find out that /p:PortableBuild=true flag that made the bootstrap portable enough for our needs.

I'm trying to say that wrapping all of this into --bootstrap and --use-bootstrap machinery is much more packager-friendly that the current state of things.

Is this more of a request for better documentation? The --bootstrap option you propose already implicitly exists via the other available options such as the following: https://github.com/dotnet/dotnet/blob/709947dd2b135e61d3ab2668b0d2979bac2bed3f/build.sh#L36-L37. It's necessary to have specific path inputs rather than just a flag. So it's not exactly clear how a --bootstrap option is useful.

But if you'd like improvements made to the existing docs (bootstrapping-guidelines.md, boostrap-new-os.md), that should be explicitly called out. Be aware that we already do have these existing bootstrapping doc issues:

• boostrap-new-os.md doc is outdated or obsolete #4729
• Provide developer documentation that describes how to do a stage 2 build #4422