Skip to content

Don't overwrite binary logs #66559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jaredpar merged 8 commits into from
Jan 26, 2023
Merged

Don't overwrite binary logs #66559

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
07f21b7
Don't overwrite binary logs
jaredpar Jan 26, 2023
43b4571
enable trace for diff issue
jaredpar Jan 26, 2023
9a6689e
Work around NuGet static graph restore bug
jaredpar Jan 26, 2023
59a537a
limit to CI only
jaredpar Jan 26, 2023
0fe1e9f
copy / paste bug
jaredpar Jan 26, 2023
8966289
it's your fault for letting me code
jaredpar Jan 26, 2023
b6887e9
why why why
jaredpar Jan 26, 2023
7991567
yml owns me
jaredpar Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,13 +222,13 @@ stages:
displayName: Restore
inputs:
filePath: eng/build.ps1
arguments: -configuration Release -prepareMachine -ci -restore -binaryLog
arguments: -configuration Release -prepareMachine -ci -restore -binaryLogName Restore.binlog

- task: PowerShell@2
displayName: Build
inputs:
filePath: eng/build.ps1
arguments: -configuration Release -prepareMachine -ci -build -pack -publish -sign -binaryLog
arguments: -configuration Release -prepareMachine -ci -build -pack -publish -sign -binaryLogName Build.binlog

- script: $(Build.SourcesDirectory)\artifacts\bin\BuildBoss\Release\net472\BuildBoss.exe -r "$(Build.SourcesDirectory)/" -c Release -p Roslyn.sln
displayName: Validate Build Artifacts
Expand Down Expand Up @@ -326,7 +326,7 @@ stages:
displayName: Restore
inputs:
filePath: eng/build.ps1
arguments: -configuration Debug -prepareMachine -ci -restore -binaryLog
arguments: -configuration Debug -prepareMachine -ci -restore -binaryLogName Restore.binlog

- powershell: .\eng\test-rebuild.ps1 -ci -configuration Release
displayName: Run BuildValidator
Expand Down
30 changes: 26 additions & 4 deletions eng/build.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ param (
[switch]$bootstrap,
[string]$bootstrapConfiguration = "Release",
[switch][Alias('bl')]$binaryLog,
[string]$binaryLogName = "",
[switch]$buildServerLog,
[switch]$ci,
[switch]$collectDumps,
Expand Down Expand Up @@ -79,6 +80,7 @@ function Print-Usage() {
Write-Host " -verbosity <value> Msbuild verbosity: q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]"
Write-Host " -deployExtensions Deploy built vsixes (short: -d)"
Write-Host " -binaryLog Create MSBuild binary log (short: -bl)"
Write-Host " -binaryLogName Name of the binary log (default Build.binlog)"
Write-Host " -buildServerLog Create Roslyn build server log"
Write-Host ""
Write-Host "Actions:"
Expand Down Expand Up @@ -166,13 +168,21 @@ function Process-Arguments() {
$script:applyOptimizationData = $false
}

if ($binaryLogName -ne "") {
$script:binaryLog = $true
Copy link
Member Author

@jaredpar jaredpar Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I required specifying both options so -binaryLog -binaryLogName "Build.binlog" but that just looked too verbose to me. Decided it was concise to allow -binaryLogName to imply -binaryLog

}

if ($ci) {
$script:binaryLog = $true
if ($bootstrap) {
$script:buildServerLog = $true
}
}

if ($binaryLog -and ($binaryLogName -eq "")) {
$script:binaryLogName = "Build.binlog"
Copy link
Member

@333fred 333fred Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm scared.

Copy link
Member Author

@jaredpar jaredpar Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well because you asked ...

It's because when powershell reads a variable it will check function scope, then script scope. But if you write a variable it will write it in the function scope, even if one exists at script scope. So you need to be very explicit that you're writing a script scoped variable here.

Copy link
Member

@333fred 333fred Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the $binaryLogName just above be $script:binaryLogName as well then?

Copy link
Member

@333fred 333fred Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And the one on line 228?

Copy link
Member Author

@jaredpar jaredpar Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The uses on 182 are reads. Those will fall back to script scope. Line 228 is creating a new variable $binaryLogPath which is intentionally function level scope.

Why don't we always use script: when accessing script variables and instead rely on fallback rules? I don't have a good argument for that other than that's just kinda how it's written in a lot of cases. It's like asking why we require this. for extension methods but not for normal instance methods.

}

$anyUnit = $testDesktop -or $testCoreClr
if ($anyUnit -and $testVsi) {
Write-Host "Cannot combine unit and VSI testing"
Expand Down Expand Up @@ -213,7 +223,14 @@ function BuildSolution() {

Write-Host "$($solution):"

$bl = if ($binaryLog) { "/bl:" + (Join-Path $LogDir "Build.binlog") } else { "" }
$bl = ""
if ($binaryLog) {
$binaryLogPath = Join-Path $LogDir $binaryLogName
$bl = "/bl:" + $binaryLogPath
if ($ci -and (Test-Path $binaryLogPath)) {
Write-LogIssue -Type "warning" -Message "Overwriting binary log file $($binaryLogPath)"
}
}

if ($buildServerLog) {
${env:ROSLYNCOMMANDLINELOGFILE} = Join-Path $LogDir "Build.Server.log"
Expand All @@ -240,6 +257,11 @@ function BuildSolution() {
$generateDocumentationFile = if ($skipDocumentation) { "/p:GenerateDocumentationFile=false" } else { "" }
$roslynUseHardLinks = if ($ci) { "/p:ROSLYNUSEHARDLINKS=true" } else { "" }

# Temporarily disable RestoreUseStaticGraphEvaluation to work around this NuGet issue
# in our CI builds
# https://github.com/NuGet/Home/issues/12373
$restoreUseStaticGraphEvaluation = if ($ci) { $false } else { $true }

try {
MSBuild $toolsetBuildProj `
$bl `
Expand All @@ -259,7 +281,7 @@ function BuildSolution() {
/p:TreatWarningsAsErrors=$warnAsError `
/p:EnableNgenOptimization=$applyOptimizationData `
/p:IbcOptimizationDataDir=$ibcDir `
/p:RestoreUseStaticGraphEvaluation=true `
/p:RestoreUseStaticGraphEvaluation=$restoreUseStaticGraphEvaluation `
/p:VisualStudioIbcDrop=$ibcDropName `
/p:VisualStudioDropAccessToken=$officialVisualStudioDropAccessToken `
$suppressExtensionDeployment `
Expand Down Expand Up @@ -734,7 +756,7 @@ try {
catch
{
if ($ci) {
echo "##vso[task.logissue type=error](NETCORE_ENGINEERING_TELEMETRY=Build) Build failed"
Write-LogIssue -Type "error" -Message "(NETCORE_ENGINEERING_TELEMETRY=Build) Build failed"
}
throw $_
}
Expand All @@ -752,7 +774,7 @@ try {
catch
{
if ($ci) {
echo "##vso[task.logissue type=error](NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed"
Write-LogIssue -Type "error" -Message "(NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed"
}
throw $_
}
Expand Down