Fix path bug in DownloadFromResultsContainer.targets #6179
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ViktorHofer
approved these changes
Sep 15, 2020
Contributor
|
Thanks @safern - When does this change get picked up in our pipeline? |
Member
Author
|
Once we get a new Helix SDK I'll give you instructions on how to consume this. |
kunalspathak
added a commit
to BruceForstall/runtime
that referenced
this pull request
Sep 25, 2020
kunalspathak
added a commit
to BruceForstall/runtime
that referenced
this pull request
Sep 25, 2020
Also, rename 'checked' -> 'Checked'
kunalspathak
added a commit
to dotnet/runtime
that referenced
this pull request
Oct 1, 2020
* Create a SuperPMI collection pipeline Start with a clone of jitstress.yml, and go from there. * Only build Windows_NT_x64, disable test builds and runs * TEMPORARY: disable normal PR triggered jobs * some template files for superpmi * Little trial error# 1 * fix the superpmi-setup.ps1 * Disable CoreClrTestBuildHost, fix the project path in superpmi-send-to-helix * Skip build, download, create core_root * Just copy superpmi scripts, import pipeline-logging-functions * add a dummy step * add missing space before message * Replace message with script * Temporarily disable dependsOn * disable another dependsOn * Pass superpmi.proj file * logging around send-to-helix * fix superpmi.proj to include ItemGroup * have placeholder.txt in WorkItem folder * fix workitem folder creation, script name * Re-enable build coreclr, libraries, download, core_root * fix superpmi.py path * clone jitutils, build and copy pmi.dll * Pass Core_Root as payload * Use hardcoded checked * Use .dotnet from runtime repo - Build jitutils before and send the payload to helix * Fix the pmi.dll location and add -output_mch_path * fix the output path * Try publishing SourcesDirectory * Upload to HELIX_WORKITEM_UPLOAD_ROOT * Include test in collection and partition core_root and test * Enable CoreClrTestBuildHost * Add dependsOn P1 test * Add DownloadFilesFromResults * Properly copy the Tests * Workaround for dotnet/arcade#6179 * Disable test run, SuperPMI just system.web.dll * Fix the mch path * Comment partition for test * Remove commented code * Increase timeout to 12 hours * fast pass - Comment build coreclr/libraries job - Instead just download coreclr from some previous run, and tell that it is core_root * Update the machine pool * Add a dummy step * Use SuperPMI_Min.zip that contains all the required files * Another way of Extracting cab * download in powershell, extract files separately * fix the zip path * Another logic of download zip * fix path * fix newdir path * more fixes * fix extract step * add test python script and modify creator * Map secret to env * decrease timeout, fix superpmi path in proj, print unzipped contents * Add a step to download and dir artifacts * Attempt to call super_pmi upload * list all contents * Also dir SourcesDir\artifacts * Use DownloadPipelineArtifact * Use the right key * Upload from helixresults * cleanup run-superpmi-job.yml * Re-enable superpmi for CORE_ROOT * fix the dependsOn * Replace checked with $(BuildConfig) * Replace checked with $(BuildConfig) Comment download artifacts * change helix queue for windows and add for arm64 * Use FileSeparateChar in superpmi.proj * Add Linux x64 * Fix the helix queue, property erroro * logging in shell * Fix PayloadDir and CorrelationPayloadDir * Make timeout to 5 hours * Print workitem contents * fix WorkItemDirectory * Just copy pmi.dll * fix the pmi_assemblies path * disable reporter * pop put JitUtilsDirectory before deleting it * chmod for superpmi-setup.sh * just copy pmi.dll for unix * fix the parameter to unix * Add logging to understand -pmi_assemblies issue * Add "binaries" folder where payload is present fix unit script that initializes array * remove precommand * Pass Core_Root as CorrelationPayload - Pass Core_Root as CorrelationPayload - Pass tag to name .mch file with - Changes to .ps1 and .sh to copy Core_Root directly * Add back the workaround for dotnet/arcade#6179 Also, rename 'checked' -> 'Checked' * fix checked for Linux * mkdir for linux * fix path in HelixResultsDestinationDir * Enable Windows x86/arm64 * fix the python path * First version of superpmi-setup.py * Attempt to use superpmi-setup.py * Fix spelling of LibrariesArtifacts, set python path * TestArtifacts commented * superpmi-setup script for linux * Enable Linux_x64 * Added pydocs * Add option for exclude_directories * Moved all the logic to superpmi-setup.py * Replace superpmi-setup.ps1 and superpmi-setup.sh with superpmi-setup.py * Fix up in superpmi-setup.py * fix display name * add missing spacE * Use differnet conditional * ignore PermissionError while cleaning up temp folder * fix workitem copy for linux * fix copy for unix * Publish logs * Print LibrariesArtifacts * remove space in front of publish logs desc * Add missing quotes for LibrariesArtifacts * Download x64 coreclr to make upload work * Enable Windows x86/arm64, Linux arm/arm64 * Delete superpmi-setup.ps1, superpmi-setup.sh * Fix space around LibrariesArtifacts * Download real coreclr x64 * ignore native dlls * Make superpmi.py Python3.6 friendly * Minor cleanup * Cleanup superpmi-send-to-helix to use helix-inner-step * Remove redundant steps * Merge mch files * Pass osGroup * Make superpmi-setup script takes tests_directory * Enable SPMI collection for P1 * General cleanup * fix merge_mch pattern for OS * Log the SPMI collection output and put it in upload_root folder * Replace robocopy/rsync with shutil.copy2 Note that shutil.copy2 is very slow specially on Linux as compared to rsync. But I think we are ok to have slowness in return of simpler code. * fix path of pmi.dll to be copied * Include linux .so and file_exe files in copy_directories * Try publishing just .mch file * Revert "Try publishing just .mch file" This reverts commit a5312c0. * Disable tests SPMI collection on 42053 - Tests run times out on Windows - Some of the partition's collection failed for tests because reply was not clean - Here are the details when libraries/tests were run: https://dev.azure.com/dnceng/internal/_build/results?buildId=832814&view=results - Hence, disable tests run and just have libraries run for now. - I didn't remove the test run part but commented the portion of tests collection with TODO intentionally so when I have to enable it, I will know what to enable. * Review feedback Co-authored-by: Kunal Pathak <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since we're using NormalizePath for a directory, it is evaluating to a literal.
https://dev.azure.com/dnceng/public/_build/results?buildId=814918&view=logs&j=fb000423-50a9-5897-1483-627612d50fb2&t=96127184-4a4c-5e48-afe7-0a46151c4af1
cc: @kunalspathak