Reduce truncation of information in the runningProcesses*.txt files #12398
Conversation
| @@ -3,7 +3,7 @@ Set-Location $args[0] | |||
| $timestamp = $(get-date -f MM-dd-HH-mm) | |||
|
|
|||
| while ($true) { | |||
There was a problem hiding this comment.
@dougbu Does this run every during the build to help diagnose hangs?
There was a problem hiding this comment.
Yup. Have a look at the log artifacts for the Windows jobs. Since the file is repeatedly overwritten, it should end with the list of processes just before AzDO gives up on us. Just a little something @JunTaoLuo cooked up 😃
There was a problem hiding this comment.
Great! We need it cross plat as well 😄
|
Just took a look at the output, I have a few suggestions on how to improve this:
It would be great to see the msbuild arguments etc. |
|
Hmm I'm surprised the command line isn't being captured, especially since I specified |
|
@davidfowl there are two files which are repeatedly overwritten; no historical data is saved -- just the last list before the PowerShell job is canceled. The second file is named |
|
For this build, the https://dev.azure.com/dnceng/_apis/resources/Containers/1977117?itemPath=Windows_Logs%2FrunningProcessesCommandLine.07-20-19-20.txt file starts with: @JunTaoLuo the @davidfowl oops, I didn't notice @JunTaoLuo left the previous files around. I'd prefer to keep at most two files around (just in case AzDO starts killing processes before the process dump last runs). @JunTaoLuo why did you keep many more files around? |
|
@davidfowl @JunTaoLuo jobs write up to 9 copies of each process dump file because
Since we never run |
|
BTW @davidfowl I'm not writing a dump_process.sh script in this PR though we have definitely seen hangs on macOS. Hangs have mostly occurred on Windows. For later use, I suspect |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
/azp run AspNetCore-ci |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run AspNetCore-ci |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Pilchie
added
the
area-infrastructure
|
Bump @davidfowl @JunTaoLuo |
JunTaoLuo
left a comment
JunTaoLuo
left a comment
There was a problem hiding this comment.
No good reason for keeping the files around. I just didn't add any logic for handling it.
My plan to handle it was just to remove the |
|
As long as the latest file is around, it sounds good to me. |
- move from deprecated `Get-WmiObject` to `Get-CimInstance`
- multiple files created today correspond to multiple '.\build.ps1' invocations
- no need to keep files from previous ./build.ps1 invocations around - by definition, job isn't hung if it can move to next build step
ghost
deleted the
Get-WmiObjecttoGet-CimInstance