Build hangs indefinitely after updating TypeScript to 4.9.3

[romainmenke]

Bug Report

Since upgrading to 4.9.3 we are seeing frequent occurrences of a "stuck build":

• never finishes.
• doesn't always get stuck at the same point.
• seems to happen mostly/only on macOS

Example build that is currently stuck : https://github.com/csstools/postcss-plugins/actions/runs/3500523561/jobs/5863287052

I think it is related to updating TypeScript but I can be wrong about that.

🔎 Search Terms

build hangs infinite

🕗 Version & Regression Information

4.9.3

• This changed between versions 4.8.4 and 4.9.3

🙁 Actual behavior

build hangs and never finished

🙂 Expected behavior

build does not hang

[RyanCavanaugh]

It doesn't look like tsc is running when the job timed out, from my reading of the log. It ran at line 23 and then other stuff runs.

I'd need a more definitive blame than this to investigate further.

[romainmenke]

Are there any flags of options we can pass to tsc to get a more useful debug log?

[romainmenke]

tsc is called directly only once. Rest of the usage is through rollup.

[RyanCavanaugh]

If tsc runs and finishes I'm not sure what you're expecting from a log. --extendedDiagnostics will print a performance overview, though.

If rollup is hanging, I'd recommend logging an issue at their repo.

[romainmenke]

If rollup is hanging, I'd recommend logging an issue at their repo.

We didn't update rollup or any plugin, we updated TypeScript to 4.9.
The only thing we changed was the TypeScript version.

I'm not sure what you're expecting from a log. --extendedDiagnostics will print a performance overview, though.

I've added extendedDiagnostics and it is now visible that TypeScript is running at the time the build hangs and is also the last thing that emits logs.

https://github.com/csstools/postcss-plugins/actions/runs/3502766465/jobs/5867377495

I've done another run with even more diagnostics enabled :

"diagnostics": true,
"explainFiles": true,
"extendedDiagnostics": true,

https://github.com/csstools/postcss-plugins/actions/runs/3502799996/jobs/5867432582

Here it is also clear that the build run on macos is doing much more than on other platforms. There are ±60k log lines when it gets stuck instead of ±36k on the other platforms.

---

Please let me know how to provide more info on this issue.

[romainmenke]

@RyanCavanaugh can you let me know which info I can provide to help you investigate?

To reproduce this locally you should be able to follow these steps and get the same issue :

• clone https://github.com/csstools/postcss-plugins on a macos intel machine (I can't reproduce on M1/M2)
• checkout branch : revert-713-typescript-4.8--intuitive-labradoodle-bd363b90e9
• npm ci
• npm run build

[sheetalkamat]

May be #51627 helps with your issue. Do you want to give that a try

[andrewbranch]

I couldn’t reproduce with those steps on an Intel Mac.

[romainmenke]

I couldn’t reproduce with those steps on an Intel Mac.

🤔 tricky

I've let it run again to make sure I wasn't encountering some glitch before :

• update all other dependencies on main branch
• verified that everything works with TypeScript 4.8.x
• update only TypeScript to 4.9.x on this test branch
• let it run for 40 minutes to make sure I wasn't killing it before it could finish

Result is the same. Build hangs indefinitely.

• https://github.com/csstools/postcss-plugins/actions/runs/3530332292/jobs/5922211817
• TypeScript 4.9 csstools/postcss-plugins#714

I will try to gather as much info as I can about the env in those runners.

[romainmenke]

@sheetalkamat That also get's stuck :/

• csstools/postcss-plugins@1ec283c
• https://github.com/csstools/postcss-plugins/actions/runs/3531300315/jobs/5924310015

[romainmenke]

Managed to track it down to this commit : 7ab89e5

Unfortunately it is a massive merge commit :/

But everything after that one fails, everything before it is fine.

[sheetalkamat]

Can you try setting watchOptions in the tsconfig files to { "watchFile": "PriorityPollingInterval" } and give that a try.
I see lot of file watcher triggered event and i am not sure if the change to use fsEvents by default would have caused this so just trying to rule it out.

[romainmenke]

Did a couple of runs with this setting and I still saw failures :

config I added : csstools/postcss-plugins@4e91ce9

• https://github.com/csstools/postcss-plugins/actions/runs/3534732762/jobs/5931951157
• https://github.com/csstools/postcss-plugins/actions/runs/3534865441/jobs/5932236194

Is there a way to disable file watching completely?
These are single build runs so I am unsure if file watching is even needed for this?

[sheetalkamat]

i am not sure who is starting file watching in your case. so i dont know how to disable it,.

[romainmenke]

It seems that the rollup plugin for typescript always creates ts.createWatchProgram which is weird.

https://github.com/rollup/plugins/blob/master/packages/typescript/src/watchProgram.ts#L202

This snippet seems to detect that Typescript has started or finished : https://github.com/rollup/plugins/blob/master/packages/typescript/src/watchProgram.ts#L81-L97

---

I wished the maintainer of the rollup showed more interest in this issue.
I don't have any context of either TypeScript internals or rollup internals.

---

I am going to ask my colleague who was able to reproduce locally to provide more info first.
Thank you all for spending time on this 🙇

[romainmenke]

I can no longer reproduce this issue when testing with the latest commit currently on the main branch.

f6628a4

Haven't pinpointed the commit yet where the issue went away.

---

I am going to ask my colleague who was able to reproduce locally to provide more info first.

Still waiting on that.

[romainmenke]

This commit is the first one that works fine : e50bb07

It doesn't make any sense to me.
Because in that commit only a dev dependency was changed.

I ran the tests so many times now and on that commit it works consistently.
Before that commit it fails.

Normally I am quite good at finding root causes but this issue really has me stumped.
Truly no idea what is going on here.

---

To pinpoint this commit I took this steps :

• ran git --no-pager log v4.9.3...f6628a4573cd37c26912f78de3d08cd1dbf687a5 --format=format:%H
• this gave me a list of commits
• manual binary search on that list
• each commit pushing to our CI to check if it fails or passes
• if it passes I re-run to make sure

package.json :

		"typescript": "git://github.com/microsoft/TypeScript.git#<commit>"
	},
	"overrides": {
		"typescript": "$typescript"
	},

Is the git history maybe not linear?
That would invalidate my approach.

[romainmenke]

I can no longer reproduce this on TypeScript 5.x or 4.9.4.
I have no idea what was really going on, hopefully this doesn't come back :)

Thank you all for taking a look at this and providing insights!

[jasonmit]

I've experienced this going from 4.8 to 4.9 also. I don't yet have anything more to add which is why I'm commenting rather than creating a new issue. After about 10 minutes, my machine runs out of memory and tsc crashes. I can provide that dump if it's of use, just let me know.