tsc crash with node OOM

[jdhuntington]

TypeScript Version: 3.4.0-dev.20190222

Search Terms: Crash, OOM, infinite

Code

Across a few files - self-contained repro at https://github.com/jdhuntington/tsc-repro

Expected behavior:
This is a trimmed down repro of an issue encountered in a larger project. It is not expected to compile successfully, but should print several error messages.

Actual behavior:

tsc runs for several minutes comsuming 100%. Eventually the node
process exits with this error:

<--- Last few GCs --->

[64465:0x102803200]   227795 ms: Scavenge 1369.9 (1422.5) -> 1368.9 (1422.5) MB, 1.5 / 0.0 ms  (average mu = 0.181, current mu = 0.142) allocation failure
[64465:0x102803200]   227799 ms: Scavenge 1369.9 (1422.5) -> 1368.9 (1422.5) MB, 1.5 / 0.0 ms  (average mu = 0.181, current mu = 0.142) allocation failure
[64465:0x102803200]   227803 ms: Scavenge 1369.9 (1422.5) -> 1368.9 (1423.0) MB, 1.5 / 0.0 ms  (average mu = 0.181, current mu = 0.142) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x1360ab75be3d]
    1: StubFrame [pc: 0x1360ab72236e]
Security context: 0x3c72e589e6e1 <JSObject>
    2: slice [0x3c72e5886d09](this=0x3c72e4502959 <JSArray[8]>,0)
    3: /* anonymous */(aka /* anonymous */) [0x3c72e4502a11] [/private/var/folders/dx/dq3cr7h55q98bgm8t_ypkqp40000gn/T/interactive.Y0po3Zfl/tsc-repro/node_modules/typescript/lib/tsc.js:~32890] [pc=0x1360ac240071](this=0x3c725b0026f1 <undefined>,t=0x3c7250d...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003b125 node::Abort() [/Users/jdh/local/n/bin/node]
 2: 0x10003b32f node::OnFatalError(char const*, char const*) [/Users/jdh/local/n/bin/node]
 3: 0x1001a8e85 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/jdh/local/n/bin/node]
 4: 0x1005742a2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/jdh/local/n/bin/node]
 5: 0x100576d75 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/jdh/local/n/bin/node]
 6: 0x100572c1f v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/jdh/local/n/bin/node]
 7: 0x100570df4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/jdh/local/n/bin/node]
 8: 0x10057d68c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/jdh/local/n/bin/node]
 9: 0x10057d70f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/jdh/local/n/bin/node]
10: 0x10054d054 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/jdh/local/n/bin/node]
11: 0x1007d4f24 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/jdh/local/n/bin/node]
12: 0x1360ab75be3d
zsh: abort      ./node_modules/.bin/tsc --pretty --target es5 --outDir lib-commonjs --module

Related Issues: #29597 , #29326

[maxdeviant]

Seeing a similar issue in our own codebase after upgrading to TypeScript v3.4.1.

Here's the error that I get:

<--- Last few GCs --->

[23572:0000021C0540A5C0]   177901 ms: Scavenge 1300.9 (1422.8) -> 1300.1 (1423.3) MB, 4.5 / 0.0 ms  (average mu = 0.360, current mu = 0.367) allocation failure
[23572:0000021C0540A5C0]   177908 ms: Scavenge 1301.0 (1423.3) -> 1300.3 (1423.8) MB, 5.0 / 0.0 ms  (average mu = 0.360, current mu = 0.367) allocation failure
[23572:0000021C0540A5C0]   177913 ms: Scavenge 1301.2 (1423.8) -> 1300.5 (1424.8) MB, 3.2 / 0.0 ms  (average mu = 0.360, current mu = 0.367) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000001B45C3D0461]
Security context: 0x02b571e9d971 <JSObject>
    1: getIntersectionType(aka getIntersectionType) [0000018A21811AF1] [C:\Projects\npm-packages\node_modules\typescript\lib\tsc.js:~33183] [pc=000001B4615A473A](this=0x02d5e79825b1 <undefined>,0x038565d03979 <JSArray[10]>,0x02d5e79825b1 <undefined>,0x02d5e79825b1 <undefined>)
    2: arguments adaptor frame: 1->3
    3: /* anonymous */(aka /* anonymous */)...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20190402.175418.23572.001.json
Node.js report completed
 1: 00007FF777B9CC3A public: __cdecl v8::internal::GCIdleTimeHandler::GCIdleTimeHandler(void) __ptr64+4618
 2: 00007FF777B498B6 uv_loop_fork+80934
 3: 00007FF777B4A411 uv_loop_fork+83841
 4: 00007FF777F40F1E void __cdecl v8::internal::FatalProcessOutOfMemory(class v8::internal::Isolate * __ptr64,char const * __ptr64)+798
 5: 00007FF777F40E57 void __cdecl v8::internal::FatalProcessOutOfMemory(class v8::internal::Isolate * __ptr64,char const * __ptr64)+599
 6: 00007FF777FF0E74 public: static bool __cdecl v8::internal::Heap::RootIsImmortalImmovable(int)+14900
 7: 00007FF777FE6994 public: bool __cdecl v8::internal::Heap::CollectGarbage(enum v8::internal::AllocationSpace,enum v8::internal::GarbageCollectionReason,enum v8::GCCallbackFlags) __ptr64+7556
 8: 00007FF777FE5068 public: bool __cdecl v8::internal::Heap::CollectGarbage(enum v8::internal::AllocationSpace,enum v8::internal::GarbageCollectionReason,enum v8::GCCallbackFlags) __ptr64+1112
 9: 00007FF777FEEAB7 public: static bool __cdecl v8::internal::Heap::RootIsImmortalImmovable(int)+5751
10: 00007FF777FEEB36 public: static bool __cdecl v8::internal::Heap::RootIsImmortalImmovable(int)+5878
11: 00007FF7781778E1 public: class v8::internal::Handle<class v8::internal::HeapObject> __cdecl v8::internal::Factory::NewFillerObject(int,bool,enum v8::internal::AllocationSpace) __ptr64+49
12: 00007FF778230CEA public: static int __cdecl v8::internal::StoreBuffer::StoreBufferOverflow(class v8::internal::Isolate * __ptr64)+27082
13: 000001B45C3D0461

I'll try to see if I can narrow it down to a simple repro case.

@RyanCavanaugh Do you have any suggestions for how I might go about debugging the issue?

[rbuckton]

@jdhuntington the culprit seems to be this line in FocusZone.types.ts:

/**
 * FocusZone component props.
 */
export interface IFocusZoneProps
  extends React.HTMLAttributes<HTMLElement | FocusZone> {
  elementType?: keyof React.ReactHTML; // <- causing the crash
}

I am continuing to investigate what is causing this.

[maxdeviant]

@rbuckton I can confirm that keyof React.ReactHTML was the issue in our codebase as well. Commenting out the line causes the compiler to no longer crash.

[rbuckton]

It seems to be a combination of keyof React.ReactHTML and something react is doing when you pass IFocusZoneProps as your props type to React.Component. While I continue to investigate, you could try typing elementType as string to avoid the crash.

[rbuckton]

I think I have discovered the simplest repro for your case:

import * as React from "react";
declare const Tag: keyof React.ReactHTML;
<Tag ref={{}} />

I'm continuing to investigate, but my guess is that this is resulting in a significant number of instantiations due to the number of elements on React.ReactHTML.

[ghost]

Upgrading from tsc 3.4.5 to 3.5.0-rc causes a similar OOM exception when compiling our codebase. Is there anything we can do to help you diagnose the root cause?

[maxdeviant]

@matthewtebbs-codaio I think the most helpful thing would be narrowing it down to a simple repro case (like this one) so that it can be used to diagnose the problem.

Do you know if you are seeing the same issue due to keyof React.ReactHTML? If not, then I think finding an additional repro case might make the root cause clearer.

[ghost]

@maxdeviant will take a look. We use React so that's a possibility.

[weswigham]

For reference, here's the issue repro without involving react or jsx. (And a second version that doesn't rely on union call resolution in its construction) (And minimum size)

When we normalize an intersection of unions, the resulting union of intersections has an exponential number of clauses - IMO, the most graceful general thing we could do would be refuse to normalize and work as best we can with the un-normalized intersection of unions. #31486 helps in cases where we don't actually use ref or another such property with an exponentially large union by refraining from actually creating that union until it is actually used.

[ahejlsberg]

With #31572 we'll issue an error if a union type resulting from intersection type normalization exceeds 100,000 constituent types. The error will highlight the construct in which the excessively large type originates. This doesn't fix the potentially explosive types in @types/react, but it's certainly better than running out of memory.

[fatcerberus]

So it’s essentially a combinatorial explosion of types? Yikes!

[weswigham]

combinatorial explosion of types

Exactly exponential, actually. Conversion of formal logic into conjunctive normal form (which unions and intersections are analogous to) is well-known to be exponential in resulting clause count.

[fatcerberus]

conjunctive normal form

@weswigham
Interestingly, it actually looks like we're normalizing into DNF, not CNF. According to Wikipedia:

• CNF: "...a formula is in conjunctive normal form (CNF) or clausal normal form if it is a conjunction of one or more clauses, where a clause is a disjunction of literals; otherwise put, it is an AND of ORs."

• DNF: "...disjunctive normal form (DNF) is a canonical normal form of a logical formula consisting of a disjunction of conjunctive clauses; it can also be described as an OR of ANDs..."

So if my interpretation is correct, TS is going from CNF which is manageable ((A | B | C) & (D | E | F)) to DNF ((A & D) | (A & E) | ...) which explodes exponentially in the process, leading to an OOM (prior to the fix).

...that being said, I suppose if we were normalizing in the opposite direction we'd face the exact same problem sooner or later.

[weswigham]

Sure, the point is normalizing clausally produces an expontential number of clauses when you need to invert the nesting order.