extractLicense: true leads to huge memory utilization (Out of memory)

[cray2015]

🐞 Bug report

Command (mark with an x)

• new
• build
• serve
• test
• e2e
• generate
• add
• update
• lint
• xi18n
• run
• config
• help
• version
• doc

Is this a regression?

No

Description

A clear and concise description of the problem...

Recently we started to see a huge jump of memory utilization and our CI failing while building our project. it was under 4GB whenever we were building our project. but the recent scenario was exhausting all the memory that is available. My system has 16GB of RAM and it is exhausting all of it. upon further inspection I found that we have a few extra License.txt files for a few modules. so I tried experimenting with extractLicense: false, so the License.txt files of those modules still remains but there was no 3rdpartylicense.txt file but the memory leak was fixed. I suspect there is a memory leak while trying to extract the licenses.

🔬 Minimal Reproduction

Not available. The only thing I know is how to fix it is by turning off the license extraction
extractLicense: false

🔥 Exception or Error

ng build --prod
92% chunk asset optimization TerserPlugin
<--- Last few GCs --->

[10441:0x27758c0]   148425 ms: Mark-sweep 1290.5 (1454.7) -> 1258.8 (1432.9) MB, 837.0 / 0.0 ms  (average mu = 0.135, current mu = 0.071) allocation failure GC in old space requested
[10441:0x27758c0]   149000 ms: Mark-sweep 1283.7 (1442.0) -> 1269.6 (1428.4) MB, 461.8 / 0.0 ms  (average mu = 0.160, current mu = 0.196) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0xe7dd74dbe1d]
    1: StubFrame [pc: 0xe7dd74cb681]
Security context: 0x0e76c7b9e6c1 
    2: replace [0xe76c7b905c1](this=0x179f3f5823c9 ,0x0a5861ec80c9 \/\u2028\u2029]>>,0x0a5861ec8161 )
    3: serialize(aka serialize) [0x2520c85e401] [/home/neurosensum/Neurosensum repository/portal/node_modules/seriali...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8fb090 node::Abort() [ng build --prod]
 2: 0x8fb0dc  [ng build --prod]
 3: 0xb0322e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [ng build --prod]
 4: 0xb03464 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [ng build --prod]
 5: 0xef74c2  [ng build --prod]
 6: 0xef75c8 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [ng build --prod]
 7: 0xf036a2 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [ng build --prod]
 8: 0xf03fd4 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [ng build --prod]
 9: 0xf06c41 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [ng build --prod]
10: 0xed688b v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [ng build --prod]
11: 0x11c5371 v8::internal::Runtime_StringBuilderConcat(int, v8::internal::Object**, v8::internal::Isolate*) [ng build --prod]
12: 0xe7dd74dbe1d 
Aborted (core dumped)

🌍 Your Environment

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/
    

Angular CLI: 8.1.3
Node: 12.16.3
OS: win32 x64
Angular: 8.1.3
... cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.803.28
@angular-devkit/build-angular      0.803.28
@angular-devkit/build-ng-packagr   0.801.3
@angular-devkit/build-optimizer    0.803.28
@angular-devkit/build-webpack      0.803.28
@angular-devkit/core               8.3.28
@angular-devkit/schematics         8.1.3
@angular/animations                8.2.14
@angular/cdk                       8.2.3
@angular/material                  8.2.3
@ngtools/webpack                   8.3.28
@schematics/angular                8.1.3
@schematics/update                 0.801.3
ng-packagr                         5.7.1
rxjs                               6.4.0
typescript                         3.4.5
webpack                            4.39.2

Anything else relevant?

We had this issue earlier also but we thought this is something related to our code so we reverted the one commit where it all started and somehow managed to get the build working. Also the build gives a Out of memory exception on Linux where as in windows it exhausts all the memory but ensures that the build happens, despite having the same amount of RAM.

[alan-agius4]

Hi @cray2015, several changes have been made to the license extraction since Angular CLI 8. Can you try to update to Angular CLI 10 and see if the problem persists?

From the above error it seems that you are not increasing the memory limit of the Node.Js process. Having 16Gb available in this case doesn't make any difference because the memory limit is that of 2GB. See: nodejs/node#25576 (comment)

To learn how to increase the memory limit you are see: https://blog.liplex.de/increase-node-memory-limit-for-build

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}