Next.js development high memory usage

[timneutkens]

Before posting a comment on this issue please read this entire post.

Previous work

The past few weeks we've been investigating / optimizing various memory usage issues. Specifically geared towards production memory usage. In investigating these we were able to find there was one memory leak in Node.js itself when using fetch() in Node.js versions before 18.17.0 (you'll want to use 18.17.1 for security patches though).

Most of the reports related to memory usage turned out to be reports of "it's higher than the previous version" rather than a memory leak. This was expected because in order to run App Router and Pages Router at the same time with different React versions two separate processes were needed. This has been resolved by reducing the amount of processes to two, one for Routing and App Router rendering, and one for Pages Router rendering. So far we haven't received new reports since the latest release.

In some issues there were reports related to Image Optimization in production, however no reproduction was provided so it could not be investigated adequately, if you have a reproduction for that please refer to this issue: #54482

New

With the memory usage in production resolved we've started investigating reports of development memory usage spikes. Unfortunately these reports suffer from the same problem as the production memory usage issues people raised before, they're full of comments saying same issue or posting screenshots of monitoring tools saying "Look, same issue".

Unfortunately, as you can image, these replies are not enough to investigate / narrow down what causes the memory usage, for example in multiple cases that we did get a reproduction and could investigate the reason for the high memory usage was:

• A bug in the application code, causing infinite looping components
• Accidental import of ~11.000 modules through icon libraries -- Yes, many icon libraries ship massive amounts of re-exports and those have to be bundled before it can be tree-shaken. We've been working on an automated way to split these up in optimize_barrel SWC transform and new optimizePackageImports config #54572 that should help a bit to reduce the size (and compilation speed too).
• Webpack customization, e.g. adding external libraries that change webpack settings, for example to write additional sourcemaps

So far I've been able to make one small change to webpack's memory caching to make it garbage collect a bit more aggressively in #54397. I'm not expecting that change to have a big impact on the reported issues though.

We'd like to investigate these reports further, however we're unable to narrow these down if there is no code to run to collect heap snapshots and profiles for, hence this issue. If you are able to please provide runnable code of what you're experiencing.

Comments that don't include runnable code will be automatically hidden in order to keep this issue productive. This includes comments that only have a screenshot and applications that can't run.

I'm going to combine the other reports into this issue as separate comments.

I've made sure that we have 2-3 engineers on our team available to investigate when we get runnable reproductions to investigate.

Thanks in advance!

_NEXT-1569

[timneutkens]

@noetix Disagree that this had anything to do with bullying. All I did was explain that my posts are continuously being ignored, including in this issue. Then I explained what to do, which is to provide a runnable application. There is nothing we can do without a runnable example, which was already shared in the initial issue, I even made it bold to highlight that further.

Happy to explain it again, the reason we can't do anything without runnable code is that in order to narrow down the memory usage we need to change the Next.js core code in the application, for example to disable client components compilation and such in order to narrow down where the memory usage comes from. There is no way to do that based on screenshots / messages / information you can provide as it would require countless hours of your time and our time (think 2 weeks full time at least) in order to investigate this.

The emoji reactions not being shown for off-topic marked posts is a bug in GitHub. As mentioned in the initial issue any posts that don't include a reproduction will be automatically hidden.

Since you didn't like the earlier explanation I'll just remove it, don't feel strongly about keeping the comment. Definitely wasn't bullying, you were reading into that. Bullying would be the threats I've received recently from anonymous developers on Twitter that they'll come visit my house soon...

@weyert we haven't made changes to development memory usage besides the PR linked in the issue so really all I need is a reproduction, luckily @AhmedChabayta posted one, hopefully that is enough, fingers crossed.

@Thinkscape please open a separate issue, that bug would be separate from this issue 👍

[limeburst]

I've posted a reproduction here: https://github.com/limeburst/vercel-nextjs-54708

Start the development server, navigate from

• http://localhost:3000/1
• http://localhost:3000/2
• ...
• http://localhost:3000/20

And watch the memory usage grow, until the server restarts.

[rajat1saxena]

You can use this code to see the issue. The server is getting aborted silently without any errors.

https://github.com/codelitdev/courselit/tree/tailwindcss-2

Logs

rajat@rajat-laptop:~/projects/courselit$ yarn dev
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env.local
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env
- ready started server on [::]:3000, url: http://localhost:3000
- event compiled client and server successfully in 545 ms (18 modules)
- wait compiling...
- event compiled client and server successfully in 263 ms (18 modules)
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env.local
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env.local
- info Loaded env from /home/rajat/projects/courselit/apps/web/.env
- wait compiling /404 (client and server)...
- wait compiling / (client and server)...
rajat@rajat-laptop:~/projects/courselit$

[stigmartins1]

I'm getting 'server out of memory' after a while by letting the server run and writing/saving code that calls the following functions a few times.

import { google } from "googleapis";

export async function authSheets() {
  //Function for authentication object
  const auth = new google.auth.GoogleAuth({
    keyFile: "./auth/auth-sa-sptk.json",
    scopes: ["https://www.googleapis.com/auth/spreadsheets"],
  });

  //Create client instance for auth
  const authClient = await auth.getClient();

  //Instance of the Sheets API
  const sheets = google.sheets({ version: "v4", auth: authClient });

  return {
    auth,
    authClient,
    sheets,
  };
}

import { authSheets } from "./authSheets";

export async function clearSheetContents(sheetName) {
  console.log("sheet =", sheetName);
  const SHEET_ID = "123";
  const sheetId = SHEET_ID;
  const { sheets } = await authSheets();

  try {
    const result = await sheets.spreadsheets.values.clear({
      spreadsheetId: sheetId,
      range: sheetName,
    });
    console.log("result.data =", result.data);
  } catch (err) {
    // TODO (developer) - Handle exception
    throw err;
  }
}

import { authSheets } from "./authSheets";

// https://developers.google.com/sheets/api/guides/values
export async function setSheetValues(sheetName, input) {
  const SHEET_ID = "123";
  const sheetId = SHEET_ID;
  const values = [input];
  const resource = { values };
  // Updates require a valid ValueInputOption parameter
  const valueInputOption = "RAW"; // The input is not parsed and is inserted as a string.
  const { sheets } = await authSheets();

  try {
    const result = await sheets.spreadsheets.values.append({
      spreadsheetId: sheetId,
      range: sheetName,
      valueInputOption: valueInputOption,
      resource,
    });
    console.log("result.data =", result.data);
  } catch (err) {
    // TODO (developer) - Handle exception
    throw err;
  }
}

{
  "name": "test",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "autoprefixer": "10.4.15",
    "axios": "^1.5.0",
    "encoding": "^0.1.13",
    "eslint": "8.48.0",
    "eslint-config-next": "13.4.19",
    "googleapis": "^126.0.1",
    "next": "13.4.19",
    "postcss": "8.4.29",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "tailwindcss": "3.3.3"
  }
}

Node v. v18.17.1

[cryptoMavrik]

Hey everyone, as part of Turbopack we've also implemented thorough memory tracing which allows for narrowing down why certain applications use a lot of memory. Can you follow these steps when using Turbopack and send us the relevant files:

If possible follow these steps which would give me the best picture to investigate:

• npm install next@canary (use the package manager you're using) -- We want to make sure you're using the very latest version of Next.js which includes the fixes mentioned earlier.

• rm -rf .next

• start development using the NEXT_CPU_PROF=1 and NEXT_TURBOPACK_TRACING=1 (regardless of if you're using Turbopack, it only affects when you do use Turbopack) environment variable. E.g.:

  • npm: NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 npm run dev
  • yarn: NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 yarn dev
  • pnpm: NEXT_TURBOPACK_TRACING=1 NEXT_CPU_PROF=1 pnpm dev

• Wait a few seconds

• Open a page that you're working on

• Wait till it's fully loaded

• Wait a few seconds

• Make an edit to a file that holds a component that is on the page

• Wait for the edit to apply

• Wait a few seconds

• Make another edit to the same file

• Wait a few seconds

• Exit the dev command (ctrl+c)

• Upload the CPU traces put in the root of the application directory to https://gist.github.com

• Upload the .next/trace file to https://gist.github.com -- Please don't run trace-to-tree yourself, as I use some other tools (e.g. Jaeger) that require the actual trace file.

• If you're using Turbopack upload the .next/trace.log as well, if it's too large for GitHub gists you can upload it to Google Drive or Dropbox and share it through that.

• Upload next.config.js (if you have one) to https://gist.github.com

• Share it here

We are experiencing very high memory usage, build times, compilation times(60s+), and have had no luck tracking down the issue. Hot refreshes can take 5s+. Also our app router migration turned 9000 modules -> 15000 modules. Any help would be greatly appreciated!

https://gist.github.com/cryptoMavrik/2697c797d720d99f32e5a93b6fb8c2b0

[rirh]

Similar to @JClackett I just hovering around 4GB memory usage. Also reloads takes like 30s for normal size pages.

[sorenhoyer]

Node.js v. 20.15 seems to be the last version, before the big memory leak got introduced in v. 20.16 (a very very minor leak can still be observed, but it's nothing in comparison).

[karlhorky]

Node.js v. 20.15 seems to be the last version, before the big memory leak got introduced in v. 20.16

There's a Node.js memory leak in one of the latest LTS releases? Do you have an official source for that? Is it an undici leak?

Oh, and did you try 20.17.0 to compare?

[rirh]

Node.js v. 20.15 seems to be the last version, before the big memory leak got introduced in v. 20.16 (a very very minor leak can still be observed, but it's nothing in comparison).

Check out my updated pictures

[sorenhoyer]

Node.js v. 20.15 seems to be the last version, before the big memory leak got introduced in v. 20.16 (a very very minor leak can still be observed, but it's nothing in comparison).

Check out my updated pictures

I'm saying you need to use exactly v. 20.15.x not a later major/minor (you're using 22.8 according to your pictures) as some leak was introduced in Node 20.16 - at least that's what we saw.

[rirh]

Node.js v. 20.15 seems to be the last version, before the big memory leak got introduced in v. 20.16 (a very very minor leak can still be observed, but it's nothing in comparison).

Check out my updated pictures

I'm saying you need to use exactly v. 20.15.x not a later major/minor (you're using 22.8 according to your pictures) as some leak was introduced in Node 20.16 - at least that's what we saw.

This node version seems way too new. Honestly, it feels like the React team is just throwing everything into one big mix, which doesn’t really match their usual style. I genuinely hope they can make it work, but right now, it feels like they’re more focused on flashy new features and buzz rather than practical memory management. sad :(

[GabenGar]

20.16 is a minor version of the latest LTS major version released almost a year ago. Not to mention NodeJS hickups has nothing to do with React, especially in NextJS thread.

[dalalRohit]

my next-server using 10.67GB memory on MacBook Air M1.

[yogithesymbian]

Potential memory leak in Nextjs Tylerastro/NCU_TOM#12

we have same issue my engine mbpm1 . i think i have wrong choosing nextjs 14 now its complicated . still looking for other solutions . the apps its not very big and its only one page already crap . antd nextjs 14 tailwind .

[wintercounter]

For those who are having issues with react-icons.

I currently have to work on a laptop with only 8G memory. I couldn't even render the first page before Next dying. I created this tool that you can use to purge all unused icons from the source files. Only for development for those who are completely blocked by this.

https://github.com/wintercounter/purge-react-icons

[elie222]

For those who are having issues with react-icons.

I currently have to work on a laptop with only 8G memory. I couldn't even render the first page before Next dying. I created this tool that you can use to purge all unused icons from the source files. Only for development for those who are completely blocked by this.

wintercounter/purge-react-icons

Does lucide react have similar issues?

[wintercounter]

Shouldn't, same for goes react-icons if you only use one icon pack. The issue is when projects are using multiple packs, then too many barrel files will get loaded. The project I'm working on currently is mainly using Lucide, but it also loads some icons from 12 other packs xD

[Royal-lobster]

next-server taking around 4-7gb for our codebase, attaching traces, hope it helps to solve this issue

• https://gist.github.com/Royal-lobster/c86c7c5dd887f4a06daa19800c229375
• https://gist.github.com/Royal-lobster/ecf87144f4d64ac304cb75276208801e
• https://gist.github.com/Royal-lobster/539c41c4e1035619810d13794da8a002

[yes-oo]

for now is there any workaround to this issue , i have a 12gb ram laptop and i can't continue programming my website , it's always 90% to 100% full after navigating to multiple web pages in dev mode , i hope this issue got solved soon , my entire system ram usage is 1.7 the remaining is eaten by dev mode

[mrgoonie]

for now is there any workaround to this issue , i have a 12gb ram laptop and i can't continue programming my website , it's always 90% to 100% full after navigating to multiple web pages in dev mode , i hope this issue got solved soon , my entire system ram usage is 1.7 the remaining is eaten by dev mode

These worked to me so far:

• Don't use fetch, switch to axios
• Use exactly Node v20.15.1 (not lower, not higher)

Read more here: #64212

[yes-oo]

• v20.15.1

Thank you for your suggestion. I appreciate your time. I tried your solution, but it seems to be the same, if not worse, than the latest version of Node.js. As for fetch and Axios, I don't use fetch anywhere in my project, so that's not the issue. I'm also using the latest version of Next.js to avoid any bugs that have already been resolved

[yogithesymbian]

for now is there any workaround to this issue , i have a 12gb ram laptop and i can't continue programming my website , it's always 90% to 100% full after navigating to multiple web pages in dev mode , i hope this issue got solved soon , my entire system ram usage is 1.7 the remaining is eaten by dev mode

These worked to me so far:

• Don't use fetch, switch to axios
• Use exactly Node v20.15.1 (not lower, not higher)

Read more here: #64212

I tested with a new project and haven’t used Axios, Fetch, or any other libraries yet—it's a fresh project, but I'm still encountering the issue. nextjs antd tailwind .

my terminal consume 7GB memory .
engine : mbpm1 ( macOS sequoia 15.0.1 )

edit : "I think it’s consuming my disk space (5GB - 7GB) because it's overloaded. My friend, working on the same project, is using 13GB of memory."

[hunghuy201280]

mine consumed ~13GB of memory which is insane

[delphinic-owl]

mine consumed ~13GB of memory which is insane

marginally beat you !

[shahargl]

is there any workaround for it?

[alex-statsig]

One thing I've uncovered recently is a stateful server can easily leak memory in development only. While nextjs does encourage a stateless server (I suppose you need to use a separate server otherwise), its easy to write code that subtly causes the same issue. These are pretty painful to fix today, but with module.hot support it could be much easier (although discussions on that seem stagnant)

Some examples:

• Using a setInterval and never clearing it (ex. to clear some global cache periodically, or flush logs, etc.)
• Adding a global hook (that captures module-scoped variables in its closure, which is easy to do accidentally). This includes configuring axios-retry, process.on lifecycle hooks, adding mongoose schemas, etc.
• Using packages that add global intervals or hooks, such as a database client without properly cleaning up any singletons

These can all lead to that version of the module (and its downstream dependencies) never getting garbage collected (since there's a dangling reference to it). This means every time you save a server module, a new copy of all your server modules is added to memory. If you re-save server files a lot, and have a large backend, this can lead to OOMs quickly.

The only workaround I've found today is to pollute the global namespace with cleanup functions that get invoked when the module is re-imported (so you can shutdown the old database client, or clear the old interval, etc.)

[yuriiholskiy]

Hey,
any updates on this?
We have a simple app with some requests with fetch, but It takes so much memory on server that it is insane?

Any workarounds on this?

[tocteman]

10gb RAM usage on a rather fresh project. I think this is a very important issue.
How can we diagnose/profile what's causing this massive RAM usage ourselves?

[jpmnteiro]

Same boat here.

Running nextjs (v14.x) takes 9.2GB~ RAM. Switching on --turbo brings the memory usage to 13.2GB~ RAM.

Is there any way to investigate what's causing the increase in memory usage?

[broscr]

The cause of the RAM overflow is using fetch. I had the same issue; I replaced all fetch requests with axios, and the project never went over 200MB of RAM after that.

[KillerCodeMonkey]

@broscr did you tried nextjs v15 before? i think fetch caching is deactivated per default, so there should no such strange memory usages, now