Nextjs custom server typescript memory leak...

[RayLuxembourg]

Bug report

Describe the bug

memory leak.
On my server i have rest api that handles requests coming from getInitialProps.
The server sometimes uses apollo-fetch to get data from other api.
emotion js is used for styling.
That's mainly what the app does.
I am running another service with nodejs and have no issues at all.
I followed the examples but couldn't fix the memory leak.
While inspecting the memory leak, I could only see webpack strings growing.

To Reproduce

Create custom server with typescript.
Use apollo fetch
Use emotion
Use mongoose
Use lru-cache

...

Expected behavior

Stable service.

Screenshots

image

System information

google cloud kubernetes

[timneutkens]

This issue is not actionable from our side as no reproducible demo is provided.

[RayLuxembourg]

This issue is not actionable from our side as no reproducible demo is provided.

I have no idea how to add complete reproduction . Shouldn’t there be documentation on how to avoid memory leaks on custom server ? Or cases?

[dont-see-big-shark]

I facing this situation either, using ts run a custom server, after 10 days or more, memory leak, cpu peaks, finally system broken.

[RayLuxembourg]

I think that this is clear that, nextjs doesn't have clear guidelines on how to avoid memory leaks and do and don'ts.
It is not a normal nodejs application and it has to have specific guidelines on how to develop without memory leaks.

[kachkaev]

Could it be just the growing size of lru-cache rather than the memory leak? What if your storage settings are just a bit too generous? One way or the other, I’m not sure how the maintainers can help given a rather vague problem definition and lack of reproduction example.

Debugging memory leaks in Next.js servers is no different to debugging leaks in Express or a plain node process. I’m not sure what guidance specific to Next.js could be written down.

[Timer]

Closing as low activity for a week. There is no special way to troubleshoot memory issues in a Next.js app versus normal Node apps.

[RayLuxembourg]

Could it be just the growing size of lru-cache rather than the memory leak? What if your storage settings are just a bit too generous? One way or the other, I’m not sure how the maintainers can help given a rather vague problem definition and lack of reproduction example.

Debugging memory leaks in Next.js servers is no different to debugging leaks in Express or a plain node process. I’m not sure what guidance specific to Next.js could be written down.

I have tried to manage everything related to lru-cache but the app is still increasing in memory usage over time.
I seriously don't know what to do. Next.js doesn't offer any guidelines to avoid memory leaks.
It's an SSR react application there is lots of holes for memory leaks.
I am running my app in production for more then a year now.
I have 2k visitors per day, more than 20k pages. and around 40k requests per day.

Please share some documentation on what things can cause memory leaks in SSR nextjs application.
from styling, webpack,babel, to caching.

[RayLuxembourg]

It might have been the cache. Nice drop finally happened after few hours.

[RayLuxembourg]

Countless reports that next.js has memory leaks and all issues are closed without resolve.

[madeinspace]

I have the exact same problem, using azure and I get the same memory behaviour with a same type of graph you're showing. @RayLuxembourg did you end up resolving this issue?

tried to disable lru-cache but issue remains

[Timer]

This should be fixed in the latest version of Next.js. What version are you on?

[madeinspace]

This should be fixed in the latest version of Next.js. What version are you on?

great news! I'm on version 9.2.1

[Timer]

Please upgrade to 9.3.5 or newer and let us know!

[madeinspace]

@Timer Unfortunately reporting that the latest version didn't fix my memory leak issue. What has changed in that version that made you guys think this would be addressed? perhaps I overlooked something, my custom server is pretty simple though (https://github.com/madeinspace/EcoNext/blob/master/server.js)

[AlbericTrancart]

Hello! Having the same issue than @madeinspace. Same type of memory behavior with Next on Azure. Updated to 9.3.5 but didn't fix it.
I will try to reproduce this in my local env but any reference to other issues might help.

[madeinspace]

I'm using styled-components and I'm reading there are some potential issues with this too, @AlbericTrancart are you using SC too?

[omeraplak]

Hello,
Having the same issue. Updated to 9.3.6 but didn't fix it. We are also using typescript and styled component.

[elliottcrif]

Hello haveing the same issue. @RayLuxembourg Have you figured out the issue?

[macrozone]

@elliottcrif @RayLuxembourg did you guys try to update 9.5.x ? I had rarely seen out of memory problems in previous versions, but with 9.5.x it happens very often. So i am wondering if its also much worse for you on 9.5.x ?

[JulienKode]

Hello I have the same issue, it is now fixed ?

[timdavish]

@macrozone I have been experiencing FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory server crashes in development at an increasing frequency, after updating to 9.5.x

It's at the point where it's severely impacting DX, needing to manually restart the app server every 5 minutes.

Adding NODE_OPTIONS=\"--max-old-space-size=4096\" in front of my start script seems to help for now..

[murilozilli]

They probably dont want to help resolve this as they want you to use Vercel servers.

@timdavish im facing the same issues using next version 9.5.1 and I did a similar thing. I am using pm2 to manage my node server

I create this file pm2.json and then you can run pm2 start pm2.json

{
  "apps": [
      {
          "name": "profiles",
          "script": "server.js",
	  "max_memory_restart": "1800M",
	  "node_args": "--max_old_space_size=2048"
      }
  ]
}

with that configuration pm2 automatically restarts the application when it reaches 1.8g of memory so it never reaches the server maximum, problem is that because when that happens the server looses all the cache of generated files then first time a user visits a page, it takes a lot of time to re-compile.

[kurtobando]

Hello team,

I have the same behavior with my custom server

const express = require("express")
const next = require("next")

const port = parseInt(process.env.PORT, 10) || 3000
const dev = process.env.NODE_ENV !== "production"
const app = next({ dev })
const handle = app.getRequestHandler()

app.prepare().then(() => {
    const server = express()

    server.all("*", (req, res) => {
        return handle(req, res)
    })

    server.listen(port, (err) => {
        if (err) throw err

        console.log(`> Ready on http://localhost:${port}`)
    })
})

At the moment, I am running in httpd with passenger.

Any recommendation or experience for this behavior?

[RayLuxembourg]

Update. I have dropped using nextjs as both frontend and backend and instead, I just use it to render React on the server-side.
I use nest.js as dedicated backend.

[timwaddell]

We had some big memory issues and it turned out being express in front of nextjs
Hopefully this can help others if you have implemented similar, try removing express and see if memory improves?

edit: wasn't express at all, it was the presence of nextjs custom server file.. a standard next start works like a dream but custom falls over

[rakaaditya]

I think I have the same pattern in my memory utilization graph. Using Express custom server on AWS ECS.

[maidi29]

@timwaddell Did you manage to find out if the memory leak came from a certain implementation inside the custom server? We also experience memory issues on our production system using a custom server.

We are still investigating our issues trying to localize the origin... We also tried to use version where the next handler was not called at all (next was not used, instead we served files from a local volume) and had no memory issues there, so the custom server itself does not have the issue, but somehow the retrival of the next pages. We use app.renderToHTML() to get the next pages.

Any further information about memory issues in combination with a custom server would be useful to us

[macso-hwangoh]

I have been experiencing the same error on Azure. I followed https://alberic.trancart.net/2020/05/how-fixed-first-memory-leak-nextjs-nodejs/ to recreate the memory leak locally as well. Specifically, you can create a script called "load.sh" with contents:

max="$1"
date
echo "url: $2
rate: $max calls / second"
START=$(date +%s);

get () {
  curl -s -v "$1" 2>&1 | tr '\r\n' '\\n' | awk -v date="$(date +'%r')" '{print $0"\n-----", date}' >> /tmp/perf-test.log
}

while true
do
  echo $(($(date +%s) - START)) | awk '{print int($1/60)":"int($1%60)}'
  sleep 1

  for i in `seq 1 $max`
  do
    get $2 &
  done
done

which can be used to send requests per second to your local server using ./load.sh 25 "http://localhost:3000/" From here, enable inspect mode and use Chrome inspect to take heap snapshots.

Not only does this eventually cause a segmentation fault on my webapp but the same occurs for the default Next.js web app example suggested in https://nextjs.org/learn/basics/create-nextjs-app/setup:

hwangoh@LAPTOP-C6317PJA:~/codes/nextjs-blog$ npm run dev

> @ dev /home/hwangoh/codes/nextjs-blog
> NODE_OPTIONS='--inspect' next dev

Debugger listening on ws://127.0.0.1:9229/d3348316-7a9d-4893-8eff-981acc66d243
For help, see: https://nodejs.org/en/docs/inspector
ready - started server on 0.0.0.0:3000, url: http://localhost:3000
event - compiled client and server successfully in 604 ms (158 modules)
Debugger attached.
wait  - compiling / (client and server)...
(node:22471) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 client/ listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit

(Use `node --trace-warnings ...` to show where the warning was created)
(node:22471) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 server/ listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
event - compiled client and server successfully in 517 ms (174 modules)
Segmentation fault
npm ERR! code ELIFECYCLE
npm ERR! errno 139
npm ERR! @ dev: `NODE_OPTIONS='--inspect' next dev`
npm ERR! Exit status 139
npm ERR!

npm ERR! Failed at the @ dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/hwangoh/.npm/_logs/2021-12-15T02_46_18_159Z-debug.log

This error is easily reproducible. Simply use npm run dev on the example webapp and load the local server with 25 requests per second as instructed in the link https://alberic.trancart.net/2020/05/how-fixed-first-memory-leak-nextjs-nodejs/. However I should mention that Chrome inspect heap snapshots do not indicate a consistent increase in memory but eventually a segmentation fault will occur.

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.