Next 9.5.1 out of memory after some hot reloads #15855
Comments
timneutkens
added
the
please add a complete reproduction
|
Please provide a reproduction so that we can investigate |
this is extremly hard to do, but i am 100% sure others have the same issue. Do you hav any idea how this could happen? Its a new problem and happens very consistantly. maybe circular dependencies? I investigate now in that |
Haven't heard anything from our development teams or others, vercel.com is a Next.js app with 250 unique Next.js pages so we would have definitely have run into it if it was as trivial as you're making it to be.
Issues with memory are practically impossible to investigate without having the application so now I can't give you pointers on where to look. You'll have to run tooling to track down where memory is allocated (e.g. the Node.js debugger profiler). If it makes it any easier to share the application we can also investigate it under (paid) enterprise support if that makes the company you work for share the application, which will likely save your development team significant amounts of time. |
will think abou that, thank you |
|
We are also seeing the same problem, including in production, I can't provide a reproduction due to sensitivity, but I can provide a next.config and plugins/env if that would be acceptable? |
Unlikely we'll be able to track it down based on just a next.config.js. As said before if the only concern is that the code is sensitive we can look at enterprise support which would include a NDA. |
This comment has been minimized.
This comment has been minimized.
|
I tried to debug it with profiler, i just see that the app takes roughly 1.6 - 1.9 GB just after the first page compilation. Maybe its just too big for nextjs to handle. I noticed that webpack seems to cache a lot of sources in memory:
|
|
i tried now setting |
|
@macrozone I'd really like to have a look at the application myself as it's pretty much impossible to judge if that's a problem based on that one screenshot. E.g. the |
|
@timneutkens ok, can you send me a message? (my work email should be under my account) |
|
I have been experiencing memory leak since 9.5.2 (I updated yesterday). I was working on the /api/ files yesterday and this morning when it will randomly crash during hot reloads. Since today afternoon, I was working on material-ui and some api After I kept making changes to makeStyles, and changing the classes back and forth (because I'm trying to get the layout right), then it will crash after quite a few hot reload compiling. Below, shows the number of hot reloads (in between the 2 crashes, I was working on material-ui's makeStyles and adding/removing classes to different component) and compiles while I'm playing with makeStyles and material-ui's Accordion component. I'm not sure whether the logs below help, but I have to say the crashes are all pretty random, so far crashed about 10-15 times in the span of 24 hours, sometimes when working on /api/ sometimes while working on material-ui.
|
|
We use styled-components (latest version 5.1.1), which similarly to material-ui creates global styes that will be injected into the head. Maybe the improvements of hot reloading interfers with that? |
|
I've been having the same issue for about 1-2 weeks as well. After starting up my local dev environment, I work for a bit and then end up having to restart due to a crash. I also get these System information
(I can provide the full Node-generated report JSON if needed). A coworker and I also noticed we were getting this error too:
When you say this do you mean a link to a PR with a live app on Vercel? We might be able to provide since Open Collective is open source and all our repos are open. (Edited to add: here's the PR I've been working on where I experienced the latest crashes, you can pull the code from our repo and see the Vercel app link in the PR.) |
|
Quick follow up: these are some messages I've been getting in my terminal warning of a memory leak.
It points specifically to this file in our frontend repo, which has an unnamed default export. It's one of the oldest files in our repo and definitely wasn't causing issues before. Perhaps there's been a change in the way Next.js handles these default unnamed exports that would cause the memory leak? This for example seems to be a new addition released within the last few weeks that was flagging that particular error, and warning of a memory leak. I played around making some changes to a component I've been working on this week while the crashes have been most notable, and was able to trigger one again just now. Here's the output from my terminal: Node had been using around ~1.9 GB for a while and I'm not sure if it helped trigger it or if it was coincidence but when I switched branches and made a few more changes that seemed to push it over the edge of the available 2GB of memory. |
|
I've noticed this issue happen 4 -5 times in the past month. It has happened when switching branches, but it also just happened when I updated my JSX to have an inline style. I tend to run |
|
I seem to get this crash after I change some CSS in the same file as tailwind imports, I am also using typescript. |
|
I've been getting this issue consistently after 1-2 hot reloads after upgrading to 9.5. It makes the dev experience slow (constantly restarting) and barely usable. The OOM occurred on 9.4 but much less frequently (once an hour) Has anyone been able to improve / resolve this issue? @timneutkens it looks like we have a few repros, I can also throw mine into the mix but would need to be under an NDA. |
|
FYI anyone having issues I was able to greatly reduce this by removing the headers function in my next config. |
|
I've been getting this error with 9.5 as my site has grown in size. It happens whenever I fast refresh and have been in dev mode for over 15 minutes. I could share my repo privately if it's helpful @timneutkens Update, fixedI refactored my entire app to make sure that there was absolutely no circular importing. I also changed my {
"scripts": {
"start": "NODE_OPTIONS=--max_old_space_size=8192 next"
}
}Now it's working just fine. Not sure what did it exactly, but I have a feeling it was a circular imports. So I recommend making sure you don't have circular imports in your app. For context, I'm on |
|
Edit: nope, it just stopped for a while, then came back 😞 |
|
Also happening to me since a few months now. I'm always on the latest version of nextjs and I'm using libraries such as elastic-ui, less, graphql (via urql), typescript. Its happening quite frequently, I'd say every 30-60 minutes of working, and always after the |
|
I'm also using |
|
Having this issue, increasing in severity + frequency lately. |
|
I'm getting this too, at least once or twice per full coding day. I rarely reset my dev server unless this happens. |
|
I'm facing the same issue if I don't keep restarting the server now and then, and after updating to the latest version there seems to be an issue with hot reloading too. Sometimes I have to refresh the site manually.
|
|
I have the same issue... i updated next version to Tried this also..
|
|
Still happening with latest v10, in case anyone was hoping it could've been solved by chance. |
|
I can also confirm that this is happening. It seems like it may be related to page navigation somehow, as, when trying to come up with reproducible steps, we noticed that if we stayed on the same page and tried a series of change+save loops to get the hot reloading to fire quickly, it wouldn't crash. But, if we tried navigating through our app for a bit, and then doing a few change+save loops we would quickly get to a point where it crashed. A few properties of the codebase I'm working on:
As far as I know, no one on our team had this issue on NextJS v9.3.6. |
Hey, it seems that going to Next 10.2.1 (including #24282) + using webpack5 has solved the issue in my case. |
|
As suggested by @Alounv I updated next. I'm currently on v10.2.2. The issue got even worse in my demo above which wouldn't start at all now. I had to update the minimum amount of memory from 128 to 160mb. To get some realistic feedback I removed |
Are you using webpack5? |
Yes |
|
I don't know if this is useful, but I found that some React components were loading the Anyways, seems one thing I learned was to cut down your generated pages to as few as possible and add them back in until you see errors so you see them. Update 1: We have a big json file that's a mapping of zip codes to coordinates. This was |
|
Deleting my |
Ok so for me the issue was it seem this: I have folder named components, where i have a lot of small components that app re-uses.
I removed index.js and changed all the imports, to direct imports e.g. :
After i did this in whole project (maaan was this a painfull job for many many hours 🤮 😭 ) it seems that the problem with memory leak no longer occurs (haven't got a single error in a month). Hope that someone has the problem as I did and it will help him :). |
|
Weird. So tree shaking doesnt work if you import + re-export?
…On Tue, 29 Jun 2021 at 19:23, Dunky45 ***@***.***> wrote:
Just want to add my experience with this and spill some of my toughts. As
the others said here. I'm also experiencing this issue in dev since
upgrading to 9.5 and in other project to latest 10.x. . It often crashes
during some CSS work or just when we navigate throughout the project (maybe
some hot-reload cache accumulating). On 9.5 I use SASS with next-sass lib
and on 10.x styled components. I just keep my fingers crossed it won't
happen in production 😄 , so far it hasn't but you never know.
To be honest I don't feel confident that the leaks can't be somewhere in
my code. I tried debugging snapshots but it's just too much information for
me and I would say that orienting in this kind of deep debugging is for
real senior hot-shots. (I've been working with react since 2017 and can't
rly understand much).
From my point of view trying to "fix" it with giving access to more RAM is
just pointless, THAS IS JUST NO FIX AT ALL and you just keep increasing the
cost of your app.
I totally understand that guys from Vercel are in tough spot here, since
helping with issue that you hardly can reproduce is almost impossible. But
we can all agree that something has changed guys, too many devs have this
problem and it shouldn't be overlooked.
Btw. Thanks for your work nevertheless, best framework out here still! 👍
Ok so for me the issue was it seem this:
I have folder named components, where i have a lot of small components
that app re-uses).
In that folder i *had* index.js file, which exported all those
components, so i could import them in pages/views like this
import {ComponentA, ComponentB, ComponentC} from './components'
I removed index.js and changed all the imports, to direct imports e.g. :
import ComponentA from './components/ComponentA'
After i did this in whole (maaan was this a painfull job from many many
hours) it seems that the problem with memory leak no longer occurs (haven't
got a single one in a month).
PS: + my app initial build size is half of what it was
Hope that someone has the problem as I did and it will help him :).
Happy coding!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#15855 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMYG5VSJUNJTXR5SKHGMRLTVHXPVANCNFSM4PUEQ6IQ>
.
|
I was also surprised, I'm not sure to this day if something was wrong with my setup, but tree shaking was not working for me it seems. Also my initial build size is now HALF of what it was before... 🤯 |
I bet you tried to find something before you started this endeavour, but does anybody by any chance know if there's an automated way to do this (using VSCode)? I'm using the same method of importing components by means of |
|
So, #12557 is probably relevant to this issue.
I ended up building a codemod for our codebase using I'll try checking if I can release the codemod to the general public. I would recommend getting rid of barrel files if all of the consumers of your code are within the same codebase. If there are consumers of your code outside of your codebase, and you really want those barrel files, have something to generate the barrels as a build artifact to help you avoid the creation of unintended circular dependencies, among other issues, while also providing a nice public api for your code. |
@jdfm That would be much appreciated! 🙏 If you need any help, happy to assist. |
|
That would be awesome @jdfm! It seemed to be the most logical way to use barrels by the time we started the project, not knowing that circular dependencies would be an issue and that tree shaking wouldn't work. |
I did, but i was worried that if i would let some script do the works I would end up with many errors and loose my head around. By doing it myself I was sure every import is correct, of course human error is a biggie here. But thankfully I am kinda good at this "copy-paste-robot like" things 😆. Also like others said, thanks to this there are improvements in build size, build speed and jest runtimes. |
This didn't work for us. We still get the |
@jdfm would you mind sharing your code mod. I am running into the same issue and need to change barrel files to deep file level imports. |
|
When I upgraded to the latest node |
|
I have tried with node |
|
I think this is a critical issue that strongly hinder adoption of Next framework in production environments. I am a big fan of the framework, so please have a DEEP look into it, after almost 1 year without a fix this deserve a focused investigation and a definitive fix, |
|
@joovnaz please read #15855 (comment) and the many other comments posted that posting heap out of memory messages will not help solve the issue for your particular application. We have had a deep look into this already and made many improvements, unfortunately no reproductions are provided so we can't investigate the particular applications referenced here including yours. @sokra has made a ton of memory usage improvements to Next.js and webpack (not particularly leaks even, just improvements to how memory is used) which is why most folks have seen significant improvements on Next.js 11 with webpack 5 (see #15855 (comment)). So I'd recommend to ensure you are on the latest version of Next.js and are using webpack 5. |
|
I no longer are affected by this problem, but its hard to tell what ultimatly fixed it. We are now on nextjs 11, but already in some version of nextjs 10, the issue was resolved I observed that once the memory crash in development did no longer ocure, i had instead some fash refresh hicups, where nextjs would force reload the whole page. I cannot tell whether this is related though. Its important to state that this issue here was never about memory leaks in production. I think this is a completly different problem that might have different reasons. for those who are affected by memory leaks in production: open up a different issue, please! Despite the activity here I would suggest to close this issue here. Reason is that a lot has changed since version 9.5.1. and information might be missleading here. Also i am no longer affected, so the initial issue is resolve for me and the information is no longer vaid.
|
This comment has been minimized.
This comment has been minimized.
|
Tried with next |
|
Same issue with latest 12.0.0 |
|
Same issue with version 12 on Github actions |
|
@revskill10 @alirezavlz your issue is likey related to #30330. Please try upgrading to 12.0.2. I'm going to lock this issue as there haven't been new reports about the particular issue @macrozone had is resolved. If you're running into any other issue please create a new issue following the issue template. |
and others
Bug report
Describe the bug
Since updating to 9.5.x (from 9.4.x), i get an out of memory error after 10 something hot reloads:
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memoryit did rarely happen in 9.4.1, but it happens very consistantly in 9.5.x
To Reproduce
thats probably tricky. it happens on big projects and might be related to some bug in the hot reload / rebuild. Maybe it happens when there are some import circles?
Expected behavior
nextjs should not go out-of-memory
System information
Additional information
we are using a custom server with typescript
The text was updated successfully, but these errors were encountered: