Significantly increased memory usage in 3.13.0

[markrian]

Issue workflow progress

Progress of the issue based on the Contributor Workflow

• 1. The issue provides a reproduction available on GitHub, Stackblitz or CodeSandbox

  Please make sure the graphql-eslint version under package.json matches yours.

• 2. A failing test has been provided
• 3. A local solution has been provided
• 4. A pull request is pending review

---

Describe the bug

3.13.0 has increased memory consumption over 3.12.0, to the point that in CI the process is OOM-killed when run against the GitLab repository.

Using 3.12.0, the job passes reliably, while on 3.13.0 it fails reliably.

To Reproduce
Steps to reproduce the behavior:

(Apologies that this isn't a minimal reproduction. If/when I have time, I'll make one and post it here.)

For now:

1. See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102654.
2. The commit fb10562ab186e39800f5006601bce362e0d0474b bumps to 3.13.0, and fails to run in CI.
3. The commit 46548bfb0e624c4eb5fc3f8120373f8ae11dab7d moves back down to 3.12.0, and runs successfully (as does latest master)

Expected behavior

Memory usage shouldn't be significantly higher than with 3.12.0.

Environment:

• OS: Linux (Docker image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bullseye-ruby-2.7.patched-golang-1.18-node-16.14-postgresql-12:rubygems-3.2-git-2.36-lfs-2.9-chrome-103-yarn-1.22-graphicsmagick-1.3.36), though probably any Linux would do.
• @graphql-eslint/eslint-plugin: 3.13.0
• Node.js: 16.14.0

Additional context

I suspect that #1222 might be implicated, since it touches some caching-related stuff and was the most significant change between 3.12.0 and 3.13.0. But that's just a hunch.

[dimaMachina]

@markrian thanks for the detailed report, and sorry for this (:

this should be fixed in 3.13.1 version

[markrian]

@B2o5T Thanks for the fast reply and fix!

Unfortunately, I'm still seeing memory issues with 3.13.1. See this job failure, which ran against 3.13.1.

So, I'm going to reopen this. Let me know if you'd like more information or a reduced test case for this!

Oh, I can't reopen this 😅

[dimaMachina]

do you have a failure locally as well?

Update:

I suspect that #1222 might be implicated

You are right, It's due to this PR.

I tried to fix it by this check https://github.com/B2o5T/graphql-eslint/blob/b1b5ec7da638a58f96295c885476c16f1fc39963/packages/plugin/src/cache.ts#L23 but seems it's not enough for you.
Does process.env.NODE defined in CI? while running locally with eslint cli it should be defined, but how about ci environment?

[markrian]

do you have a failure locally as well?

I ran some tests locally, and I found 3.13.0 uses around 1.5GiB compared to 1.4GiB for 3.12.0 and 3.13.1 for that particular job. That doesn't sound like much, but maybe that's just enough to get killed in our CI (I'm not sure exactly what resource limits are like there).

As far as I can tell, yes, NODE is defined in our CI for that job: https://gitlab.com/gitlab-org/gitlab/-/jobs/3296964272#L72-74.

Given that I see effectively identical memory usage between 3.12.0 and 3.13.1 (but not 3.13.0) locally, I wonder if there's some Ci-related caching issue going on. I'll have a look into this.

---

I don't think this would help this particular case, but looking at the ModuleCache implementation, would it make sense to evict stale entries? I'm not sure if those stale entries are ever used, otherwise, and could then be GCed.

diff --git a/packages/plugin/src/cache.ts b/packages/plugin/src/cache.ts
index 915c391..748af55 100644
--- a/packages/plugin/src/cache.ts
+++ b/packages/plugin/src/cache.ts
@@ -24,6 +24,9 @@ export class ModuleCache<T, K = any> {
       process.hrtime(lastSeen)[0] < settings.lifetime
     ) {
       return result;
+    } else {
+      // evict stale result
+      this.map.delete(cacheKey);
     }
   }
 }

---

Edit: Ah, no, I see it's getting overwritten elsewhere anyway.

[dimaMachina]

Between 3.12.0 and 3.13.0 I also updated graphql-tools/utils, I tried to downgrade it to the same version as in 3.12.0, could you test following alpha version if it fixes your issue? 3.13.2-alpha-20221109140613-1815aa1

[markrian]

Sadly it still OOMs. It's frustrating that I can't replicate this locally 😩

I think I might just set NODE_OPTIONS=--max-old-space-size=3584 for now instead, or at least try different values.

[markrian]

Huh, even with NODE_OPTIONS=--max-old-space-size=3584 it got killed 🤔

I'm going to have to put this aside for now and stay on 3.12.0 until I have more time to look into this.

[dimaMachina]

I will close this issue due to no activity, will reopen it if it still exist for somebody