fix: memory leak caused by how marked library custom renderer being used #30
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi,
We have been using react-email in production under fairly heavy load and have noticed a memory leak.
After some investigation I have narrowed it down to this repo and specifically how the underlying
marked
library is being used currently.I found this in marked library documentation: "Be careful: marked.use(...) should not be used in a loop or function. It should only be used directly after new Marked is created or marked is imported." (here: https://marked.js.org/using_advanced).
And the above warning from the marked documentation seems to be violated here in this repo: https://github.com/codeskills-dev/md-to-react-email/blob/master/src/parser.ts#L13
I have updated the code to use an alternative way of calling marked with a custom renderer instance.
Here are some memory snapshots from a small test I ran (using clinic.js to analyze memory):
![Screenshot 2024-11-18 at 15 47 35](https://private-user-images.githubusercontent.com/2354863/387289928-548711c3-c29a-4b8c-b561-793c925a7b88.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk4MTUwOTMsIm5iZiI6MTczOTgxNDc5MywicGF0aCI6Ii8yMzU0ODYzLzM4NzI4OTkyOC01NDg3MTFjMy1jMjlhLTRiOGMtYjU2MS03OTNjOTI1YTdiODgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMTdUMTc1MzEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YmFjMzFkZjUzY2ExMWUxNDVjYmEzYWVmZGM3MGY0OWI5Y2JiOTJlZjNkZjA5ZjllM2IxMDBlNjBkZDE2MDI0MCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.R8ktXDPAPN7-khb0akM4UVItxmm2rS8ZgLN1_ndvTfc)
Before fix:
After fix:
![Screenshot 2024-11-18 at 15 47 42](https://private-user-images.githubusercontent.com/2354863/387289990-e6eadb41-c5b7-404d-8e09-eecff5519c49.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk4MTUwOTMsIm5iZiI6MTczOTgxNDc5MywicGF0aCI6Ii8yMzU0ODYzLzM4NzI4OTk5MC1lNmVhZGI0MS1jNWI3LTQwNGQtOGUwOS1lZWNmZjU1MTljNDkucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMTdUMTc1MzEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NjAyY2NmZjAwMWQ3YmQ3YjNkMjg5OGQ4YzJmMmY5ZDFlNjljMDI4ODM1YzRkNWY1ZjJlYTJkYTJlZjI3MGUxOCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ._ysBpp9SUWrwP1-l8z7tABQGCB5Yba0L3zZ6rmcvYv0)
You can see that the memory profile is flat after the changes have been made (before it would eventually just crash with OOM error).
I also took this screenshot from running our production code locally and load testing the application and you can see that it retains the marked.js instances and does not release them. Eventually these build up and cause the OOM errors:
![Screenshot 2024-11-12 at 14 26 29](https://private-user-images.githubusercontent.com/2354863/387291031-3b5ab01b-f3ab-4dd3-a74e-f7cc932425de.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzk4MTUwOTMsIm5iZiI6MTczOTgxNDc5MywicGF0aCI6Ii8yMzU0ODYzLzM4NzI5MTAzMS0zYjVhYjAxYi1mM2FiLTRkZDMtYTc0ZS1mN2NjOTMyNDI1ZGUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIxNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMTdUMTc1MzEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NjAxMmU2NjA5YjVmZWIyYzY0OTlkYzQ5NjAyNDY0NWU4N2ZiNDdlZTk0NzM3NTJmZTdkNmU1MzAxZjExODhiMCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.vn-VPHbfpbbsyM7z08IIpVftyMA31HK7ncRk1oEK7OI)
I am not sure if more evidence is needed - I can push up my test case if you want? It was basically just calling
parseMarkdownToJSX
inside a nested loop thousands of times, similar to this (I used longer markdown example):Feel free to update my changes to modify what I have done (I don't have a lot of context on this codebase tbh). When I ran the tests locally all but one pass, and it fails snapshot test on quotes being encoded. However, this fails for me on master as well, so not too sure and might need advice on this.