Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why nlohmann does not release memory #1924

Closed
mohammad-nazari opened this issue Feb 3, 2020 · 18 comments
Closed

Why nlohmann does not release memory #1924

mohammad-nazari opened this issue Feb 3, 2020 · 18 comments

Comments

@mohammad-nazari
Copy link

mohammad-nazari commented Feb 3, 2020

We have two problems:
1- Why nlohmann use huge memory to parse data
2- After call parser in a function locally like below code it does not release memory. My JSON data size is about 8MB and the parser use more then 50MB for parsing. I parsed this JSON data 10 times and memory usage goes up to 600MB and after the function is finished memory did not released.

    GenerateNlohmann() {
      std::string filePath{FILE_ADDRESS};
      std::ifstream iFile(filePath.c_str(), std::ios::in);
      std::string data{};
      if (iFile.is_open()) {
        data = std::string((std::istreambuf_iterator<char>(iFile)),
                           std::istreambuf_iterator<char>()); // About 8MB size
        iFile.close();
      }
      if (!data.empty()) {
        nlohmann::json json = nlohmann::json::parse(data); // Use memory about 50MB
        std::vector<nlohmann::json> jsons{};
        for (int i = 0; i < 10; ++i) {
          nlohmann::json j = nlohmann::json::parse(data);
          jsons.emplace_back(j);
        }
        while (!jsons.empty()) {
          jsons.pop_back();
        }
      }
    }

        int main() {
          GenerateNlohmann();

    // Now memory usage is about 600MB
          std::cout << "Input a numberto exit" << std::endl;
          int i;
          std::cin >> i;

          return 0;
        }

Our platform is Ubuntu 18.04, CMake 3.15.3, g++ 7.4.0

@nlohmann
Copy link
Owner

nlohmann commented Feb 3, 2020

This is strange, as the library does release all allocated memory in the destructor, and we have no issues with Valgrind and ASAN. I need to double check your code.

@mohammad-nazari
Copy link
Author

Hi nlohmann, It is more strange if I say any things is OK in windows. So I am waiting for your response.
Thanks.

@nickaein
Copy link
Contributor

nickaein commented Feb 4, 2020

This is indeed strange. It could be due to allocator not releasing memory and caching it for later use. More info here and here.

I couldn't release the memory using malloc_trim() as the SO answers mentioned, but reproduced a similar behavior (~300MB for VmSize reported at /proc/<pid>/status) after calling the following function:

void TestMemAllocation() 
{
    std::vector<std::vector<int>> all_v{};
    for (int i = 0; i < 100; i++) 
    {
        std::vector<int> v;
        v.resize(5000000);
        all_v.emplace_back(v);
    }
}

@mohammad-nazari
Copy link
Author

mohammad-nazari commented Feb 4, 2020

This is indeed strange. It could be due to allocator not releasing memory and caching it for later use. More info here and here.

I couldn't release the memory using malloc_trim() as the SO answers mentioned, but reproduced a similar behavior (~300MB for VmSize reported at /proc/<pid>/status) after calling the following function:

void TestMemAllocation() 
{
    std::vector<std::vector<int>> all_v{};
    for (int i = 0; i < 100; i++) 
    {
        std::vector<int> v;
        v.resize(5000000);
        all_v.emplace_back(v);
    }
}

This is a memory paging policy.

@mohammad-nazari
Copy link
Author

This is strange, as the library does release all allocated memory in the destructor, and we have no issues with Valgrind and ASAN. I need to double check your code.

No idea or solution?

@dota17
Copy link
Contributor

dota17 commented Feb 7, 2020

This is indeed strange. It could be due to allocator not releasing memory and caching it for later use. More info here and here.

I couldn't release the memory using malloc_trim() as the SO answers mentioned, but reproduced a similar behavior (~300MB for VmSize reported at /proc/<pid>/status) after calling the following function:

void TestMemAllocation() 
{
    std::vector<std::vector<int>> all_v{};
    for (int i = 0; i < 100; i++) 
    {
        std::vector<int> v;
        v.resize(5000000);
        all_v.emplace_back(v);
    }
}

It is strange. I had try to reproduce your problem both in windows and linux, but all is OK and there are no issues with Valgrind and ASAN.

I agree with @nickaein, and will try to test and verify.

@nlohmann
Copy link
Owner

nlohmann commented Feb 8, 2020

I cannot reproduce your example. In my tests, all allocated memory is released.

@mohammad-nazari
Copy link
Author

I cannot reproduce your example. In my tests, all allocated memory is released.

Did you test my example? How to check memory is released? If you test with Valgrind or ASAN or ... it's OK. but in monitoring system you can see the memory is not released. We continually call the parser in a big project and the memory goes up more and more. We checked all other modules and libraries, They were OK.

@nlohmann
Copy link
Owner

nlohmann commented Feb 9, 2020

Yes:

image

You see the memory is going up during parsing, but then goes flat once GenerateNlohmann() is left.

@mohammad-nazari
Copy link
Author

Yes:

image

You see the memory is going up during parsing, but then goes flat once GenerateNlohmann() is left.

Very thanks.
What is this tool? Please send me download page.

@nlohmann
Copy link
Owner

nlohmann commented Feb 9, 2020

It’s Xcode.

@mohammad-nazari
Copy link
Author

Xcode

I test example in Ubuntu. Check it in Ubuntu please;

@nickaein
Copy link
Contributor

nickaein commented Feb 9, 2020

I still believe this is an optimization by the allocator (probably glibc in your case) and unrelated to the library.

After a closer examination, I see that by calling malloc_trim(0), the memory is indeed returned to the OS. Note that the virtual memory size might doesn't shrink which is non-issue, since glibc will (hopefully) keep the VM size below max). However, the size of resident memory (amount of memory residing on physical RAM) shrinks. Note the values of VmSize and VmRSS in two cases

# without calling malloc_trim
$ cat /proc/$(pidof without-malloc_trim.out)/status | grep Vm
VmPeak:	  571516 kB
VmSize:	  388364 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	  528560 kB
VmRSS:	  385600 kB
VmData:	  382576 kB
VmStk:	     136 kB
VmExe:	      84 kB
VmLib:	    3380 kB
VmPTE:	     800 kB
VmSwap:	       0 kB

# with calling malloc_trim
$ cat /proc/$(pidof with-malloc_trim.out)/status | grep Vm
VmPeak:	  571516 kB
VmSize:	  370860 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	  528632 kB
VmRSS:	    3436 kB
VmData:	  365072 kB
VmStk:	     136 kB
VmExe:	      84 kB
VmLib:	    3380 kB
VmPTE:	     764 kB
VmSwap:	       0 kB

@dota17
Copy link
Contributor

dota17 commented Feb 10, 2020

I had reproduce successful in ubuntu.
My json data size about 1.4 KB but i parsed this json data for many times.
Here are my test result:

Before Run:
KiB Mem :  8167476 total,  5461204 free,   284120 used,  2422152 buff/cache

1000 times:
KiB Mem :  8167476 total,  5456600 free,   288724 used,  2422152 buff/cache

10000 times:
KiB Mem :  8167476 total,  5405916 free,   339376 used,  2422184 buff/cache

100000 times:
KiB Mem :  8167476 total,  4893176 free,   852104 used,  2422196 buff/cache

After input the int (After run)
KiB Mem :  8167476 total,  5462208 free,   283116 used,  2422152 buff/cache

There is indeed a problem, but this is an optimization by the allocator (probably glibc in your case) and unrelated to the library, as @nickaein said.

If you add malloc_trim(0) in your code:

	while (!jsons.empty()) {
	  jsons.pop_back();
	}
	
+	malloc_trim(0);
  }

you will find everythink will be OK.
In windows we can not reproduce because what we use is not glibc, i think.

Other Test:
I had wrote other program to malloc many small memory with glibc, and the problem will still alive.My program is unrelated to the library, it just malloc and free many small memory.

Anyway, the probleam is unrelated to the library.
If we add malloc_trim(0) into the library, there will be many calls during the parsering, which will reduce performance.So the better solution is add malloc_trim(0) in your code.

@mohammad-nazari
Copy link
Author

Very thanks @dota17. And thanks to @nlohmann , @nickaein . You saved me and our team.
Just the last question: Is it possible to ignore glibc in linux based projects to avoid using malloc_trim(0)? We should find bottlenecks places in code and inject malloc_trim(0).

@nickaein
Copy link
Contributor

I highly doubt this behavior can cause any problems in most applications, except maybe and only maybe in very memory-constrained situations like an embedded systems with very small physical memory. Note that the memory allocated by this optimization isn't going to grow infinitely. You can verify that by calling GenerateNlohmann() multiple times. The cached memory stays the same.

Nevertheless, if this is really problematic in your case, these are the workarounds I can think of:

  1. glibc provides a tuning framework which you can use to tune memory allocation parameters. You might play around the parameters to see which combination can diminish the caching (e.g. trim_threshold, arena_max, mxfast, etc).

  2. As mentioned earlier, calling malloc_trim(0) after a large de-allocation would be a solution.

  3. Use other standard library implementations (a comparison on some libraries). Note that other libraries probably have implemented such caching too, since calling into kernel for each free() would be significantly impacts the performance.

  4. JSON library supports overriding the default allocator (an example). You might try other allocators like tcmalloc to see if they are helpful.

@mohammad-nazari
Copy link
Author

I highly doubt this behavior can cause any problems in most applications, except maybe and only maybe in very memory-constrained situations like an embedded systems with very small physical memory. Note that the memory allocated by this optimization isn't going to grow infinitely. You can verify that by calling GenerateNlohmann() multiple times. The cached memory stays the same.

Nevertheless, if this is really problematic in your case, these are the workarounds I can think of:

  1. glibc provides a tuning framework which you can use to tune memory allocation parameters. You might play around the parameters to see which combination can diminish the caching (e.g. trim_threshold, arena_max, mxfast, etc).
  2. As mentioned earlier, calling malloc_trim(0) after a large de-allocation would be a solution.
  3. Use other standard library implementations (a comparison on some libraries). Note that other libraries probably have implemented such caching too, since calling into kernel for each free() would be significantly impacts the performance.
  4. JSON library supports overriding the default allocator (an example). You might try other allocators like tcmalloc to see if they are helpful.

Great suggestions @nickaein. I will analyse your options and select the best solution.
Thanks.

@dota17
Copy link
Contributor

dota17 commented Feb 10, 2020

Unfortunately, I had tested the library - tcmalloc,the problem will be alive.
More info : link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants