Increase NOFILE limit for tox-bootstrapd #1234

nurupo · 2018-10-18T08:57:01Z

tox-bootstrapd can use around 600 TCP sockets during TCP server's normal
functioning. Many systems default to having a soft limit of 1024 open file
descriptors, which we are close to reaching, so it was suggested we bump that
limit to a higher number. iphy suggested increasing it to 32768.

Related to the discussion at #1227.

This change is

CLAassistant · 2018-10-18T08:57:08Z

Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

codecov · 2018-10-18T09:40:52Z

Codecov Report

Merging #1234 into master will increase coverage by <.1%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           master   #1234     +/-   ##
========================================
+ Coverage      83%   83.1%   +<.1%     
========================================
  Files          82      82             
  Lines       14685   14684      -1     
========================================
+ Hits        12192   12203     +11     
+ Misses       2493    2481     -12

Impacted Files	Coverage Δ
toxcore/LAN_discovery.c	`83% <0%> (-2.9%)`	⬇️
toxcore/friend_connection.c	`93.4% <0%> (-0.9%)`	⬇️
toxcore/Messenger.c	`86.8% <0%> (-0.1%)`	⬇️
toxcore/DHT.c	`76.8% <0%> (ø)`	⬆️
toxav/toxav.c	`67.7% <0%> (+0.6%)`	⬆️
auto_tests/toxav_many_test.c	`93.2% <0%> (+0.6%)`	⬆️
toxav/msi.c	`65.6% <0%> (+1.1%)`	⬆️
toxav/audio.c	`70.3% <0%> (+4.6%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b56166f...d89f83f. Read the comment docs.

iphydf

Reviewed 7 of 7 files at r1.
Reviewable status: 0 of 1 approvals obtained (waiting on @nurupo)

.travis/cmake-freebsd-stage1, line 0 at r1 (raw file):
Any idea why reviewable thinks this is a binary file? Are there any weird control characters in it accidentally?

other/bootstrap_daemon/tox-bootstrapd.sh, line 41 at r1 (raw file):

		cat << EOF

Warning: Your system has very few filedescriptors available in total.

"file descriptors"

other/bootstrap_daemon/tox-bootstrapd.sh, line 44 at r1 (raw file):

Maybe you should try raising that by adding 'fs.file-max=100000' to your
/etc/sysctl.conf file.  Feel free to pick any number that you deem appropriate.

Typically, users may not have an idea what value is appropriate. We should recommend 32768, which is an arbitrary but somewhat widely spread number.

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

                          "Consider raising the limit to at least %ju or the recommended %ju. "
                          "Continuing using the current limit (%ju).\n",
                          (uintmax_t)limit.rlim_cur, (uintmax_t)rlim_min, (uintmax_t)rlim_suggested, (uintmax_t)limit.rlim_cur);

uint32_t or even "unsigned" should be enough, so we can use %u instead of %ju (I'm not sure how portable that is - mingw32 seems to have some issues with it according to robinli).

nurupo

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

other/bootstrap_daemon/tox-bootstrapd.sh, line 44 at r1 (raw file):

Previously, iphydf wrote…

Typically, users may not have an idea what value is appropriate. We should recommend 32768, which is an arbitrary but somewhat widely spread number.

That's a bit different. We recommend 32768 limit for the tox-bootstrapd process, however what we are talking in here about, the fs.file-max of /etc/sysctl.conf, is the limit of the entire Linux system, e.g. how many file descriptors can the entire system have open at the same time, all of the processes combined (summed). That's the same value that /proc/sys/fs/file-max reports, which we check a few lines above to make sure we don't exhaust the system of all file descriptors. Well, the system can still run out of fds if there are a lot of other programs/daemons running, but at least we make an attempt to pick a reasonable number.

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, iphydf wrote…

uint32_t or even "unsigned" should be enough, so we can use %u instead of %ju (I'm not sure how portable that is - mingw32 seems to have some issues with it according to robinli).

rlim_t is defined as a some unspecified unsigned integer type. On some systems it's unsigned long, on other it's unsigned long long. I initially used %llu but got a format warning saying that I use format for unsigned long long for a value of unsigned long type on either Linux or FreeBSD build on Travis-CI. If you make it %lu, you'd get a format warning on some systems sayingthat you are using unsigned long format for a value of the type unsigned long long. It appears like casting it to the biggest unsigned integer type and printing it out as the biggest u signed integer type -- %ju format string, I think it's defined in C99 -- is the best option. If %ju is not portable enough (MSVC has spotty C99 support), we should come up with something else. I don't see your suggestion of %u working though, it would cause the same format warnings.

iphydf

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo and @iphydf)

other/bootstrap_daemon/tox-bootstrapd.sh, line 44 at r1 (raw file):

Previously, nurupo wrote…

That's a bit different. We recommend 32768 limit for the tox-bootstrapd process, however what we are talking in here about, the fs.file-max of /etc/sysctl.conf, is the limit of the entire Linux system, e.g. how many file descriptors can the entire system have open at the same time, all of the processes combined (summed). That's the same value that /proc/sys/fs/file-max reports, which we check a few lines above to make sure we don't exhaust the system of all file descriptors. Well, the system can still run out of fds if there are a lot of other programs/daemons running, but at least we make an attempt to pick a reasonable number.

Ok, sounds reasonable.

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, nurupo wrote…

rlim_t is defined as a some unspecified unsigned integer type. On some systems it's unsigned long, on other it's unsigned long long. I initially used %llu but got a format warning saying that I use format for unsigned long long for a value of unsigned long type on either Linux or FreeBSD build on Travis-CI. If you make it %lu, you'd get a format warning on some systems sayingthat you are using unsigned long format for a value of the type unsigned long long. It appears like casting it to the biggest unsigned integer type and printing it out as the biggest u signed integer type -- %ju format string, I think it's defined in C99 -- is the best option. If %ju is not portable enough (MSVC has spotty C99 support), we should come up with something else. I don't see your suggestion of %u working though, it would cause the same format warnings.

I'll be more specific. I thought I was, by putting this comment on the line containing "uintmax_t". More specifically I meant: replace uintmax_t with either uint32_t or unsigned, so that %u can be used.

nurupo

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, iphydf wrote…

I'll be more specific. I thought I was, by putting this comment on the line containing "uintmax_t". More specifically I meant: replace uintmax_t with either uint32_t or unsigned, so that %u can be used.

You realize that you are suggesting potentially downcasting, for example, 64-bit unsigned integer to 32-bit unsigned integer? For example, rlim_t can be defined as a 64-bit unsigned long longon a system, but we would downcast it to unsigned which is 32-bit on that particular system. The issue with downcasting is that you might truncate/overflow the value in that log message. While I wouldn't expect rlim_t to be a such large number that it wouldn't fit in either uint32_t or unsigned, for some reason its exact type/width are not defined in Linux, FreeBSD and OpenBSD manpages, and some systems do use 64-bit unsigned integers for it, in fact one (or more) of our systems on Travis-CI does, so the fd number could technically be set to a 64-bit unsigned integer (or even 128 bit if some weird ABI with 128 bit integers comes out, as the manpages don't specify exact type/width of rlimit_t), so downcasting, while it would generally work, is not safe, it might result in us printing the wrong number.

nurupo · 2018-10-20T03:47:56Z

Looks like MSVC supports uintmax_t since at least 2013 version (there is no info on any earlier version, perhaps because they are discontinued).

The j in format string is not supported by MSVC though.

The hh, j, z, and t length prefixes are not supported.

nurupo · 2018-10-20T03:53:45Z

Oh, and for everyone reading this who is not iphy and thus probably doesn't understand the context, we kind of want to make tox-bootstrapd run on Windows in the future, possibly, maybe, so that's why we a bit concerned about MSVC despite it being linux/unix (posix?) daemon.

nurupo · 2018-10-20T04:11:21Z

Huh, this documentation for MSVC 2017 (and 2015?) C Runtime says that you can use j, so it's supported? But the other documentation for MSVC 2015 I linked in an earlier comment said that it isn't. Perhaps it's supported only starting MSVC 2017?

iphydf

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

other/bootstrap_daemon/src/tox-bootstrapd.c, line 452 at r1 (raw file):

Previously, nurupo wrote…

You realize that you are suggesting potentially downcasting, for example, 64-bit unsigned integer to 32-bit unsigned integer? For example, rlim_t can be defined as a 64-bit unsigned long longon a system, but we would downcast it to unsigned which is 32-bit on that particular system. The issue with downcasting is that you might truncate/overflow the value in that log message. While I wouldn't expect rlim_t to be a such large number that it wouldn't fit in either uint32_t or unsigned, for some reason its exact type/width are not defined in Linux, FreeBSD and OpenBSD manpages, and some systems do use 64-bit unsigned integers for it, in fact one (or more) of our systems on Travis-CI does, so the fd number could technically be set to a 64-bit unsigned integer (or even 128 bit if some weird ABI with 128 bit integers comes out, as the manpages don't specify exact type/width of rlimit_t), so downcasting, while it would generally work, is not safe, it might result in us printing the wrong number.

Yes. Yet I don't expect systems to have set a specific number greater than 4 billion that is not numeric_limits<rlim_t>::max(). Regardless, rlim is not going to work on Windows, so it doesn't really matter.

iphydf · 2018-10-20T10:51:04Z

Can you rebase on master?

nurupo · 2018-10-22T00:24:53Z

Regardless, rlim is not going to work on Windows, so it doesn't really matter.

Huh. Somehow I went along with

I'm not sure how portable that is - mingw32 seems to have some issues with it according to robinli

without realizing that this won't be an issue as we would compile out that portion of the code on Windows.

nurupo

Reviewable status: complete! 1 of 1 approvals obtained (waiting on @nurupo)

.travis/cmake-freebsd-stage1, line at r1 (raw file):

Previously, iphydf wrote…

Any idea why reviewable thinks this is a binary file? Are there any weird control characters in it accidentally?

There are control characters on lines 60 and 66, they are there intentionally.

other/bootstrap_daemon/tox-bootstrapd.sh, line 41 at r1 (raw file):

Previously, iphydf wrote…

"file descriptors"

Fixed.

nurupo · 2018-10-23T02:50:08Z

Removing the .travis.yml commit. Here is the last build with everything enabled https://travis-ci.org/TokTok/c-toxcore/builds/444894774.

nurupo · 2018-10-23T02:51:10Z

Oh, and rebased on master.

iphydf · 2018-11-04T23:20:14Z

Can you rebase again, and then merge?

tox-bootstrapd can use around 600 TCP sockets during TCP server's normal functioning. Many systems default to having a soft limit of 1024 open file descriptors, which we are close to reaching, so it was suggested we bump that limit to a higher number. iphy suggested increasing it to 32768.

nurupo · 2018-11-05T02:29:51Z

Rebased.

I can merge it, I guess.

git checkout master
git merge --ff-only bootstrapd-increase-nofile

right? Seems to produce the right result locally.

nurupo force-pushed the bootstrapd-increase-nofile branch 5 times, most recently from bea0e2f to 1bdc736 Compare October 18, 2018 09:40

nurupo force-pushed the bootstrapd-increase-nofile branch from 1bdc736 to 48e699f Compare October 19, 2018 05:07

iphydf approved these changes Oct 19, 2018

View reviewed changes

nurupo commented Oct 19, 2018

View reviewed changes

iphydf approved these changes Oct 19, 2018

View reviewed changes

nurupo commented Oct 20, 2018

View reviewed changes

iphydf reviewed Oct 20, 2018

View reviewed changes

iphydf added this to the v0.2.x milestone Oct 21, 2018

nurupo force-pushed the bootstrapd-increase-nofile branch 2 times, most recently from d220dc5 to e264a8d Compare October 23, 2018 02:16

nurupo commented Oct 23, 2018

View reviewed changes

nurupo force-pushed the bootstrapd-increase-nofile branch from e264a8d to 7fe6f0f Compare October 23, 2018 02:50

nurupo added 2 commits November 4, 2018 20:10

Build tox-bootstrapd on FreeBSD Travis-CI

d89f83f

nurupo force-pushed the bootstrapd-increase-nofile branch from 7fe6f0f to d89f83f Compare November 5, 2018 01:10

nurupo merged commit d89f83f into TokTok:master Nov 5, 2018

robinlinden modified the milestones: v0.2.x, v0.2.9 Jan 3, 2019

iphydf changed the title ~~Increase NOFILE limit for tox-bootstrapd~~ Increase NOFILE limit for tox-bootstrapd Apr 30, 2020

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Increase NOFILE limit for tox-bootstrapd #1234

Increase NOFILE limit for tox-bootstrapd #1234

nurupo commented Oct 18, 2018 •

edited by iphydf

Loading

CLAassistant commented Oct 18, 2018

codecov bot commented Oct 18, 2018 •

edited

Loading

iphydf left a comment

nurupo left a comment

iphydf left a comment

nurupo left a comment

nurupo commented Oct 20, 2018

nurupo commented Oct 20, 2018

nurupo commented Oct 20, 2018

iphydf left a comment

iphydf commented Oct 20, 2018

nurupo commented Oct 22, 2018

nurupo left a comment

nurupo commented Oct 23, 2018

nurupo commented Oct 23, 2018

iphydf commented Nov 4, 2018

nurupo commented Nov 5, 2018

Increase NOFILE limit for tox-bootstrapd #1234

Increase NOFILE limit for tox-bootstrapd #1234

Conversation

nurupo commented Oct 18, 2018 • edited by iphydf Loading

CLAassistant commented Oct 18, 2018

codecov bot commented Oct 18, 2018 • edited Loading

Codecov Report

iphydf left a comment

Choose a reason for hiding this comment

nurupo left a comment

Choose a reason for hiding this comment

iphydf left a comment

Choose a reason for hiding this comment

nurupo left a comment

Choose a reason for hiding this comment

nurupo commented Oct 20, 2018

nurupo commented Oct 20, 2018

nurupo commented Oct 20, 2018

iphydf left a comment

Choose a reason for hiding this comment

iphydf commented Oct 20, 2018

nurupo commented Oct 22, 2018

nurupo left a comment

Choose a reason for hiding this comment

nurupo commented Oct 23, 2018

nurupo commented Oct 23, 2018

iphydf commented Nov 4, 2018

nurupo commented Nov 5, 2018

nurupo commented Oct 18, 2018 •

edited by iphydf

Loading

codecov bot commented Oct 18, 2018 •

edited

Loading