-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rustboard's GLIBC dependency is too high, need to lower GLIBC requirement #6578
Comments
I see an error message that I think comes from the same issue (but I'm not using rustboard AFAIK). I'm on Ubuntu 20.04, using python3.11. This ubuntu version has glibc 2.31 as you mentioned: $ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.7) 2.31
... Using But using any of these more recent versions: $ tensorboard
TensorFlow installation not found - running with reduced feature set.
tensorboard --logdir checkpoints/tutorial8/tensorboards
/.../venv/lib/python3.11/site-packages/tensorboard_data_server/bin/server: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /.../venv/lib/python3.11/site-packages/tensorboard_data_server/bin/server)
/.../venv/lib/python3.11/site-packages/tensorboard_data_server/bin/server: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /.../venv/lib/python3.11/site-packages/tensorboard_data_server/bin/server)
/.../venv/lib/python3.11/site-packages/tensorboard_data_server/bin/server: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /.../venv/lib/python3.11/site-packages/tensorboard_data_server/bin/server)
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.12.3 at http://localhost:6006/ (Press CTRL+C to quit) Even though these errors are printed, I can still access the tensorboard server at I found this old issue that seems somewhat related: #4928 If this is actually an error (and not just a warning/something I can ignore) - could I have detected this incompatibility somehow? |
@nik-sm As I wrote in #4784 (comment), rustboard ("fast" mode) can't run in your situation, so tensorboard is automatically falling back to painfully slow python implementations. You can see the plots and TB runs OK but data loading speed will be very, very, very slow. If you add |
Ok I understand now - thanks for explaining. I'll stick to a version that's compatible with Ubuntu 20.04 / libc 2.31 for now. I think this issue arose because the wheel provided for tensorboard 2.12, 2.13, 2.14 is not named correctly (https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#manylinux, https://peps.python.org/pep-0600/). For example, maybe it should use (Tensorboard still does run even with an older libc version. You've explained it will be extremely slow and I should avoid this situation, so I guess I'm not sure whether one would say that "Tensorboard is technically compatible with libc 2.31"...) |
The produced tensorboard-data-server wheels do specify manylinux2014, but the tensorboard wheels do not. Looks like 0.6.1 wheels specify Would either of you be able to test running the rustboard binary directly (i.e. not via the
|
Hi - I ran the commands above, but I get the same error message:
Let me know if you'd like me to try anything else. |
Thanks, confirmed that this is due to the
|
Thanks @groszewn, I can confirm that the minimum GLIBC version for tensorboard-data-server==0.7.2 is 2.29 and it works well on Ubuntu 20.04 (18.04 is not supported).
For record, related to #6636 as a part of TensorBoard 2.15. (The actual fix comes from the internal release process) |
From #4784 (comment)
The rustboard extension (
--load_fast=True
) of Tensorboard 2.12 ~ 2.14 cannot run on old(?) linux systems, including Ubuntu 20.04, because of the GLIBC dependency >= 2.34. I think this is a too high or strict requirement; Ubuntu 20.04 LTS is still popular in many dev environments and should still be supported. I request that the minimum required GLIBC version should be at most 2.31, but the lower the better as long as we don't lose any performance (e.g., as low as 2.18).This is probably due to the base image of CI being bumped up to 22.04 from 20.04 (for previous releases) -- see #5992 (/cc @bmd3k). While I find it may cause another problem if we revert it back to 20.04, we could instead link against an old version of glibc for the sake of best compatibility. Possible approaches include adding
__asm__
directives, use a cross-compile toolchain, etc.Ref)
Some useful facts, copied from #4784:
tensorboard_data_server>=0.7
requires GLIBC>=2.34:tensorboard_data_server==0.6.1
(tensorboard 2.11) requires GLIBC>=2.18We could even release a minor/maintenance release (say tensorboard_data_server 0.7.1) that relaxes GLIBC requirement without an need to wait until the next TB release cycle.
This can be done mostly on the CI side, so it'd be much convenient to be done by Googlers. But I'd be happy to contribute if any help is needed from my side.
The text was updated successfully, but these errors were encountered: