Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proper TensorBoard Usage #147

Open
ghost opened this issue Feb 11, 2023 · 0 comments
Open

Proper TensorBoard Usage #147

ghost opened this issue Feb 11, 2023 · 0 comments

Comments

@ghost
Copy link

ghost commented Feb 11, 2023

TensorBoard Usage

Hello, can someone help me properly use tensorboard to monitor the training process of human_seg? I am new to tensorboard so I apologize if this is a dumb question.

I am running python3.8-dev and python3.8-venv

Steps to reproduce error:

create python environment:

python3.8 -m venv PythonEnv/mesh
source !$/bin/activate

clone repository:

git clone --recurse-submodules https://github.com/ranahanocka/MeshCNN
cd MeshCNN

install dependencies:

pip install --upgrade pip
pip install cython==0.29.33
pip install numpy==1.23
pip install torchvision==0.14.1
pip install tensorboard

collect dataset:

source ./scripts/human_seg/get_data.sh

begin training:

source ./scripts/human_seg/train.sh

However, when I run tensorboard with:

tensorboard --host 0.0.0.0 --logdir=runs

I get the following error:

Screen Shot 2023-02-11 at 12 35 07 PM

And the result from bash:

tensorboard --host 0.0.0.0 --logdir=runs
2023-02-11 13:12:07.977485: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-11 13:12:08.505366: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-02-11 13:12:08.505415: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-02-11 13:12:08.505423: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-02-11 13:12:08.902349: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-02-11 13:12:08.902530: I tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2023-02-11 13:12:08.921833: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2023-02-11 13:12:08.921861: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...

NOTE: Using experimental fast data loading logic. To disable, pass
    "--load_fast=false" and report issues on GitHub. More details:
    https://github.com/tensorflow/tensorboard/issues/4784

TensorBoard 2.11.2 at http://0.0.0.0:6006/ (Press CTRL+C to quit)

Can someone help?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants