Tensorboard Eval Images with TF-Vision

[RayanMoarkech]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• I am reporting the issue to the correct repository. (Model Garden official or research directory)
• I checked to make sure that this issue has not been filed already.

1. The entire URL of the file you are using

https://www.tensorflow.org/tfmodels/vision/object_detection#load_logs_in_tensorboard

2. Describe the bug

I am following this documentation, https://www.tensorflow.org/tfmodels/vision/object_detection#load_logs_in_tensorboard
When I open tensorboard, and select images, I get "No image data was found."

I also tried to add EXPERIMENT_CONFIG.task.allow_image_summary = True, but I got an error, even with the dataset and code given by the documentation.

The error:

ValueError: Expected scalar shape, saw shape: (1, 640, 640, 3).

The code:

model, eval_logs = tfm.core.train_lib.run_experiment(
    distribution_strategy=distribution_strategy,
    task=task,
    mode='train_and_eval',
    params=EXPERIMENT_CONFIG,
    model_dir=paths['MODEL_CHECKPOINT_PATH'],
    run_post_eval=True,
)

3. Steps to reproduce

• Following this documentation: https://www.tensorflow.org/tfmodels/vision/object_detection#load_logs_in_tensorboard
• After training, open tensorboard, and select images
• See "No image data was found."

Now, try to train again with

• EXPERIMENT_CONFIG.task.allow_image_summary = True
• see error:

ValueError: Expected scalar shape, saw shape: (1, 640, 640, 3).

4. Expected behavior

I would like to see the evaluated images per epochs saved on tensorboard.

5. Additional context

Let me know if you need anything extra

6. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS 15.0
• Mobile device name if the issue happens on a mobile device: N/A
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): v2.17.0-rc1-2-gad6d8cc177d 2.17.0
• Python version: 3.10
• Bazel version (if compiling from source): N/A
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A -> using CPU

[bharatjetti]

Hi @RayanMoarkech
I worked on the problem and reproduced the issue of No image data was found, I added piece of code to the existing,
i.e in the show_batch function added this line and made necessary changes.

with summary_writer.as_default():
tf.summary.image(f'Image_with_bboxes_{i+1}', np.expand_dims(image, axis=0), step=train_steps)

and I found that it is working, here is the notebook that I worked on. Please check it here is the screenshot.

[RayanMoarkech]

But this will not create an image log at every summary interval while training the model with:

tfm.core.train_lib.run_experiment

Correct me if I'm wrong. But I was not able to connect the training to produce an image summary at the same time I am doing a summary_interval. So this means this option is only a manual code that I should run on every train step I want to stop at?

Based on your screenshot, you can see the data is from a . RUN

[bharatjetti]

Hi @RayanMoarkech,
It seems there is no issue with model training and we can observe the results on few example images. However to get the entire image summary automatically, Could you please raise it in tensorboard repo, please feel free to close this issue.