Skip to content

"Failed to load plugin WhatIfToolPluginLoader" caused by protobuf compatibility error #6195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nfelt opened this issue Feb 10, 2023 · 1 comment

Comments

@nfelt
Copy link
Contributor

nfelt commented Feb 10, 2023

The current release of the What-If Tool plugin includes generated Python protos that are incompatible with the 4.x protobuf runtime, leading to this error at startup when the virtual environment only has the 4.x protobuf runtime:

E0210 14:56:01.342828 139767216988224 application.py:125] Failed to load plugin WhatIfToolPluginLoader.load; ignoring it.
Traceback (most recent call last):
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard/backend/application.py", line 123, in TensorBoardWSGIApp
    plugin = loader.load(context)
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/wit_plugin_loader.py", line 57, in load
    from tensorboard_plugin_wit.wit_plugin import WhatIfToolPlugin
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/wit_plugin.py", line 40, in <module>
    from tensorboard_plugin_wit._utils import common_utils
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/_utils/common_utils.py", line 17, in <module>
    from tensorboard_plugin_wit._vendor.tensorflow_serving.apis import classification_pb2
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/_vendor/tensorflow_serving/apis/classification_pb2.py", line 15, in <module>
    from tensorboard_plugin_wit._vendor.tensorflow_serving.apis import input_pb2 as tensorflow__serving_dot_apis_dot_input__pb2
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/tensorboard_plugin_wit/_vendor/tensorflow_serving/apis/input_pb2.py", line 37, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/local/google/home/nickfelt/.tf-venvs/tf-nightly-py3/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
TensorBoard 2.12.0a20230209 at http://xn--260a.mtv.corp.google.com:6006/ (Press CTRL+C to quit)

Note: this doesn't prevent TensorBoard overall from running, it only leads to the What-If Tool plugin being unavailable, but it's a large and scary error message that users are likely to notice.

We didn't notice this previously because TensorBoard itself had generated protos incompatible with the 4.x runtime (issue #5703), but that restriction has recently been lifted with #6147 and released in TensorBoard 2.12. For users who get tensorboard as a dependency of tensorflow, this won't yet affect them because TensorFlow also has required the 3.x protobuf runtime, but the soon-to-be-released TF 2.12 require the 4.x runtime, which will likely result in many more users noticing the error and not having a good workaround.

The solution is for the What-If Tool plugin to release a new version with protos generated with protoc 3.19 or higher.

@jameswex
Copy link
Contributor

jameswex commented Mar 1, 2023

As discussed, with WIT being no longer actively developed, we will remove it from the TB package. Users can still use it in standalone mode in notebooks outside of TensorBoard, or can move over to using the actively-developed LIT tool https://pair-code.github.io/lit/

groszewn added a commit to groszewn/tensorboard that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead.

Contributes to tensorflow#6195
groszewn added a commit to groszewn/tensorboard that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead.

Contributes to tensorflow#6195
groszewn added a commit to groszewn/tensorboard that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead.

Contributes to tensorflow#6195
groszewn added a commit to groszewn/tensorboard that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead.

Contributes to tensorflow#6195
groszewn added a commit to groszewn/tensorboard that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead.

Contributes to tensorflow#6195
groszewn added a commit to groszewn/tensorboard that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead.

Contributes to tensorflow#6195
groszewn added a commit that referenced this issue Apr 24, 2023
The WIT is no longer actively maintained, so redirect users to install
and use the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead via a dynamic plugin. I
attempted to get the WIT package building again to push out an updated
release that does this redirection outside of the TB code, but that
proved more difficult than expected.

The resulting dashboard can be seen below, which links to the [LIT
homepage](https://pair-code.github.io/lit/) and [setup
guide](https://pair-code.github.io/lit/setup/), respectively.


![image](https://user-images.githubusercontent.com/13921855/234000839-7affc80a-dc89-478f-8a9e-98e5522cc456.png)

Contributes to #6195
dna2github pushed a commit to dna2fork/tensorboard that referenced this issue May 1, 2023
The WIT is no longer actively maintained, so redirect users to install
and use the newer [Learning Interoperability Tool
(LIT)](https://pair-code.github.io/lit/) instead via a dynamic plugin. I
attempted to get the WIT package building again to push out an updated
release that does this redirection outside of the TB code, but that
proved more difficult than expected.

The resulting dashboard can be seen below, which links to the [LIT
homepage](https://pair-code.github.io/lit/) and [setup
guide](https://pair-code.github.io/lit/setup/), respectively.


![image](https://user-images.githubusercontent.com/13921855/234000839-7affc80a-dc89-478f-8a9e-98e5522cc456.png)

Contributes to tensorflow#6195
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants