Enable WIT usage in JupyterLab/Cloud AI Platform notebooks #2301

jameswex · 2019-06-03T13:52:20Z

Motivation for features / changes

Update WitWidget packaging/logic to enable it to work in JupyterLab notebooks, which handle extensions like WitWidget differently than standard Jupyter notebooks. This also enables use in Cloud AI Platform notebooks.

Technical description of changes

Update packaging code to create correct, public NPM package, as the HTML/JS side of lab extensions are NPM packages.
Update witwidget jupyter js code to handle loading wit_jupyter.html from correct packaged location when running in JupyterLab, while keeping the same initial logic for loading that file when in standard Jupyter, as they are different paths.
Update package building script and webpack config to correctly bundle wit_jupyter.html with the NPM package for the JupyterLab extension.
Update documentation.

Screenshots of UI changes

No visual changes.

Detailed steps to verify changes work correctly (as executed by you)

Installed/ran WIT in Cloud AI Platform notebooks (which are JupyterLab notebooks).
Installed/ran WIT in standard Jupyter notebooks to ensure this change didn't break that case.

jameswex · 2019-06-03T15:15:52Z

@tolga-b please review

tafsiri · 2019-06-03T15:39:26Z

tensorboard/plugins/interactive_inference/README.md

+Install and enable WIT for JupyterLab by running a cell containing:
+```
+!pip install witwidget
+!jupyter labextension install wit-widget


Check out npm postinstall scripts, it might be able to do these steps. (but your script will need to be compatible with all the environments it could run—or check which environment it is in).

i'm going to leave this as-is for now, as in some environments it may need to be done with sudo (cloud ai platform notebooks), but not in your own jupyterlab instance.

tafsiri · 2019-06-03T15:44:38Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

@@ -13,6 +13,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 ==============================================================================*/

+var witHtmlLocation = require('file-loader!./wit_jupyter.html');


I'd double check to see if you need the file-loader! part. Given your webpack config i wouldn't think that you would need it.

the jupyter labextension install fails if i leave it out:

ERROR in ./node_modules/wit-widget/lib/wit_jupyter.html 1:0
Module parse failed: Unexpected token (1:0)

Agreed with @tafsiri—perhaps you need to add module: {rules: rules} to
the notebook extension config, as in the other two configs?

i tried that as well - no luck :(

Hmm, okay; strange, but I’ll let it be.

tafsiri · 2019-06-03T15:44:43Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

+
+    // Adjust WIT html location if running in a jupyter notebook
+    // and not in jupyterlab.
+    if (document.querySelector('body').getAttribute('data-base-url') != null) {


do you know what the value of data-base-url would be? would that make it a more robust check? I admit this is a hard thing to test. Is there a version string somewhere that might have the project name or something.

I believe it may depend on your own Jupyter installation, so I'm not looking to test a specific value of that field, just that it exists

tafsiri · 2019-06-03T15:44:46Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

+    // Adjust WIT html location if running in a jupyter notebook
+    // and not in jupyterlab.
+    if (document.querySelector('body').getAttribute('data-base-url') != null) {
+      witHtmlLocation = window.__webpack_public_path__ + 'wit_jupyter.html';


I'd recommend renaming __webpack_public_path__ to something like __nbextension_path__.

tafsiri · 2019-06-03T15:46:50Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/package.json

@@ -16,8 +14,8 @@
    "jupyterlab-extension"
  ],
  "files": [
-    "lib/**/*.js",
-    "dist/*.js"
+    "lib/*",


totally up to you, but longterm might be nice to have both of them use dist? (Though there may be valid reasons to have them use diff sources/builds)

done, removed lib

was actually mistaken about this - labextensions fail to install unless the npm package contains lib/labplugin.js - added it back in

tafsiri · 2019-06-03T15:46:52Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/package.json

-    "@jupyter-widgets/base": "^1.0.0"
+    "@jupyter-widgets/base": "^1.0.0",
+    "@jupyter-widgets/jupyterlab-manager": "^0.38.1",
+    "file-loader": "^3.0.1"


I think this can go in devDependencies

tafsiri

LGTM

wchargin · 2019-06-04T20:15:02Z

tensorboard/plugins/interactive_inference/README.md

+!jupyter labextension install wit-widget
+!jupyter labextension install @jupyter-widgets/jupyterlab-manager
+```
+For TensorFlow GPU support, use the witwidget-gpu package instead of witwidget.


nit: Code spans around witwidget-gpu, witwidget, and the !sudo
command on the next line.

wchargin · 2019-06-04T20:19:35Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

@@ -13,6 +13,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 ==============================================================================*/

+var witHtmlLocation = require('file-loader!./wit_jupyter.html');


Agreed with @tafsiri—perhaps you need to add module: {rules: rules} to
the notebook extension config, as in the other two configs?

wchargin · 2019-06-04T20:21:55Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

+
+    // Adjust WIT html location if running in a jupyter notebook
+    // and not in jupyterlab.
+    if (document.querySelector('body').getAttribute('data-base-url') != null) {


Superfluous querySelector (just use document.body). If you like, you
can also use document.body.dataset.baseUrl instead of getAttribute,
but up to you.

changed to document.body

wchargin · 2019-06-04T20:24:52Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

+      witHtmlLocation = window.__nbextension_path__ + 'wit_jupyter.html';
+    }
+
+    const src = `<link rel="import" href="${witHtmlLocation}">


Please note that this will break in Chrome M75, which is scheduled to
be released… uh, today. Maybe test with Chrome beta/canary?

See my previous comment for migration instructions.

thanks. will handle this in separate PR ASAP

Sounds good; thanks!

I tested WIT in colab in chrome canary (which is running 77.0.3815.0) and it works as currently implemented.

wchargin · 2019-06-04T20:28:46Z

tensorboard/plugins/interactive_inference/witwidget/pip_package/RELEASE.md

 at https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives.
-4. Commit the version.py change.
+5. Publish the new NPM package through running "npm publish" in the "js/" subdirectory of the package files generated in step 3.


nit: Code spans: `npm publish`; `js/`.

wchargin · 2019-06-04T20:32:15Z

tensorboard/plugins/interactive_inference/witwidget/pip_package/RELEASE.md

 at https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives.
-4. Commit the version.py change.
+5. Publish the new NPM package through running "npm publish" in the "js/" subdirectory of the package files generated in step 3.
+6. Commit the version changes.


Shouldn’t the process be to commit and merge any version changes
before publishing to PyPI and NPM, then always publish from a
checked-in, known-good commit? Publishing immutable archives from a
dirty work tree doesn’t sound like something that we want to encourage.
Not only are you vulnerable to typos or bad local state, but others
might not be able to reliably reproduce the built artifacts. It’s much
simpler if the actual deployment process is just “clone fresh; check out
some fixed commit; run npm publish, et al.”.

sounds better. updated instructions.

wchargin · 2019-06-05T17:23:50Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

@@ -13,6 +13,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 ==============================================================================*/

+var witHtmlLocation = require('file-loader!./wit_jupyter.html');


Hmm, okay; strange, but I’ll let it be.

wchargin · 2019-06-05T17:24:11Z

tensorboard/plugins/interactive_inference/witwidget/notebook/jupyter/js/lib/wit.js

+      witHtmlLocation = window.__nbextension_path__ + 'wit_jupyter.html';
+    }
+
+    const src = `<link rel="import" href="${witHtmlLocation}">


Sounds good; thanks!

jameswex added 5 commits May 31, 2019 09:59

works in lab

b97bda6

update

2f379ce

fix

452868e

readme

b0380d7

readme updates

27c2ab7

tafsiri reviewed Jun 3, 2019

View reviewed changes

yannick comments

ce76dca

tafsiri approved these changes Jun 4, 2019

View reviewed changes

jameswex requested a review from wchargin June 4, 2019 19:40

wchargin reviewed Jun 4, 2019

View reviewed changes

review comments

833819c

wchargin approved these changes Jun 5, 2019

View reviewed changes

jameswex merged commit dcda58b into tensorflow:master Jun 5, 2019

jameswex deleted the lab branch June 5, 2019 17:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable WIT usage in JupyterLab/Cloud AI Platform notebooks #2301

Enable WIT usage in JupyterLab/Cloud AI Platform notebooks #2301

jameswex commented Jun 3, 2019

jameswex commented Jun 3, 2019

tafsiri Jun 3, 2019

jameswex Jun 4, 2019

tafsiri Jun 3, 2019

jameswex Jun 4, 2019

wchargin Jun 4, 2019

jameswex Jun 5, 2019

wchargin Jun 5, 2019

tafsiri Jun 3, 2019

jameswex Jun 4, 2019

tafsiri Jun 3, 2019

jameswex Jun 4, 2019

tafsiri Jun 3, 2019

jameswex Jun 4, 2019

jameswex Jun 5, 2019

tafsiri Jun 3, 2019

jameswex Jun 4, 2019

tafsiri left a comment

wchargin Jun 4, 2019

jameswex Jun 5, 2019

wchargin Jun 4, 2019

wchargin Jun 4, 2019

jameswex Jun 5, 2019

wchargin Jun 4, 2019

jameswex Jun 5, 2019

wchargin Jun 5, 2019

jameswex Jun 6, 2019

wchargin Jun 4, 2019

jameswex Jun 5, 2019

wchargin Jun 4, 2019

jameswex Jun 5, 2019

wchargin Jun 5, 2019

wchargin Jun 5, 2019

Enable WIT usage in JupyterLab/Cloud AI Platform notebooks #2301

Enable WIT usage in JupyterLab/Cloud AI Platform notebooks #2301

Conversation

jameswex commented Jun 3, 2019

jameswex commented Jun 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tafsiri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment