Add optional packages jupyterlab, jupyterlab_widgets to support interacts in JupyterLab

[slel]

Before this ticket: interacts don't work in JupyterLab.

Reported at:

• Ask Sage question 50194: Interact in Jupyterlab vs Jupyter
• comment to an answer at Ask Sage question 52707

Related:

• Meta-ticket: improve compatibility with the Python library and Python packages #29728: Meta-ticket: improve compatibility with the Python library

CC: @slel

Component: interact

Keywords: jupyterlab

Author: Joshua Campbell

Branch: e169060

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/30246

[mkoeppe]

comment:1

Could this just be a documentation issue? Is it documented somewhere what extensions need to be installed in jupyterlab so that sage works?

[mkoeppe]

Dependencies: #26059

[jcamp0x2a]

comment:3

Replying to @mkoeppe:

Could this just be a documentation issue? Is it documented somewhere what extensions need to be installed in jupyterlab so that sage works?

I was able to get interacts working in jupyterlab 2.2.5 using the following command:
ipywidgets install instructions

./sage --jupyter labextension install @jupyter-widgets/[email protected]

This requires nodejs >= 10 to be installed though. Looks like jupyterlab's extension ecosystem is all based on npm. Grabbing the extension tarball directly, still need node for bundling. (webpack, if I remember; been awhile since I've done anything with node.)

So unless we want to make nodejs an optional->standard package as well, maybe just leave it to the user to install and make a note in the documentation regarding interact functionality? Similar to how imagemagick and ffmpeg are noted in animated plots.

[mkoeppe]

comment:4

Is installation via https://pypi.org/project/nodeenv/ an option?

[jcamp0x2a]

comment:5

Replying to @mkoeppe:

Is installation via https://pypi.org/project/nodeenv/ an option?

Cool stuff. I was indeed able to get interacts working without a system nodejs using ./sage --pip install nodeenv and creating a nodeenv to do the jupyter-widgets extension installation. Since node isn't needed for running jupyterlab, that nodeenv could probably get wiped out automatically following installation.

Not having node though does prevent you from browsing/installing other extensions in the extensions sidebar of jupyterlab. I played around with modifying the main sage script to activate the nodeenv when using ./sage -n jupyterlab and was able to install, for example, the jupyterlab-toc extension and use it without issue.

I'll continue experimenting with this for a bit longer, do some tidying up, and push a branch for review.

[jcamp0x2a]

Commit: d2d8c40

[jcamp0x2a]

Author: Joshua Campbell

[jcamp0x2a]

Branch: u/gh-jcamp0x2a/30246

[jcamp0x2a]

comment:6

I've pushed a branch for review. It adds 4 new optional packages: nodeenv, nodejs, jupyterlab, and jupyterlab_widgets. The last one, jupyterlab_widgets, gets you everything.

I opted to break them up in this way for (1) convenience in packaging them and (2) flexibility in installing them. For example, if you just want the JupyterLab client and don't care about interacts or other extensions or if you don't want JupyterLab but would like nodejs present in your Sage environment for some other purpose. Please let me know if you'd prefer a different structuring.

Shell scripting isn't my forte, so I welcome any feedback regarding coding conventions and best practices!

[jcamp0x2a]

comment:7

I suppose this also includes what should've been worked under #26059 first and separately (just the jupyterlab package). I apologize for that; got carried away :)

[slel]

Changed dependencies from #26059 to none

[slel]

comment:8

Replying to @jcamp0x2a:

includes what should've been worked under #26059
first and separately

We can mark the other ticket as solved by this one.

I would use https in the home page url for JupyterLab.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

c0b02c4	Fix sage-sh/-buildsh prompt getting wiped out by nodeenv
b7010f6	Use https url in doc for jupyterlab package

[sagetrac-git]

Changed commit from d2d8c40 to b7010f6

[mkoeppe]

comment:10

This looks great overall.

I'm getting the following with this branch (without installing any packages):

./sage -h
/Users/mkoeppe/s/sage/sage-rebasing/src/bin/sage-env: line 69: cd: /Users/mkoeppe/s/sage/sage-rebasing/local/share/nodejs/: No such file or directory
SageMath version 9.2.beta10, Release Date: 2020-08-23
...
  --notebook=[...]    -- start the Sage notebook (valid options are
                         'default', 'jupyter', and 'export')
                         Current default is 'export' from sagenb to jupyter
  -n, --notebook      -- shortcut for --notebook=default
...
$ ./sage -n jupyterlab
/Users/mkoeppe/s/sage/sage-rebasing/src/bin/sage-env: line 69: cd: /Users/mkoeppe/s/sage/sage-rebasing/local/share/nodejs/: No such file or directory
Jupyterlab is not installed (at least not in this Sage installation).
You can install it by running
  sage --pip install jupyterlab

[sagetrac-git]

Changed commit from b7010f6 to 4770d3b

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

4770d3b

Resolve nodeenv_activate symlink `_after_` checking that it exists

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

29359e1

Instruct user to install jupyterlab using -i instead of --pip install

[sagetrac-git]

Changed commit from 4770d3b to 29359e1

[jcamp0x2a]

comment:14

Thank you for catching that!

I've pushed a couple more changes, the first making sure to check that nodeenv_activate symlink exists before trying to resolve it, and the second mentioning to the user to install using sage -i jupyterlab instead of sage --pip install jupyterlab.

Oh, do you happen to know of a good way to uninstall Sage packages? I tested these last couple changes by manually removing/moving files around in SAGE_LOCAL. I'd like to get back to a state without, for example, jupyterlab for testing, but I'd also like to avoid having to do a full clean and rebuild. I suppose I could be doing this in a VM and just snapshot and rollback.

[sagetrac-tmonteil]

comment:15

This ticket breaks the patchbots (at least the ones i run). It seems due to some modification in src/bin/sage-env that tries to cd into a nonexistent local/share/nodejs directory. When the patchbots run:

./sage --python --version

They get:

/home/sagemath/sage-9.1/src/bin/sage-env: line 69: cd: /home/sagemath/sage-9.1/local/share/nodejs/: No such file or directory
Python 3.7.3

The error should be catched (e.g. stderr could be redirected to /dev/null).

[mkoeppe]

comment:19

Replying to @jcamp0x2a:

mentioning to the user to install using sage -i jupyterlab instead of sage --pip install jupyterlab.

Would it perhaps be better to direct the user to do sage -i jupyterlab_widgets so that a full-featured jupyterlab is installed?

[mkoeppe]

comment:20

Also, could you update the help messages in src/bin/sage so that jupyterlab is mentioned as an option. (Also I would guess that the default that is mentioned there is not correct.)

[mkoeppe]

comment:21

on macOS:

[nodejs-12.18.3] usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[nodejs-12.18.3] 	[-e pattern] [-f file] [--binary-files=value] [--color=when]
[nodejs-12.18.3] 	[--context[=num]] [--directories=action] [--label] [--line-buffered]
[nodejs-12.18.3] 	[--null] [pattern] [file ...]
[nodejs-12.18.3] Error determining which nodejs version to install ... exiting

[mkoeppe]

comment:22

grep -P is not portable

[jcamp0x2a]

comment:24

Replying to @mkoeppe:

Would it perhaps be better to direct the user to do sage -i jupyterlab_widgets so that a full-featured jupyterlab is installed?

I will update the message to suggest jupyterlab_widgets by default, with a short aside about a minimal installation using just jupyterlab.

Replying to @mkoeppe:

Also, could you update the help messages in src/bin/sage so that jupyterlab is mentioned as an option. (Also I would guess that the default that is mentioned there is not correct.)

I will add mention of jupyterlab to the help messages. I'm not sure I understand you about the default. The help messages I see list jupyter as the default, and indeed running sage -n or sage -n default starts jupyter.

Replying to @mkoeppe:

grep -P is not portable

I should know better. I remember this bit me on Solaris a few years ago, too. I will fix this.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

3f1aad9	Recommend jupyterlab_widgets, mention jupyterlab as minimal install
bcbccd6	Mention jupyterlab in basic and advanced help messages
0a06353	Don't use grep -P in nodejs spkg-install

[sagetrac-git]

Changed commit from 29359e1 to 0a06353

[jcamp0x2a]

comment:26

Upon re-reading the basic help message, I saw what you mentioned about the default needing to be updated. I've pushed changes that I believe address your feedback items.

I also came across sage --notebook --help which had already mentioned jupyterlab and so didn't need updating, but I found that it also mentions an ipython option:

--notebook [NOTEBOOK], -n [NOTEBOOK], -notebook [NOTEBOOK]
                      The notebook to run [one of: default, ipython,
                      jupyter, jupyterlab, export]. Default is jupyter

Looking at the code in src/bin/sage-notebook, this option is currently just an alias for jupyter. Would it be valuable to mention this option in the basic and/or advanced help messages for sage as well?

[mkoeppe]

Changed branch from u/gh-jcamp0x2a/30246 to u/mkoeppe/30246

[mkoeppe]

Reviewer: Matthias Koeppe

[mkoeppe]

comment:28

This works for me on macOS and is looking great overall.

I have added dependencies to make sure that a parallel build works correctly.

---

New commits:

e169060

build/pkgs/{jupyterlab_widgets,nodeenv}: Add dependencies

[mkoeppe]

Changed commit from 0a06353 to e169060

[jcamp0x2a]

comment:30

Replying to @mkoeppe:

This works for me on macOS and is looking great overall.

I have added dependencies to make sure that a parallel build works correctly.

Awesome! Thank you for all your time and assistance on this ticket.

[vbraun]

Changed branch from u/mkoeppe/30246 to e169060

[EmmanuelCharpentier]

comment:32

Minor booboo : after (successful) installlation, jupyterlab_widgets is still marked as not installed:

charpent@zen-book-flip:/usr/local/sage-exp$ ./sage -optional | grep jupyter
gap_jupyter.............................0.9 (0.9)
jupyterlab..............................2.2.6 (2.2.6)
jupyterlab_widgets......................none (2.0)
pari_jupyter............................1.3.2 (1.3.2)
r_jupyter...............................none (not_installed)
singular_jupyter........................0.9.7 (0.9.7)

Untidy, and potentially confusing...

[EmmanuelCharpentier]

Changed commit from e169060 to none

[jcamp0x2a]

comment:33

I believe the version in parentheses is the installed version while the other is the available version. I'm not sure why the available version isn't displayed since there is a package-version.txt present for jupyterlab_widgets, but I'm thinking this may be a problem for "script" packages in general. For example:

$ ./sage --installed | grep none
jupyterlab_widgets......................none (2.0)
nodejs..................................none (12.18.3)
sage_conf...............................none (none)
sagelib.................................none (9.2.beta10)
zope_interface..........................none (none)

[dimpase]

comment:34

Has anyone figured out how to use MathJax in JupyterLab markdown cells?