BUG: Startup Failure in jupyter-ai Due to Missing Persona Class in jupyter-ai-magics Dependency

[suzuki-shm]

Description

As of version 2.13.0, jupyter-ai fails to launch. The issue stems from the Persona class that is newly utilized in jupyter-ai-magics, which jupyter-ai depends on. Currently, jupyter-ai does not specify a version dependency for jupyter-ai-magics, so version 2.2 gets installed by default. However, this version does not define the Persona class, leading to an error message during startup.

Reproduce

I tested this using Python 3.11 + poetry.

1. mkdir jupyterai-dev
2. poetry init
3. poetry add jupyter-ai
4. poetry run jupyter lab

Expected behavior

The issue is resolved if jupyter-ai-magics version 2.13.0 or higher is installed.

Context

• Operating System and version: Windows10 + WSL2 Ubuntu 20.04.6 LTS
• Browser and version: Edge for Business 122.0.2365.52
• JupyterLab version: 4.1.6
• Python version: 3.11.1
• Poetry version: 1.5.0

Command Line Output

[D 2024-04-12 15:40:56.486 ServerApp] Searching ['/home/USER_NAME/jupyterai-dev/.venv/etc/jupyter', '/home/USER_NAME/.jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 2024-04-12 15:40:56.487 ServerApp] Looking for jupyter_config in /etc/jupyter
[D 2024-04-12 15:40:56.487 ServerApp] Looking for jupyter_config in /usr/local/etc/jupyter
[D 2024-04-12 15:40:56.487 ServerApp] Looking for jupyter_config in /home/USER_NAME/.jupyter
[D 2024-04-12 15:40:56.487 ServerApp] Looking for jupyter_config in /home/USER_NAME/jupyterai-dev/.venv/etc/jupyter
[D 2024-04-12 15:40:56.488 ServerApp] Looking for jupyter_server_config in /etc/jupyter
[D 2024-04-12 15:40:56.488 ServerApp] Looking for jupyter_server_config in /usr/local/etc/jupyter
[D 2024-04-12 15:40:56.489 ServerApp] Looking for jupyter_server_config in /home/USER_NAME/.jupyter
[D 2024-04-12 15:40:56.489 ServerApp] Looking for jupyter_server_config in /home/USER_NAME/jupyterai-dev/.venv/etc/jupyter
[W 2024-04-12 15:40:58.393 ServerApp] jupyter_ai | error adding extension (enabled: True): The module 'jupyter_ai' could not be found (cannot import name 'Persona' from 'jupyter_ai_magics' (/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_ai_magics/__init__.py)). Are you sure the extension is installed?
    Traceback (most recent call last):
      File "/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 323, in add_extension
        extpkg = ExtensionPackage(name=extension_name, enabled=enabled)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 187, in __init__
        self._load_metadata()
      File "/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 202, in _load_metadata
        raise ExtensionModuleNotFound(msg) from None
    jupyter_server.extension.utils.ExtensionModuleNotFound: The module 'jupyter_ai' could not be found (cannot import name 'Persona' from 'jupyter_ai_magics' (/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_ai_magics/__init__.py)). Are you sure the extension is installed?
[I 2024-04-12 15:40:58.405 ServerApp] jupyter_lsp | extension was successfully linked.
[D 2024-04-12 15:40:58.409 TerminalsExtensionApp] Config changed: {'ServerApp': {'log_level': 'DEBUG', 'jpserver_extensions': }, 'ExtensionApp': {'log_level': 'DEBUG'}}
[D 2024-04-12 15:40:58.410 ServerApp] Config changed: {'ServerApp': {'log_level': 'DEBUG', 'jpserver_extensions': }, 'ExtensionApp': {'log_level': 'DEBUG'}}
[I 2024-04-12 15:40:58.411 ServerApp] jupyter_server_terminals | extension was successfully linked.
[D 2024-04-12 15:40:58.416 LabApp] Config changed: {'NotebookApp': {}, 'ServerApp': {'use_redirect_file': False, 'log_level': 'DEBUG', 'jpserver_extensions': }, 'ExtensionApp': {'log_level': 'DEBUG'}}
[I 2024-04-12 15:40:58.417 ServerApp] jupyterlab | extension was successfully linked.
[W 2024-04-12 15:40:58.421 ServerApp] jupyter_ai | error adding extension (enabled: True): The module 'jupyter_ai' could not be found (cannot import name 'Persona' from 'jupyter_ai_magics' (/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_ai_magics/__init__.py)). Are you sure the extension is installed?
    Traceback (most recent call last):
      File "/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 323, in add_extension
        extpkg = ExtensionPackage(name=extension_name, enabled=enabled)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 187, in __init__
        self._load_metadata()
      File "/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 202, in _load_metadata
        raise ExtensionModuleNotFound(msg) from None
    jupyter_server.extension.utils.ExtensionModuleNotFound: The module 'jupyter_ai' could not be found (cannot import name 'Persona' from 'jupyter_ai_magics' (/home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyter_ai_magics/__init__.py)). Are you sure the extension is installed?
[I 2024-04-12 15:40:58.422 ServerApp] notebook_shim | extension was successfully linked.
[D 2024-04-12 15:40:58.423 ServerApp] Config changed: {'ExtensionApp': {'log_level': 'DEBUG'}, 'NotebookApp': {}, 'ServerApp': {'use_redirect_file': False, 'log_level': 'DEBUG', 'jpserver_extensions': }}
[D 2024-04-12 15:40:58.423 ServerApp] Raising open file limit: soft 1024->4096; hard 1048576->1048576
[I 2024-04-12 15:40:58.434 ServerApp] notebook_shim | extension was successfully loaded.
[D 2024-04-12 15:40:58.436 ServerApp] [lsp] rootUri will be file:///home/USER_NAME/jupyterai-dev
[D 2024-04-12 15:40:58.436 ServerApp] [lsp] virtualDocumentsUri will be file:///home/USER_NAME/jupyterai-dev/.virtual_documents
[I 2024-04-12 15:40:58.436 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-04-12 15:40:58.437 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-04-12 15:40:58.438 LabApp] JupyterLab extension loaded from /home/USER_NAME/jupyterai-dev/.venv/lib/python3.11/site-packages/jupyterlab
[I 2024-04-12 15:40:58.438 LabApp] JupyterLab application directory is /home/USER_NAME/jupyterai-dev/.venv/share/jupyter/lab
[I 2024-04-12 15:40:58.439 LabApp] Extension Manager is 'pypi'.
[D 2024-04-12 15:40:58.439 LabApp] Plugins in PyPIExtensionManager will managed on the sys_prefix level
[D 2024-04-12 15:40:58.445 LabApp] Extensions list will be fetched from https://pypi.org/pypi.
[D 2024-04-12 15:40:58.445 LabApp] Plugins in PluginManager will managed on the sys_prefix level
[I 2024-04-12 15:40:58.447 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-04-12 15:40:58.447 ServerApp] Serving notebooks from local directory: /home/USER_NAME/jupyterai-dev
[I 2024-04-12 15:40:58.448 ServerApp] Jupyter Server 2.14.0 is running at:
[I 2024-04-12 15:40:58.448 ServerApp] http://localhost:8888/lab?token=0e813e0a70c0231df733b454076c6e2e556d5e5bab7b8106
[I 2024-04-12 15:40:58.448 ServerApp]     http://127.0.0.1:8888/lab?token=0e813e0a70c0231df733b454076c6e2e556d5e5bab7b8106
...
[W 2024-04-12 15:41:08.194 ServerApp] 404 GET /api/ai/chats?token=[secret] ([email protected]) 1.46ms referer=None
[D 2024-04-12 15:41:08.446 ServerApp] Using contents: services/contents

Browser Output

Skipped.

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[suzuki-shm]

When you open Jupyter in this state, the widgets appear as shown in the following screenshot.

[krassowski]

It looks a version pin needs to be added in:

jupyter-ai/packages/jupyter-ai/pyproject.toml

Line 29 in c12b7fd

"jupyter_ai_magics",