Layout builder example not working #6600

MarcSkovMadsen · 2024-03-28T09:38:02Z

I was trying to use the layout builder with example in the how-to guide https://panel.holoviz.org/how_to/notebook/layout_builder.html.

But it does not work for me as the matplotlib plot is not displayed??

panel==1.4.0 bokeh=3.4.0 pyviz_comms=3.0.2

MarcSkovMadsen · 2024-03-28T09:44:44Z

When I restart jupyterlab and preview again it looks like

If I drag the panels I can see content in the 3 first ones. But I have no way to reset the configuration from the UI. This will make it really hard for newbies.

I tried clearing the notebook ("Kernel" - "Restart and clear" + "Edit" - "Clear output of all cells") . But it did not help.

MarcSkovMadsen · 2024-03-28T09:55:37Z

If I add matplotlib.use('agg') the plot displays.

But if I click either the refresh or save button nothing visible happens. The buttons don't have any visual effect when clicked. And I don't see anything refreshing or saved? The buttons have no tooltips that can explain to me what I should expect.

MarcSkovMadsen · 2024-03-28T09:59:30Z

The natural question is also why do I have two "refresh" buttons? What is the difference?

MarcSkovMadsen · 2024-03-28T10:03:08Z

A comment is also that the guide is a mix of a tutorial, how-to guide and an explanation. Its not really clear what it is. As a tutorial it lacks content and is hard for users (like me) to reproduce.

My recommendation would be to make it a tutorial. The target audience is newbees and non-panel users that want to share their notebooks as dashboards. So I believe it should be really, really detailed in the descriptions such that users "cannot fail".

MarcSkovMadsen · 2024-03-28T10:06:56Z

Ahh. I can now see that the new, second "refresh" button is not a refresh button but a "clear settings button". Something happened when the matplotlib pane did not take up the full width.

MarcSkovMadsen · 2024-03-28T10:11:30Z

Now I start getting the error below :-) And matplotlib will not display in the notebook.

philippjfr · 2024-03-28T11:17:22Z

Good feedback, thanks. On balance I think a video is probably the best way to convey how to use it.

But it does not work for me as the matplotlib plot is not displayed??

This is very surprising to me, the code forces the backend to be agg.

Now I start getting the error below :-) And matplotlib will not display in the notebook.

Now that you can't blame on Panel 😄

philippjfr · 2024-03-28T11:18:20Z

Something happened when the matplotlib pane did not take up the full width.

Matplotlib plots are set to scale, if you forced it to fill the space you'd have to distort it horribly.

MarcSkovMadsen · 2024-03-28T11:18:21Z

But I end up in a state like this and cannot solve. That is on Panel to solve.

bad-state.mp4

philippjfr · 2024-03-28T11:19:21Z

Could you look at your browser console? Are there any weird errors?

MarcSkovMadsen · 2024-03-28T11:19:56Z

Could you look at your browser console? Are there any weird errors?

There are always weird warnings or errors when you use Jupyter.

MarcSkovMadsen · 2024-03-28T11:23:02Z

This is what I get when I clear the browser and hit "refresh" in the Jupyter Panel Preview.

And in the terminal.

philippjfr · 2024-03-28T11:23:47Z

Doesn't look like there's anything related to Panel, but it does seem like there's an issue with the layout persistence. The layout persists in two levels:

As you do any action in the preview the changes are written to a .layout file that lives next to the notebook.
When you save the notebook the data in the .layout file is persisted into the notebook metadata.

I have not been able to reproduce the issue you seem to be having which is that the data written to the .layout file does not persist.

MarcSkovMadsen · 2024-03-28T11:25:06Z

I have not been able to identify a .layout file so far.

I'm on windows.

philippjfr · 2024-03-28T11:26:05Z

It's a hidden file, try ls -a

philippjfr · 2024-03-28T11:26:39Z

I think I've also encountered the issue, could you try removing spaces from the filename?

MarcSkovMadsen · 2024-03-28T11:27:01Z

Its there

{"cells": {"ae3c447a-fa81-499e-a9ce-1d4f03c0865d": {"width": 62.6789, "height": 506, "visible": true}, "e1dbb77f-7860-4b2f-9837-03d9f513b7c3": {"width": 100, "height": 10, "visible": true}, "7352e53c-e521-45ba-8e3f-83050c4e8561": {"width": 100, "height": 10, "visible": true}, "3cbfb569-34fc-44c2-b8fe-096452cdf5e0": {"width": 100, "height": 10, "visible": true}}, "order": ["ae3c447a-fa81-499e-a9ce-1d4f03c0865d", "e1dbb77f-7860-4b2f-9837-03d9f513b7c3", "7352e53c-e521-45ba-8e3f-83050c4e8561", "3cbfb569-34fc-44c2-b8fe-096452cdf5e0"]}

MarcSkovMadsen · 2024-03-28T11:29:44Z

I think I've also encountered the issue, could you try removing spaces from the filename?

I've renamed. Restarted Jupyter lab. Is see no change.

MarcSkovMadsen · 2024-03-28T11:33:16Z

When I restart jupyterlab and reload my notebook too, I see

(I am running on fresh virtual environment)

philippjfr · 2024-03-28T11:44:40Z

Think I may have found an issue when restoring the layout data from the hidden file.

MarcSkovMadsen · 2024-03-28T11:56:10Z

Reading the docs again it appears to me the save button is only for use outside of Jupyter Panel Preview? If that is the case I think it should not be shown when running in the Jupyter Panel Preview.

philippjfr · 2024-03-28T12:27:44Z

If that is the case I think it should not be shown when running in the Jupyter Panel Preview.

Yes I agree.

MarcSkovMadsen · 2024-03-28T20:06:23Z

UPDATED

I can use spaces in the name
I can get the example from the docs working

My Setup

Keeps resizing to something not useful

not-useful.mp4

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "862ab307-51f8-4a4e-83bb-0b04acb25b22",
      "metadata": {},
      "source": [
        "# Matplotlib Plot"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "afaa1656-8aa0-41e0-a01d-4b0c581c178f",
      "metadata": {},
      "source": [
        "A cell with text and a code snippet\n",
        "\n",
        "```python\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "\n",
        "fruits = [\"apple\", \"blueberry\", \"cherry\", \"orange\"]\n",
        "counts = [40, 100, 30, 55]\n",
        "bar_labels = [\"red\", \"blue\", \"_red\", \"orange\"]\n",
        "bar_colors = [\"tab:red\", \"tab:blue\", \"tab:red\", \"tab:orange\"]\n",
        "\n",
        "ax.bar(fruits, counts, label=bar_labels, color=bar_colors)\n",
        "\n",
        "ax.set_ylabel(\"fruit supply\")\n",
        "ax.set_title(\"Fruit supply by kind and color\")\n",
        "ax.legend(title=\"Fruit colors\")\n",
        "\n",
        "plt.show()\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "93933ba6-da4b-4def-80ad-b9a480b6cac3",
      "metadata": {},
      "source": [
        "## TODO: Document this properly"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "04e0edc3-ae14-43ad-8e5f-7669d7f09f80",
      "metadata": {},
      "outputs": [],
      "source": [
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "\n",
        "fruits = [\"apple\", \"blueberry\", \"cherry\", \"orange\"]\n",
        "counts = [40, 100, 30, 55]\n",
        "bar_labels = [\"red\", \"blue\", \"_red\", \"orange\"]\n",
        "bar_colors = [\"tab:red\", \"tab:blue\", \"tab:red\", \"tab:orange\"]\n",
        "\n",
        "ax.bar(fruits, counts, label=bar_labels, color=bar_colors)\n",
        "\n",
        "ax.set_ylabel(\"fruit supply\")\n",
        "ax.set_title(\"Fruit supply by kind and color\")\n",
        "ax.legend(title=\"Fruit colors\")\n",
        "\n",
        "plt.show()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.4"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}

I've tried resetting everything using script below. But without any change.

import json
import sys
from pathlib import Path

def reset_notebook_metadata(notebook_path):
    with open(notebook_path, 'r', encoding='utf-8') as f:
        notebook = json.load(f)

    for cell in notebook['cells']:
        if 'metadata' in cell:
            metadata = cell['metadata']
            if 'panel-layout' in metadata:
                metadata.pop('panel-layout')
        
    metadata = notebook['metadata']
    if 'panel-cell-order' in metadata:
        metadata.pop('panel-cell-order')


    with open(notebook_path, 'w', encoding='utf-8') as f:
        json.dump(notebook, f, indent=2)

    print(f"'panel-layout' metadata removed for {notebook_path}")

def delete_layout_file(notebook_path):
    path = Path(notebook_path)
    layout = "." + path.name + ".layout"
    path = path.parent/layout
    if path.exists():
        path.unlink()
        print(f"{path} removed for {notebook_path}")



if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python reset_layout.py NameOfNotebook.ipynb")
    else:
        notebook_path = sys.argv[1]
        reset_notebook_metadata(notebook_path)
        delete_layout_file(notebook_path)

I tried renaming and suddenly it works.

Is something persisted in my browser cache and should not be?

You can see the rename below

not-working-with-name.mp4

Resize button not shown if panel is scrolled

resize-not-shown-if-scrolled.mp4

Make it clear how error message should be and are handled.

I would like to see a clear error message in the preview in this situation. For example on a JupyterHub it would not be easy for users to see the error in the (terminal) logs.

Reset Height Nicely

When I delete panes and reset the layout they don't always show up with a good height.

reset-height-nicely.mp4

Not clear how to fully clear all layout state

I'm not sure I can do anything but delete .ipynb and .layout file to start from scratch? It would be nice if it was documented how to start from scratch without notebook. Probably it consists of closing preview, deleting .layout file and clearing all meta data (how?) in the notebook.

Make it clear when state has not been synced to notebook

The ?editable preview is nice. The last missing piece would be to make it clear when the state has not been persisted to the notebook. I.e. when the save button should be clicked - otherwise changes will be lost.

editable.mp4

Make sure older .layout file is not used with new .ipynb notebook.

It is not clear to me what will happen if I have an old .layout file and I rename or create a new corresponding notebook. I would prefer the .layout file to be fully reset in that case in stead of reused.

Maybe its possible to do this via a unique id in the notebook? Or by comparing creation timestamps?

Rename .layout file with Notebook

If a notebook is renamed it would be nice if the .layout file was renamed too.

Delete .layout file with Notebook

If a notebook is deleted it would be nice if the .layout file was deleted too.

philippjfr · 2024-03-28T20:57:58Z

That old error page seems to be indicating it's using an old version of Panel.

MarcSkovMadsen · 2024-03-29T04:56:46Z

That old error page seems to be indicating it's using an old version of Panel.

I've redone my tests and updated the above post.

philippjfr · 2024-03-29T10:03:50Z

Thanks so much for the detailed testing.

I can get into a state where I cannot get a working layout

I absolutely cannot reproduce this.

The save button is still shown

It is not, you still seem to be picking up a wrong version of Panel. Jupyter installs are finnicky, check what jupyter serverextension list says.

Avoid making height smaller and smaller for each reset click

Again, this is due to an older version of Panel.

Make it clear how error message should be and are handled.

Very open to suggestions here, worth noting that apps that have an error have the same behavior but I suppose there you usually have access to the logs/terminal.

Not clear how to fully clear all layout state

Yeah, there needs to be some mechanism for this. If I change the reset to be a full reset then I could probably achieve this.

Make it clear when state has not been synced to notebook

Great idea, UI suggestions very welcome.

Rename/Delete .layout file with Notebook

Indeed, will have to be changed in pyviz_comms.

MarcSkovMadsen · 2024-03-30T07:19:01Z

My mistake was testing panel==1.4.0a1 instead of 1.4.1a1. I've retested with 1.4.1a1 and updated posted above.

MarcSkovMadsen · 2024-03-30T08:35:44Z

The example from the docs still does not work for me. Added video and info above.

philippjfr · 2024-03-30T08:42:05Z

Very strange, absolutely cannot reproduce that issue.

philippjfr · 2024-03-30T08:45:38Z

Could it be that 1.4.1a1 still isn't the version it's running as the jupyter serverextension?

philippjfr · 2024-03-30T08:48:08Z

Jupyter sadly is very finicky since extensions can be installed in various places and precedence of each one is unclear to me, e.g. you can have an extension installed globally and in a particular environment.

MarcSkovMadsen · 2024-03-30T08:50:35Z

Are you sure that nothing is persisted in the browser cache/ storage system when doing this. To me that could be an explanation.

Under one name it works. Under another it does not.

Also I cannot "reset" the not working name by removing metadata and .layout file.

MarcSkovMadsen · 2024-03-30T08:54:09Z

MarcSkovMadsen · 2024-03-30T08:56:51Z

Hmm. Clearing the local storage and hard refreshing does not help.

MarcSkovMadsen · 2024-03-30T08:57:37Z

The console error is just missing favicon

philippjfr · 2024-03-30T08:57:58Z

Seems like the first step to getting to the bottom of this will be to make the reset actually reset everything, local storage, .layout file and notebook metadata.

MarcSkovMadsen · 2024-03-30T08:59:31Z

I've

cleared the metadata and .layout via the script
cleared the localstorage
restarted jupyter lab

Then it works

philippjfr · 2024-03-30T09:00:31Z

If the jupyterlab restart was actually required the only thing I can think of was that your JLab server was still running the old version before the restart.

philippjfr · 2024-03-30T09:00:59Z

Anyway, I'll work on the full reset next.

MarcSkovMadsen · 2024-03-30T09:01:04Z

I've have been restarting it many times this morning after installing panel==1.4.1a1.

MarcSkovMadsen · 2024-03-30T09:02:04Z

I think docs need improvement. I did not understand localstorage also involved for notebooks in preview. I thought that was only for served apps with ?editable=True. Now I don't understand what takes precedence?

philippjfr · 2024-03-30T09:03:53Z

I think docs need improvement. I did not understand localstorage also involved for notebooks in preview. I thought that was only for served apps with ?editable=True. Now I don't understand what takes precedence?

Since we removed the save button in the preview it should no longer allow you to use localstorage (in the preview). Seems likely you had something in localstorage from before we hid the save button. The precedence hierarchy generally is:

.layout file
Notebook metadata
Localstorage

MarcSkovMadsen · 2024-03-30T09:03:57Z

Do you cache anything during the "layout" computation? Could that be an explanation?

MarcSkovMadsen added TRIAGE Default label for untriaged issues type: bug Something isn't correct or isn't working labels Mar 28, 2024

MarcSkovMadsen added this to the next milestone Mar 28, 2024

MarcSkovMadsen removed the TRIAGE Default label for untriaged issues label Mar 28, 2024

MarcSkovMadsen mentioned this issue Mar 28, 2024

Improve layout builder docs #6601

Merged

2 tasks

philippjfr modified the milestones: next, v1.4.1 Mar 29, 2024

philippjfr mentioned this issue Apr 1, 2024

Allow full reset of dashboard builder layout #6625

Merged

philippjfr closed this as completed in #6625 Apr 1, 2024

Layout builder example not working #6600

Layout builder example not working #6600

Comments

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

MarcSkovMadsen commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

philippjfr commented Mar 28, 2024

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

philippjfr commented Mar 28, 2024

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 28, 2024 • edited Loading

Keeps resizing to something not useful

Resize button not shown if panel is scrolled

Make it clear how error message should be and are handled.

Reset Height Nicely

Not clear how to fully clear all layout state

Make it clear when state has not been synced to notebook

Make sure older .layout file is not used with new .ipynb notebook.

Rename .layout file with Notebook

Delete .layout file with Notebook

philippjfr commented Mar 28, 2024

MarcSkovMadsen commented Mar 29, 2024 • edited Loading

philippjfr commented Mar 29, 2024 • edited Loading

MarcSkovMadsen commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024

philippjfr commented Mar 30, 2024

philippjfr commented Mar 30, 2024

philippjfr commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024 • edited Loading

MarcSkovMadsen commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024

philippjfr commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024

philippjfr commented Mar 30, 2024

philippjfr commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024

MarcSkovMadsen commented Mar 30, 2024

philippjfr commented Mar 30, 2024 • edited Loading

MarcSkovMadsen commented Mar 30, 2024

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 28, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 29, 2024 •

edited

Loading

philippjfr commented Mar 29, 2024 •

edited

Loading

MarcSkovMadsen commented Mar 30, 2024 •

edited

Loading

philippjfr commented Mar 30, 2024 •

edited

Loading