Changing fig.set_size_inches() interactively shrinks the figure/canvas

[Brunox13]

Problem Description

I am trying to create a figure and subsequently change it using interactive input. Specifically, I would like to change the number of subplot axes based on the input. The following code works as expected:

%matplotlib widget

import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import ipywidgets as widgets
from IPython.display import display

# Create a plot with one (main) axes
fig, axs = plt.subplots(1, figsize = (10,5), sharex = True)
axs_height = 4
sub_axs_height = 0.2

number_widget = widgets.IntSlider(
    value = 1,
    min = 0,
    max = 5,
    step = 1,
    continuous_update = False,
    description = 'Sub-axes:'
    )

def editExistingFigure(number_of_additional_axes):
    # Clear the "main" axes
    axs.cla()
    # Remove any currently existing "additional" axes
    for ax in fig.axes[1:]: fig.delaxes(ax)
    # Set up the grid in preparation for the new axes
    gs = gridspec.GridSpec(
        nrows = 1 + number_of_additional_axes,
        ncols = 1,
        figure = fig,
        height_ratios = [axs_height] + [sub_axs_height]*number_of_additional_axes
        )
    # Place the "main" axes in the grid
    axs.set_position(gs[0,0].get_position(fig))
    # Plot something on the main axes
    axs.plot((1,2), (1,2))
    
    for i in range(number_of_additional_axes):
        # Add the additional axes
        additional_ax = fig.add_subplot(gs[i+1,0], sharex = axs)
        # Plot something on the new axes
        additional_ax.plot((1,2), (1,1))
        
ui = widgets.Box([number_widget])
out = widgets.interactive_output(
    editExistingFigure,
    {'number_of_additional_axes' : number_widget}
    )

# Display the UI with the output
display(ui, out)

The output looks like this, while the additional axes are added/removed as expected based on the interaction with the slider, and the size of the figure remains constant throughout:

However, I would like to keep the absolute size of the "main axes" constant, while changing the overall size of the figure based on the number of "additional axes." I was hoping I'd achieve that by removing the figsize setting from the initial plt.subplots() call and then setting fig.set_size_inches() based on the interactive input, as follows:

%matplotlib widget

import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import ipywidgets as widgets
from IPython.display import display

# Create a plot with one (main) axes
fig, axs = plt.subplots(1, sharex = True)    # figsize removed from here
axs_height = 4
sub_axs_height = 0.2

number_widget = widgets.IntSlider(
    value = 1,
    min = 0,
    max = 5,
    step = 1,
    continuous_update = False,
    description = 'Sub-axes:'
    )

def editExistingFigure(number_of_additional_axes):
    
    # Clear the main axes
    axs.cla()
    # Remove all currently existing "additional axes"
    for ax in fig.axes[1:]: fig.delaxes(ax)
    # Set up the grid in preparation for the new axes
    gs = gridspec.GridSpec(
        nrows = 1 + number_of_additional_axes,
        ncols = 1,
        figure = fig,
        height_ratios = [axs_height] + [sub_axs_height]*number_of_additional_axes
        )
    # Place the "main" axes in the grid
    axs.set_position(gs[0,0].get_position(fig))
    # Plot something on the main axes
    axs.plot((1,2), (1,2))
    
    for i in range(number_of_additional_axes):
        # Add the additional axes
        additional_ax = fig.add_subplot(gs[i+1,0], sharex = axs)
        # Plot something on the new axes
        additional_ax.plot((1,2), (1,1))
    
    # THIS LINE IS NEW:
    fig.set_size_inches(10, axs_height + sub_axs_height*number_of_additional_axes)
        
ui = widgets.Box([number_widget])
out = widgets.interactive_output(
        editExistingFigure,
        {'number_of_additional_axes' : number_widget}
        )

# Display the UI with the output
display(ui, out)

When the cell is run for the first time, the resulting figure looks identical to the one above. However, when the slider is changed, now the output canvas suddenly shrinks ~2-fold, as follows:

What is interesting is that the underlying figure seems to be the correct size but the canvas is what shrinks. Additionally, the size of the canvas is proportionate to the size of the underlying figure - i.e., it does respond to changes in the interactive input (height increases a little with slider number increasing, while the width is constant), but is just smaller.
The moment I interactively pull the right lower corner of the canvas with my mouse, the figure resizes to fit the canvas.

Note: This issue may be related to the following issues: #117, #36, and especially #17 (due to my impression that the output shrinks proportionately by ~2x and I am using a high-dpi diplay), though because there's no loop involved, it seems to be caused by fig.set_size_inches(), and none of the solutions suggested in the issues above worked, I decided to open a separate issue.

Versions

 3.7.3 (default, Mar 27 2019, 22:11:17) 
[GCC 7.3.0]
ipympl version: 0.5.6
jupyter core     : 4.5.0
jupyter-notebook : 5.7.9
qtconsole        : 4.5.1
ipython          : 7.6.0
ipykernel        : 5.1.1
jupyter client   : 5.2.4
jupyter lab      : 2.1.4
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.6
traitlets        : 4.3.2
Known nbextensions:
  config dir: /dartfs-hpc/rc/home/h/f002b9h/.conda/envs/scRNAseq/etc/jupyter/nbconfig
    notebook section
      jupyter-matplotlib/extension  enabled 
      - Validating: OK
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
JupyterLab v2.1.4
Known labextensions:
   app dir: /dartfs-hpc/rc/home/h/f002b9h/.conda/envs/scRNAseq/share/jupyter/lab
        @jupyter-widgets/jupyterlab-manager v2.0.0  enabled  OK
        @lckr/jupyterlab_variableinspector v0.5.1  enabled  OK
        jupyter-matplotlib v0.7.2  enabled  OK

[jkochNU]

I would also be interested whether there is a workaround for this behavior, as it is making our ipywidgets GUI become near unusable.

Smaller code example for the same (I think) issue:

import matplotlib.pyplot as plt

fig, [ax1, ax2] = plt.subplots(1, 2, figsize=(8,4))

Output (as expected):

Now try to change the figure height (after the above has already been displayed):
fig.set_figheight(8)

The output suffers from sudden shrinking of the displayed area / position of the resize handle:

Versions

ipympl version: 0.8.2
Selected Jupyter core packages...
IPython          : 8.0.0
ipykernel        : 6.7.0
ipywidgets       : 7.6.5
jupyter_client   : 7.1.0
jupyter_core     : 4.9.1
jupyter_server   : 1.13.2
jupyterlab       : 3.0.14
nbclient         : 0.5.9
nbconvert        : 6.4.0
nbformat         : 5.1.3
notebook         : 6.4.0
qtconsole        : not installed
traitlets        : 5.1.1
Known nbextensions:
  config dir: C:\Users\drjen\.jupyter\nbconfig
    notebook section
      nbdime/index enabled
      - Validating: ok
      toc2/main enabled
      - Validating: problems found:
        - require?  X toc2/main
      hinterland/hinterland disabled
      jupyter-js-widgets/extension enabled
      - Validating: ok
  config dir: C:\Users\drjen\miniconda3\etc\jupyter\nbconfig
    notebook section
      jupyter-materialui/extension enabled
      - Validating: ok
      jupyter-matplotlib/extension enabled
      - Validating: ok
      jupyter-vue/extension enabled
      - Validating: ok
      jupyter-vuetify/extension enabled
      - Validating: ok
      nbdime/index enabled
      - Validating: ok
      jupyter-js-widgets/extension enabled
      - Validating: ok
 JupyterLab v3.0.14
C:\Users\drjen\miniconda3\share\jupyter\labextensions
        jupyter-matplotlib v0.10.2 enabled ok
        jupyter-vue v1.7.0 enabled ok
        jupyter-vuetify v1.8.1 enabled ok
        @jupyter-widgets/jupyterlab-manager v3.0.1 enabled ok (python, jupyterlab_widgets)
        @pyviz/jupyterlab_pyviz v2.1.0 enabled ok (python, pyviz_comms)
        @ryantam626/jupyterlab_code_formatter v1.4.10 enabled ok (python, jupyterlab-code-formatter)

Other labextensions (built into JupyterLab)
   app dir: C:\Users\drjen\miniconda3\share\jupyter\lab
        jupyter-materialui v0.1.4 disabled  X
        nbdime-jupyterlab v2.1.1 enabled ok

   The following extension are outdated:
        jupyter-materialui

   Consider running "jupyter labextension update --all" to check for updates.



Disabled extensions:
    jupyter-materialui (all plugins)```