JNB:BUG:ENH: Fix Suite2p example, and update Binder environment for Python 3.9 #220

swkeemink · 2021-06-29T16:25:05Z

With this PR (which replaces an earlier PR which I accidentally botched a bit) I:

Update the Suite2p notebook to work with the latest Suite2p API changes
Update the Suite2p notebook to fix the bug in which multiprocessing freezes (fixing Stuck on 'Doing signal separation...' #147)
Update the binder environment to work with the current Suite2p in Python 3.9
Removed the link to the Sima notebook in binder, and clarified that it does not currently work in Python 3.9

codecov · 2021-06-29T16:31:52Z

Codecov Report

Merging #220 (7ae8584) into master (ba0680e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #220   +/-   ##
=======================================
  Coverage   93.05%   93.05%           
=======================================
  Files           8        8           
  Lines         922      922           
  Branches      196      196           
=======================================
  Hits          858      858           
  Misses         33       33           
  Partials       31       31

Flag	Coverage Δ
nbsmoke	`65.29% <ø> (ø)`
unittests	`92.73% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba0680e...7ae8584. Read the comment docs.

swkeemink · 2021-06-29T17:11:36Z

On Binder there are currently two bugs.

Matplotlib is missing, which of course breaks Holoviews.

Suite2p does not import and throws the following error (from pyqt5):
ImportError: libGL.so.1: cannot open shared object file: No such file or directory

scottclowe

Some commits (4de477d and 9d5e34b) are undoing formatting style fixes I made to the suite2p and SIMA notebooks. I would appreciate if these commits were dropped from the PR. I expect this has come from checking out the new branch, using your notebooks that were forked from the previous (old) copies of these notebooks, overwriting the notebooks, and then committing everything in the notebook without checking what it is that you're committing. If you're actually including these changes on purpose, please let me know.

scottclowe · 2021-07-04T20:38:20Z

examples/SIMA example.ipynb

@@ -40,7 +43,7 @@
    "import holoviews as hv\n",
    "\n",
    "%load_ext holoviews.ipython\n",
-    "%output widgets=\"embed\""
+    "%output widgets='embed'"


Unwanted side-effect.

scottclowe · 2021-07-04T20:38:35Z

examples/SIMA example.ipynb

+    "    i_cell: hv.Curve(exp.raw[i_cell][i_trial][0, :], label=\"SIMA\")\n",
+    "    * hv.Curve(exp.result[i_cell][i_trial][0, :], label=\"FISSA\")\n",


Unwanted side-effect.

scottclowe · 2021-07-04T20:39:04Z

examples/Suite2p example.ipynb

+    "    i_cell: hv.Curve(exp.raw[i_cell][i_trial][0, :], label=\"suite2p\")\n",
+    "    * hv.Curve(exp.result[i_cell][i_trial][0, :], label=\"FISSA\")\n",


Unwanted side-effect.

scottclowe · 2021-07-04T20:39:06Z

examples/SIMA example.ipynb

+    "avg_img * hv.HoloMap(region_plots, kdims=[\"Cell\"]) * fig + hv.HoloMap(\n",
+    "    traces_plots, kdims=[\"Cell\"]\n",


Unwanted side-effect.

scottclowe · 2021-07-04T20:39:18Z

examples/Suite2p example.ipynb

+    "avg_img * cell_locs * hv.HoloMap(region_plots, kdims=[\"Cell\"]) + hv.HoloMap(\n",
+    "    traces_plots, kdims=[\"Cell\"]\n",


Unwanted side-effect.

README.rst

scottclowe · 2021-07-04T20:54:47Z

.binder/environment.yml

+  - scipy==1.7.0
+  - pip=21.1.2
+  - six==1.16.0
+  - holoviews==1.13.2


Maybe current issues with holoviews on Binder come from it being conda installed instead of pip installed somehow?

.binder/environment.yml

scottclowe · 2021-07-04T20:56:46Z

.binder/environment.yml

+  - python=3.9.5
+  - numpy==1.21.0
+  - scipy==1.7.0
+  - pip=21.1.2
+  - six==1.16.0
+  - holoviews==1.13.2


Alphabetise

Suggested change

- python=3.9.5

- numpy==1.21.0

- scipy==1.7.0

- pip=21.1.2

- six==1.16.0

- holoviews==1.13.2

- holoviews==1.13.2

- numpy==1.21.0

- pip=21.1.2

- python=3.9.5

- scipy==1.7.0

- six==1.16.0

scottclowe · 2021-07-04T21:00:54Z

Maybe it would be better if we move the Suite2p example and its binder to a separate repo? Then the binder would boot up quickly for the non-Suite2p examples and be more stable. What do you think?

swkeemink · 2021-07-09T10:07:51Z

Not a terrible idea, although I don't like the repository bloat that comes with that, but given that Suite2p is so widely used it could be ok. We could also sway we only use Binder for our core notebooks and not the external packages (like suite2p and Sima).
With could add a note that we don't provide support for Suite2p, but only for the interaction with FISSA or something.

If we do a separate repository for Suite2p: that would just be an empty repository with only the notebook for suite2p and the binder stuff? Would you be ok to set up the repository with Binder support, and I'll add and debug the suite2p stuff?

…fissa

…n't work

Co-authored-by: Scott Lowe <[email protected]>

swkeemink · 2021-07-12T12:43:33Z

Just to confirm that #227 has fixed the multiprocessing bugs with suite2p and fissa together. So the changes to the notebook for multiprocessing are no longer needed.

scottclowe · 2021-07-12T12:59:46Z

Just to confirm that #227 has fixed the multiprocessing bugs with suite2p and fissa together. So the changes to the notebook for multiprocessing are no longer needed.

Great to hear! In that case, commit b852708 can be dropped from the PR.

If #228 is merged in, that might fix the issues with the matplotlib backend to holoviews (by not using holoviews any more and just matplotlib directly).

scottclowe · 2021-07-13T15:52:03Z

Let's drop suite2p from this binder and stay with Python 3.6 to support SIMA. Then we can make a new repository named fissa-example-suite2p for suite2p.

swkeemink · 2021-07-19T15:09:52Z

Closing this PR since most of it is no longer relevant.

swkeemink requested a review from scottclowe June 29, 2021 16:25

scottclowe requested changes Jul 4, 2021

View reviewed changes

swkeemink and others added 7 commits July 12, 2021 11:58

ENH: updated binder environment for binder to be Python 39

444f125

JNB:BUG: fix to be able to run multiprocessing with both suite2p and …

b852708

…fissa

JNB:DOC: clarified that the SIMA notebook does not run in Binder

4e02a86

DOC: removed Binder link for SIMA notebook, and clarified why it does…

69f6a84

…n't work

JNB:BUG: updated API use of Suite2p

31ab44d

RF: alphabetised dependencies

0070992

Co-authored-by: Scott Lowe <[email protected]>

DOC: RF: slight rewording

7ae8584

Co-authored-by: Scott Lowe <[email protected]>

swkeemink force-pushed the suite2p_python39 branch from 5c537bc to 7ae8584 Compare July 12, 2021 09:58

swkeemink closed this Jul 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JNB:BUG:ENH: Fix Suite2p example, and update Binder environment for Python 3.9 #220

JNB:BUG:ENH: Fix Suite2p example, and update Binder environment for Python 3.9 #220

swkeemink commented Jun 29, 2021

codecov bot commented Jun 29, 2021 •

edited

Loading

swkeemink commented Jun 29, 2021

scottclowe left a comment

scottclowe Jul 4, 2021

scottclowe Jul 4, 2021

scottclowe Jul 4, 2021

scottclowe Jul 4, 2021

scottclowe Jul 4, 2021

scottclowe Jul 4, 2021

scottclowe Jul 4, 2021

scottclowe commented Jul 4, 2021

swkeemink commented Jul 9, 2021

swkeemink commented Jul 12, 2021

scottclowe commented Jul 12, 2021

scottclowe commented Jul 13, 2021

swkeemink commented Jul 19, 2021

		" i_cell: hv.Curve(exp.raw[i_cell][i_trial][0, :], label=\"SIMA\")\n",
		" * hv.Curve(exp.result[i_cell][i_trial][0, :], label=\"FISSA\")\n",

		"avg_img * hv.HoloMap(region_plots, kdims=[\"Cell\"]) * fig + hv.HoloMap(\n",
		" traces_plots, kdims=[\"Cell\"]\n",

		"avg_img * cell_locs * hv.HoloMap(region_plots, kdims=[\"Cell\"]) + hv.HoloMap(\n",
		" traces_plots, kdims=[\"Cell\"]\n",

JNB:BUG:ENH: Fix Suite2p example, and update Binder environment for Python 3.9 #220

JNB:BUG:ENH: Fix Suite2p example, and update Binder environment for Python 3.9 #220

Conversation

swkeemink commented Jun 29, 2021

codecov bot commented Jun 29, 2021 • edited Loading

Codecov Report

swkeemink commented Jun 29, 2021

scottclowe left a comment

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe Jul 4, 2021

Choose a reason for hiding this comment

scottclowe commented Jul 4, 2021

swkeemink commented Jul 9, 2021

swkeemink commented Jul 12, 2021

scottclowe commented Jul 12, 2021

scottclowe commented Jul 13, 2021

swkeemink commented Jul 19, 2021

codecov bot commented Jun 29, 2021 •

edited

Loading