ENH: improve dipole plotting with MRI

[agramfort]

see fieldtrip ft_plot_dipole.m and ft_plot_slice.m

see their tuto at : http://www.fieldtriptoolbox.org/tutorial/natmeg/dipolefitting

EDIT:

Enhancements to plot_dipole_mri_orthoview (#3966):

• Fix bound calculation in head coordinates
• We should project the orientation vector onto each of the three 2D planes that we show in matplotlib mode
• Allow backend='mayavi' to use VolumeSlicer (http://docs.enthought.com/mayavi/mayavi/mlab_case_studies.html)
• Add head mesh in mayavi version

[dengemann]

good idea.

[larsoner]

@kambysese is this the sort of thing XFit does, too?

[larsoner]

Now that nibabel has a slice viewer we could use that. Or should we use nilearn?

[agramfort]

I think nibabel would be more adapted. We already have a weak dep on nibabel

[kingjr]

And nilearn is not going to support interactive viewing any time soon. So
+1 for the nibabel viewer.

On 8 March 2016 at 10:09, Alexandre Gramfort [email protected]
wrote:

I think nibabel would be more adapted. We already have a weak dep on
nibabel

—
Reply to this email directly or view it on GitHub
#2995 (comment)
.

[ktavabi]

@Eric89GXL yes regional/spatiotemporal or moving dipole fits that may be exported as ASCII or binary files. The ASCII files contain a header block followed by >1 lines of fit data per dipole; I'll upload the ASCII files you asked for shortly.

[joewalter]

I really like the idea. While improving the plotting for dipolar sources, we should also think of modifying the sparse source reconstruction methods, which we have in mne-python, to give dipole output (and stc?), so that we can use the same plot functions for sparse imaging, dipole mapping, and RAP-MUSIC. That would also make it easier to compare these methods.

[agramfort]

+100 !

[larsoner]

I'll probably take this on soonish, unless someone else was planning to.

[joewalter]

@Eric89GXL have you made any progress on this issue?
Are there already ideas how to generalize the representation of sparse stcs and dipoles?

[larsoner]

No progress yet. I guess we could plot them in slices of an MRI viewer but it probably won't end up being all that satisfying. A Mayavi window with translucent head and BEM shells actually would work. Now that I think about it, neither of these should be all that much work. WDYT?

[joewalter]

I think that showing the dipoles on MRI slices is a good idea as this is what people are used to see from other toolboxes.

Maybe it is also a good idea to combine both of your approaches. We could have four axes, three showing orthogonal MRI slices with dipoles and a fourth axes showing a glas brain with the dipoles and orthogonal planes indicating the slices. This fourth plot could then be used e.g. to control the crosshair for the other three axes (e.g. to navigate between dipoles). Furthermore, the glass brain could be used to demonstrate the variability of the source orientation over time by adding a time index (for this a fifth axes showing the time courses could be useful too).

[larsoner]

Have you looked at nibabel orthoview? It's close to what you mention with cross hairs and four axes. Ideally we would just need to add a fifth for glass brain, but since the first four axes are mpl and the fifth mayavi it's probably easiest just to make separate windows, lest we hit awful backend problems. (Matplotlib and mayavi have enough backend problems by themselves, combining them will surely make the problems worse.) Maybe matplotlib itself has decent enough 3D for this simple task, but I doubt it.

[agramfort]

here are examples: http://cognitrn.psych.indiana.edu/busey/temp/eeglabtutorial4.301/dipfittut/dipplot_multifit.gif http://cognitrn.psych.indiana.edu/busey/temp/eeglabtutorial4.301/dipfittut/dipplot3.gif http://journals.plos.org/plosbiology/article/figure/image?download&size=large&id=info:doi/10.1371/journal.pbio.0020176.g005 for the last one with glass brain it means using a template brain and we could/should reuse nilearn glass brains.

[larsoner]

Those are... not the most aesthetically pleasing. Hopefully we can do better :)

Didn't know nilearn had a glass brain. Neuromag software also has one in a FIF that I asked about re-using. We can hopefully provide multiple options. should be fairly easy to do that.

But in any case, giving orthoview MRI cuts and a complementary glass brain (probably a different window) should get us a good start.

[ktavabi]

+1e7

[larsoner]

This probably needs something like #3527

[ktavabi]

@Eric89GXL have you made any progress on this (I see milestone been removed 😟)? BTW why do you think this requires warping?

[larsoner]

I haven't started on it and we release in two weeks, so I removed the marker. In the general case, we need some way to coregister the head frame of the given subject (which could be e.g. a child or infant) with the template head (which will almost certainly be an adult). Warping is a good way to do this. If we have individualized MRI data it should be a lot easier because we could skip that step, but I haven't been motivated to work on just that bit yet. Would it be helpful to you?

[ktavabi]

Yes that would be tremendous help. I don't have individual MRIs but (still dealing) with case where I need to plot either scatter or group level dipole coordinates on slices of an average MRI template. Overall I understand how this issue, in particular localization benefits from warping, but in terms of visualization it's difficult to show multiple slices that may contain dipoles from more than one subject or condition. It's hard to imagine how anything other than a glass brain in standardized space approach could be more informative or tidy.

[larsoner]

Hmm... I was asking if the have-individualized-MRI case would be helpful, but it sounds like you're talking about the case where you do not have those available. So to be clear, you need the case without individualized MRIs, where you want the data coregistered to some template, right?

That is the harder one to get right, as we need some way to coregister the frames. Elekta does something reasonable and maybe not too difficult, but from talking to Jukka about it, I'm inclined to use warping if possible.

[ktavabi]

The have-individualized-MRI case should be most generally useful, but in the case of no-individual-MRIs, which is often the case with infant and toddler data, some related way to viz dipole localization e.g., using individual spherical conductor models is useful.

I am not sure if I understand what you mean by coregistering the frames, but if you recall on your last visit to SEA we used translation & scaling to transform dipole locations (coregister frames?) in template MRI space. In this case, given spheres and head shapes alone without MRI data, I presume warping or linear transform makes little difference in the end result.

In either case, I think for viz multiple slices the glass brain approach (from Alex's examples above) may be most reasonable esp for ms illustrations.

[ktavabi]

@Eric89GXL in hope of making sense I am thinking whether it maybe possible to use localization data shown above as M+SEM scatter, to create a mask image for nilearn.plotting.plot_glass_brain? What I don't know is whether it's possible/necessary to make a glass brain based on MRI template I am using?

[larsoner]

In this case, given spheres and head shapes alone without MRI data, I presume warping or linear transform makes little difference in the end result.

It probably won't make a big difference for plotting. But it would be good to have for surrogate subjects at some point, so I'd like to get that going if possible instead of using a stand-in method now that might require deprecation later.

[larsoner]

Just a note on this topic that in addition to using a glass head with translucent brain, we could add cut-plane MRI slices using Mayavi (which I didn't know was possible).

[hoechenberger]

:D

[ktavabi]

Mayavi...swell 😐

[larsoner]

@agramfort to modernize this issue, I would probably make vtkImagePlaneWidget support to Brain, and then add dipole plotting to brain. It seems better than adding a mayavi or 3D backend to plot_dipole_mri_orthoview.

[agramfort]

sounds reasonable to me to build this on top of brain in terms of public API we'll have to see what is the easiest to use and find via the doc

…

[larsoner]

• Brain.add_mri to add the slicer
• Brain.add_dipoles to add a Dipole instance

[agramfort]

let's make it work then we discuss API

…