Create a pure Python SWC -> NeuroML converter

[pgleeson]

Is your feature request related to a problem? Please describe.
There should be a pure Python way to import an SWC file and convert it to the latest valid NeuroML 2 format

Describe the solution you'd like
A file in here: https://github.com/NeuroML/pyNeuroML/tree/master/pyneuroml/swc, which can be used in scripts, or potentially at the command line, to take an SWC file and create a valid NML file. Some options could be included for handing ambiguities, e.g. when the soma is absent, e.g. see here

Describe alternatives you've considered
The current recommended way to get SWC to NeuroML is to use neuroConstruct, but any updates to this would require updates to the Java implementation here. See discussions here.

Additional context
The closest thing to a reference implementation of SWC is the https://github.com/pgleeson/Cvapp-NeuroMorpho.org which is based on @robertcannon's original CVapp application. This is used on NeuroMorpho.Org for viewing SWC files.

The main issue when importing SWC files is deciding what to do when the soma is missing/badly specified/only present as a traced outline, and how to handle connection to the first real point (on dendrites etc.). There are a number of examples here: https://github.com/pgleeson/Cvapp-NeuroMorpho.org/tree/master/caseExamples for the various options, as well as a doc discussing these: https://github.com/pgleeson/Cvapp-NeuroMorpho.org/blob/master/caseExamples/SomaFormat-NMOv5.3.pdf

@MRIO and colleagues are interested in working on this functionality.

[pgleeson]

Note 1: The notebook here: https://github.com/NeuralEnsemble/libNeuroML/blob/development/notebooks/CellMorphology.ipynb may be useful for libNeuroML helper methods for morphologies.
Note 2: Ideally for a simple SWC file exported to NML via neuroConstruct and via this new script, they should be pretty much identical.

cc @WardDPeeters

[WardDPeeters]

Thanks! That should really help speed the process along! I'll probably also approach you via Slack regarding some other functions and/or specific variables in the near future.

[WardDPeeters]

Hi!

I had a couple of questions regarding a few different variables within a cell morphology in nml;

My first question is regarding the value 'proximal'. Every segment has a point for proximal and distal - where the segment begins and where it ends, but in some nml files I've looked at I've noticed that for some segments, proximal points are missing while for others they are not. I figured this maybe had to do with assuming that the distal point of the parent segment is also the proximal point of the next, but I'm not sure on that one.
My second question is alro regarding the segments, namely the value 'fraction_along'. SWC-files appear to only be able to handle values that are 0 or 1 here, but what does this value actually represent exactly?

[pgleeson]

Hi @WardDPeeters. Regarding the first point, yes, if the proximal is missing you use a point on the parent segment. If fractionAlong is missing or 1 (default) it's the distal point of the parent, and if fractionAlong==0, it's the proximal point. For vals between 0 nad 1 use interpolation The helper method: cell.get_actual_proximal(seg.id) can help you get the real proximal point, see https://github.com/NeuralEnsemble/libNeuroML/blob/master/notebooks/CellMorphology.ipynb.

Regarding the second point, there are no segments in swc, only points. Points are connected to their parent points. The simplest interpretation of 2 connected points is to make a corresponding NML segment for this. The complexities arise when the start point is the soma (often it's intentionally left out of the swc) and whether the daughter dendrites inherit the soma's (larger) radius... The pointers in the first comment above outline the various cases...

[WardDPeeters]

Hello @pgleeson! I have some good news, I have a first working version of a converter! I need to finetune it a bit, specifically to incorporate the detection and handling of weird structures like loops, or anything of the like.
Regarding this, I had the question, what should I do with the loops that might be present in a morphology? Do they need to be processed, or just detected and throw an error message? Is it even possible to have a loop encoded in an swc file?

[pgleeson]

@WardDPeeters Apologies for the delay in getting back to you. Glad you have an initial version.

Regarding loops, certainly NeuroML doesn't allow loops, and probably SWC too, though there's nothing stopping someone entering a sequence of points in an SWC file forming a loop. For your purposes here it's best to throw an error giving the specifics on which points are connected in loops.

[sanjayankur31]

Hi @WardDPeeters

How are you getting on with this task? Is there anything we can help you with?

Would the code you have so far be available somewhere for us to look at too please?

Cheers,

[sanjayankur31]

Hi @WardDPeeters , would this be the current state of the converter please?

https://github.com/WardDPeeters/Morphologies

[WardDPeeters]

@sanjayankur31 Yess! The main converter is in the Main branch of the folder (it's called SWC_to_NML_Converter_Dev.py), it should work just fine, but I haven't touched the code in quite a while, so there might be some unresolved bugs that I did not yet know of.

Furthermore, the file that you want to convert can only be pasted into the code, not yet converted through a command terminal yet, however there should be someone working on that in the coming time.

[sanjayankur31]

That's great, thanks very much. I'll play with it when I can, and I'll get in touch before I jump in to making any tweaks.