Skip to content

Commit

Permalink
Add Bulding Workflow Documentation (#10)
Browse files Browse the repository at this point in the history
  • Loading branch information
eleftherioszisis authored Oct 10, 2024
1 parent 8815ee3 commit 14f9d64
Show file tree
Hide file tree
Showing 9 changed files with 685 additions and 3 deletions.
2 changes: 1 addition & 1 deletion .pylintrc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[MESSAGES CONTROL]
disable=fixme,invalid-name,len-as-condition,no-else-return,no-member,empty-docstring
disable=fixme,invalid-name,len-as-condition,no-else-return,no-member,empty-docstring,too-many-positional-arguments

[FORMAT]
# Regexp for a line that is allowed to be longer than the limit.
Expand Down
15 changes: 13 additions & 2 deletions bbp_workflow/generation/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,18 @@

@luigi.util.inherits(KgCfg, RemoteHostCfg)
class SBOWorkflow(luigi.WrapperTask):
"""Workflow class."""
"""Generator Workflow class for building detailed circuits.
Args:
output_dir (Path): Output directory to write generated data.
config_url (str): ModelBuildingConfig NEXUS resource id.
account (str): Project account.
isolated (bool):
Whether the workflow will run in isolated mode. Isolated mode ignores activities from
previous executions.
target (str): The target generator execute. The workflow will run all tasks up until the
target task.
"""

output_dir = luigi.PathParameter(
significant=True,
Expand Down Expand Up @@ -45,7 +56,7 @@ def host_config(self) -> dict:
return self.clone(RemoteHostCfg).param_kwargs

def requires(self):

"""Return required leaf task in the workflow DAG, determined by the `target` argument."""
generator_class = get_class_from_config_name(self.target)

return generator_class(
Expand Down
1 change: 1 addition & 0 deletions doc/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ API Documentation
bbp_workflow.simulation.util
bbp_workflow.target
bbp_workflow.task
bbp_workflow.generation.workflow
1 change: 1 addition & 0 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,6 @@ Overview of the tasks available in the workflow engine:

Home <self>
api
workflow
dependencies
changelog
9 changes: 9 additions & 0 deletions doc/source/workflow.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
The Building Workflow
=====================

.. toctree::

workflow_components/quickstart.rst
workflow_components/overview.rst
workflow_components/generator_hierarchy.rst
workflow_components/generator_definitions.rst
324 changes: 324 additions & 0 deletions doc/source/workflow_components/generator_definitions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,324 @@
Generator Definitions
======================

.. _cell_composition_generator:

CellCompositionGenerator
~~~~~~~~~~~~~~~~~~~~~~~~

Config: `CellCompositionConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#cellcompositionconfig>`_

The ``CellCompositionGenerator`` is responsible for manipulating an input CellComposition in terms of density.


The final resource generated by the generator is a new CellComposition incorporating the changes listed in overrides.


.. _cell_position_generator:

CellPositionGenerator
~~~~~~~~~~~~~~~~~~~~~

Config: `CellPositionConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#cellpositionconfig>`_

The ``CellPositionGenerator`` consumes the upstream CellComposition registered by :ref`CellCompositionGenerator <cell_composition_generator>` and places cell somata, along with properties such as mtypes, and etype.

The final resource generated by the generator is a partial ``DetailedCircuit`` with a circuit config as follows:

.. code-block:: json
{
"version": 2,
"manifest": {
"$BASE_DIR": "."
},
"node_sets_file": "path/to/nodesets/json",
"networks": {
"nodes": [
{
"nodes_file": "path/to/nodes/h5",
"populations": {
"root__neurons": {
"type": "biophysical",
}
}
}
],
"edges": []
},
"metadata": {
"status": "partial"
}
}
where the node population has the following properties:

.. code-block:: text
etype
hemisphere
morph_class
mtype
region
subregion
synapse_class
x
y
z
See `SONATA documentation <https://sonata-extension.readthedocs.io/en/latest/sonata_tech.html#fields-for-biophysically-detailed-neurons-model-type-biophysical>`__ for column definitions.

.. _mmodel_generator:

MorphologyAssignmentGenerator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Config: `MorphologyAssignmentConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#morphologyassignmentconfig>`_

The ``MorphologyAssignmentGenerator`` takes the ``DetailedCircuit`` from :ref:`CellPositionGenerator <cell_position_generator>` and depending on the configuration assigns or/and synthesizes morphologies for each cell in the node population.

The final resource generated by the generator is a partial ``DetailedCircuit`` with a circuit config as follows:

.. code-block:: json
{
"version": 2,
"manifest": {
"$BASE_DIR": "."
},
"metadata": {
"status": "partial"
},
"node_sets_file": "path/to/nodests/json",
"networks": {
"nodes": [
{
"nodes_file": "path/to/nodes/h5",
"populations": {
"root__neurons": {
"type": "biophysical",
"alternate_morphologies": {
"h5v1": "$BASE_DIR/morphologies",
"neurolucida-asc": "$BASE_DIR/morphologies"
}
}
}
}
],
"edges": []
}
}
where the node population has the following properties:

.. code-block:: text
etype
hemisphere
morph_class
morphology
morphology_producer
mtype
orientation_w
orientation_x
orientation_y
orientation_z
region
subregion
synapse_class
x
y
z
See `SONATA documentation <https://sonata-extension.readthedocs.io/en/latest/sonata_tech.html#fields-for-biophysically-detailed-neurons-model-type-biophysical>`__ for column definitions.


.. _me_model_generator:

MEModelGenerator
~~~~~~~~~~~~~~~~

Config: `MEModelConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#memodelconfig>`_

The ``MEModelGenerator`` takes the ``DetailedCircuit`` from :ref:`MorphologyAssignmentGenerator <mmodel_generator>` and assigns emodels and emodel properties.

The final resource generated by the generator is a partial ``DetailedCircuit`` with a circuit config as follows:

.. code-block:: json
{
"version": 2,
"manifest": {
"$BASE_DIR": "."
},
"node_sets_file": "path/to/nodesets/json",
"networks": {
"nodes": [
{
"nodes_file": "path/to/nodes/h5",
"populations": {
"root__neurons": {
"type": "biophysical",
"partial": [
"cell-properties",
"morphologies"
],
"alternate_morphologies": {
"h5v1": "path/to/morphologies/dir",
"neurolucida-asc": "path/to/morphologies/dir"
},
"biophysical_neuron_models_dir": "path/to/hoc/dir"
}
}
}
],
"edges": []
},
"metadata": {
"status": "partial"
}
}
where the node population has the following properties:

.. _me_model_properties:

.. code-block:: text
dynamics_params/AIS_scaler
dynamics_params/holding_current
dynamics_params/input_resistance
dynamics_params/resting_potential
dynamics_params/soma_scaler
dynamics_params/threshold_current
etype
hemisphere
model_template
morph_class
morphology
morphology_producer
mtype
orientation_w
orientation_x
orientation_y
orientation_z
region
subregion
synapse_class
x
y
z
See `SONATA documentation <https://sonata-extension.readthedocs.io/en/latest/sonata_tech.html#fields-for-biophysically-detailed-neurons-model-type-biophysical>`__ for column definitions.

.. _macro_generator:

MacroConnectomeGenerator
~~~~~~~~~~~~~~~~~~~~~~~~

Config: `MacroConnectomeConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#macroconnectomeconfig>`_

The ``MacroConnectomeGenerator`` is a :ref:`relay generator <generator_types>` that propagates its config downstream to :ref:`MicroConnectomeGenerator <micro_generator>`.

The final resource of `MacroConnectomeGenerator` is a clone of its input config.


.. _micro_generator:

MicroConnectomeGenerator
~~~~~~~~~~~~~~~~~~~~~~~~

Config: `MicroConnectomeConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#microconnectomeconfig>`_

The ``MicroConnectomeGenerator`` takes as an input the `MacroConnectomeConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#macroconnectomeconfig>`_ from :ref:`MacroConnectomeGenerator <macro_generator>` and the ``DetailedCircuit`` from :ref:`MEModelGenerator <me_model_generator>` and establishes the connectivity of the node population.


The final resource generated by the generator is a partial ``DetailedCircuit`` with a circuit config as follows:

.. code-block:: json
{
"version": 2,
"manifest": {
"$BASE_DIR": "."
},
"node_sets_file": "path/to/nodesets/json",
"networks": {
"nodes": [
{
"nodes_file": "path/to/nodes/h5",
"populations": {
"root__neurons": {
"type": "biophysical",
"alternate_morphologies": {
"h5v1": "path/to/morphologies/dir",
"neurolucida-asc": "path/to/morphologies/dir"
},
"biophysical_neuron_models_dir": "path/to/hoc/dir"
}
}
}
],
"edges": [
{
"edges_file": "path/to/edges/h5",
"populations": {
"root__neurons__root__neurons__chemical": {
"type": "chemical"
}
}
}
]
},
"metadata": {
"status": "partial"
}
}
where the nodes file has the same properties as in the :ref:`MEModelGenerator ones <me_model_properties>` and the edges file has the following properties:


.. code-block:: text
afferent_center_x
afferent_center_y
afferent_center_z
afferent_section_id
afferent_section_pos
afferent_section_type
delay
syn_type_id
See `SONATA documentation <https://sonata-extension.readthedocs.io/en/latest/sonata_tech.html#fields-for-chemical-connection-type-edges>`__ for column definitions.


ConnectomeFilteringGenerator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Config: `SynapseConfig <https://entity-management.readthedocs.io/en/latest/building_entities.html#synapseconfig>`_

The ``ConnectomeFilteringGenerator`` takes the ``DetailedCircuit`` from :ref:`MicroConnectomeGenerator <micro_generator>` and filters the according to the configuration.

The final resource is a simulation-ready ``DetailedCircuit`` where the where the nodes file has the same properties as in the :ref:`MEModelGenerator ones <me_model_properties>` and the edges file has the following properties:

.. code-block:: text
afferent_center_x
afferent_center_y
afferent_center_z
afferent_section_id
afferent_section_pos
afferent_section_type
conductance
conductance_scale_factor
decay_time
delay
depression_time
facilitation_time
n_rrp_vesicles
syn_property_rule
syn_type_id
u_hill_coefficient
u_syn
See `SONATA documentation <https://sonata-extension.readthedocs.io/en/latest/sonata_tech.html#fields-for-chemical-connection-type-edges>`__ for column definitions.
Loading

0 comments on commit 14f9d64

Please sign in to comment.