Merge pull request #1365 from Dominik-Vogel/default_subscription

Add Subscribers from config

docs/examples/plotting/live_plotting.ipynb

@@ -0,0 +1,187 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Live Plotting\n",
+    "(back to overview [offline](../Main.ipynb),[online](https://nbviewer.jupyter.org/github/QCoDeS/Qcodes/tree/master/docs/examples/Main.ipynb))\n",
+    "\n",
+    "\n",
+    "[read on nbviewer](https://nbviewer.jupyter.org/github/QCoDeS/Qcodes/tree/master/docs/examples/plotting/live_plotting.ipynb)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## TL;DR\n",
+    "Enable live plotting by specifying 'subscripition.default_subscriber' in your qcodesrc.json."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Introduction\n",
+    "Live plotting in qcodes can be achieved by sending the acquired data to other applications. For this an adapter for the specific application is necessary. These adapter register as subscribers to the qcodes dataset and expose a callback that gets called on regular intervals that can be specified.\n",
+    "\n",
+    "## Live plotting with plottr as an example\n",
+    "An example of a live plotting application is for example [*plottr*](https://github.com/kouwenhovenlab/plottr). After installing *plottr* you can use it by simply setting `subscription.default_subscribers` to `[\"Plottr\"]` in your `qcodesrc.json` file or alternatively by updating your qcodes config for your current session:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import qcodes\n",
+    "qcodes.config.subscription.default_subscribers = [\"Plottr\"]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "With this simple chang you can run all your notebooks without any changes and live data will be streamed to plottr. For that lets consider the simple example adapted from the dataset docs:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%matplotlib notebook\n",
+    "import numpy as np\n",
+    "from time import sleep\n",
+    "from qcodes import find_or_create_instrument, initialise_database, new_experiment, Measurement\n",
+    "from qcodes.tests.instrument_mocks import DummyInstrument"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dac = find_or_create_instrument(DummyInstrument, 'dac', gates=['ch1'], recreate=True)\n",
+    "dmm = find_or_create_instrument(DummyInstrument, 'dmm', gates=['v1'], recreate=True)\n",
+    "\n",
+    "def customgetter():\n",
+    "    return 5*np.exp(-0.2*dac.ch1()) + 0.02*5*np.random.randn()\n",
+    "    \n",
+    "dmm.v1.get = customgetter"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "tutorial_exp#no sample#7@./exp_container_tutorial.db\n",
+       "----------------------------------------------------"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "initialise_database()\n",
+    "new_experiment(name='tutorial_exp', sample_name=\"no sample\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Running the next cell should render the data in the live plotting front end:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Starting experimental run with id: 17\n"
+     ]
+    }
+   ],
+   "source": [
+    "meas = Measurement()\n",
+    "meas.register_parameter(dac.ch1)  # register the first independent parameter\n",
+    "meas.register_parameter(dmm.v1, setpoints=(dac.ch1,))  # now register the dependent oone\n",
+    "# make live plotting faster:\n",
+    "meas.write_period = 0.1\n",
+    "\n",
+    "with meas.run() as datasaver:      \n",
+    "    for set_v in np.linspace(0, 25, 100):\n",
+    "        dac.ch1.set(set_v)\n",
+    "        get_v = dmm.v1.get()\n",
+    "        datasaver.add_result((dac.ch1, set_v),\n",
+    "                             (dmm.v1, get_v))\n",
+    "        # we added some extra sleep so that the live plotting can be observed\n",
+    "        sleep(0.1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.6"
+  },
+  "nbsphinx": {
+   "execute": "never"
+  },
+  "toc": {
+   "base_numbering": 1,
+   "nav_menu": {},
+   "number_sections": true,
+   "sideBar": true,
+   "skip_h1_title": false,
+   "title_cell": "Table of Contents",
+   "title_sidebar": "Contents",
+   "toc_cell": false,
+   "toc_position": {
+    "height": "1225px",
+    "left": "1336px",
+    "top": "111.133px",
+    "width": "165px"
+   },
+   "toc_section_display": true,
+   "toc_window_display": false
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

qcodes/config/qcodesrc.json

@@ -14,6 +14,34 @@
             "pyvisa": "INFO"
         }
     },
+    "subscription":{
+        "subscribers":{
+            "QCPlot":{
+                "factory": "qcplotutils.qcodes_dataset.QCPlotDatasetSubscriber",
+                "factory_kwargs":{
+                    "log": false
+                },
+                "subscription_kwargs":{
+                    "min_wait": 0,
+                    "min_count": 1,
+                    "callback_kwargs": {}
+                }
+            },
+            "Plottr":{
+                "factory": "plottr.qcodes_dataset.QcodesDatasetSubscriber",
+                "factory_kwargs":{
+                    "log": false
+                },
+                "subscription_kwargs":{
+                    "min_wait": 0,
+                    "min_count": 1,
+                    "callback_kwargs": {}
+                }
+            }
+
+        },
+        "default_subscribers": []
+    },
     "gui" :{
         "notebook": true,
         "plotlib": "all",

qcodes/config/qcodesrc_schema.json

@@ -97,6 +97,60 @@
             },
             "required":["console_level", "file_level"]
         },
+        "subscription":{
+            "type": "object",
+            "description": "Controls how subscriptions to the dataset, for e.g. live plotting, are handled. Here subscribers can be registered so that they will be available through `DataSet.subscribe_from_config`. Additionally default subscribers for every DataSet that is generated by the `Measurement` object can be defined.",
+            "properties":{
+                "subscribers":{
+                    "type": "object",
+                    "description": "List of available pre-configured subscribers. The names of the subscriber entries are available as arguments of the `DataSet.subscribe_from_config` method and can be used in this conifg in the `subscription.default_subscribers` property.",
+                    "additional_properties":{
+                        "type": "object",
+                        "properties":{
+                            "factory": {
+                                "type": "string",
+                                "description": "Full location of a factory class in the form '<package_name>.<subpackage>.<classname>'. The factory class has the following footprint:\n class Subscriber:\n    def __init__(self, dataset, **factory_kwargs):\n    def __call__(self, results, length, state):\n"
+                            },
+                            "factory_kwargs": {
+                                "type": "object",
+                                "additionnal_properties":{},
+                                "description": "kwargs that get passed to the factory when creating a new subscriber."
+                            },
+                            "subscription_kwargs": {
+                                "type": "object",
+                                "description": "kwargs with which subscribe will be called.",
+                                "properties":{
+                                    "min_wait":{
+                                        "type": "integer",
+                                        "default": 0
+                                    },
+                                    "min_count":{
+                                        "type": "integer",
+                                        "default": 1
+                                    },
+                                    "callback_kwargs": {
+                                        "description": "kwargs passed to the callback.",
+                                        "type": "object",
+                                        "additionnal_properties": {},
+                                        "default": {}
+                                    }
+                                },
+                                "default_subscribers":{
+                                    "description": "List of default subscribers as defined in 'subscription.subscribers'. All subscribers in the list will be added to any DataSet created by the `Measurement.run()` call.",
+                                    "type": "array",
+                                    "items":{
+                                        "type": "string"
+                                    },
+                                    "default": []
+                                }
+
+                            }
+                        }
+                    }
+                }
+
+            }
+        },
         "gui" : {
             "type" : "object",
             "description": "controls legacy gui of qcodes",

qcodes/dataset/data_set.py

@@ -3,6 +3,7 @@
 from typing import Any, Dict, List, Optional, Union, Sized, Callable
 from threading import Thread
 import time
+import importlib
 import logging
 import uuid
 from queue import Queue, Empty
@@ -37,6 +38,7 @@
 from qcodes.dataset.database import get_DB_location
 from qcodes.dataset.guids import generate_guid
 from qcodes.utils.deprecate import deprecate
+import qcodes.config
 
 # TODO: as of now every time a result is inserted with add_result the db is
 # saved same for add_results. IS THIS THE BEHAVIOUR WE WANT?
@@ -99,7 +101,7 @@ def __init__(self,
         self._loop_sleep_time = loop_sleep_time / 1000  # convert milliseconds to seconds
         self.min_queue_length = min_queue_length
 
-        if callback_kwargs is None:
+        if callback_kwargs is None or len(callback_kwargs) == 0:
             self.callback = callback
         else:
             self.callback = functools.partial(callback, **callback_kwargs)
@@ -756,6 +758,38 @@ def subscribe(self,
         subscriber.start()
         return subscriber_id
 
+    def subscribe_from_config(self, name: str) -> str:
+        """
+        Subscribe a subscriber defined in the `qcodesrc.json` config file to
+        the data of this `DataSet`. The definition can be found at
+        `subscription.subscribers`.
+
+        Args:
+            name: identifier of the subscriber. Equal to the key of the entry
+                in 'qcodesrc.json::subscription.subscribers'.
+        """
+        subscribers = qcodes.config.subscription.subscribers
+        try:
+            subscriber_info = getattr(subscribers, name)
+        # the dot dict behind the config does not convert the error and
+        # actually raises a `KeyError`
+        except (AttributeError, KeyError):
+            keys = ','.join(subscribers.keys())
+            raise RuntimeError(
+                f'subscribe_from_config: failed to subscribe "{name}" to DataSet '
+                f'from list of subscribers in `qcodesrc.json` (subscriptions.'
+                f'subscribers). Chose one of: {keys}')
+        # get callback from string
+        parts = subscriber_info.factory.split('.')
+        import_path, type_name = '.'.join(parts[:-1]), parts[-1]
+        module = importlib.import_module(import_path)
+        factory = getattr(module, type_name)
+
+        kwargs = {k: v for k, v in subscriber_info.subscription_kwargs.items()}
+        kwargs['callback'] = factory(self, **subscriber_info.factory_kwargs)
+        kwargs['state'] = {}
+        return self.subscribe(**kwargs)
+
     def unsubscribe(self, uuid: str) -> None:
         """
         Remove subscriber with the provided uuid

qcodes/dataset/measurements.py

@@ -17,6 +17,7 @@
 from qcodes.dataset.param_spec import ParamSpec
 from qcodes.dataset.data_set import DataSet
 from qcodes.utils.helpers import NumpyJSONEncoder
+import qcodes.config
 
 log = logging.getLogger(__name__)
 
@@ -72,6 +73,9 @@ def __init__(self, dataset: DataSet, write_period: numeric_types,
                                     callback_kwargs={'run_id':
                                                          self._dataset.run_id,
                                                      'snapshot': snapshot})
+        default_subscribers = qcodes.config.subscription.default_subscribers
+        for subscriber in default_subscribers:
+            self._dataset.subscribe_from_config(subscriber)
 
         self.write_period = float(write_period)
         self.parameters = parameters