Merge branch 'release/0.1.8'

CHANGES.rst

@@ -1,3 +1,10 @@
+0.1.8
+-----
+
+- Fix a bug which could cause descriptor fetching to crash and stall if an
+  old instance descriptor was retrieved from a HSDir. #64
+- Minors fixes to documentation and addition of a tutorial.
+
 0.1.7
 -----
 

README.rst

@@ -25,7 +25,9 @@ Installation and usage documentation is available at https://onionbalance.readth
 Contact
 -------
 
-This software is under active development and likely contains many bugs. Please open bugs on Github if you discover any issues with the software or documentation.
+This software is under active development and likely contains bugs. Please
+open bug reports on Github if you discover any issues with the software or
+documentation.
 
 I can also be contacted by PGP email or on IRC at ``#onionbalance`` on the OFTC network.
 

docs/conf.py

@@ -21,7 +21,7 @@
 from docutils.utils import get_source_line
 
 # Documentation configuration
-__version__ = '0.1.7'
+__version__ = '0.1.8'
 __author__ = "Donncha O'Cearbhaill"
 __contact__ = "[email protected]"
 

docs/getting-started.rst

@@ -10,7 +10,7 @@ servers and multiple backend application servers.
 Architecture
 ------------
 
-The management server runs the OnionBalance daemon. OnionBalance combines the routing information (the introduction points) for multiple backend onion services instances and publishes this information in a master descriptor..
+The management server runs the OnionBalance daemon. OnionBalance combines the routing information (the introduction points) for multiple backend onion services instances and publishes this information in a master descriptor.
 
 .. image:: ../onionbalance.png
 

docs/index.rst

@@ -31,6 +31,10 @@ regularly:
 Quickstart
 ==========
 
+The OnionBalance ::ref::`tutorial` describes the deployment of an onion service with
+multiple backed Tor instances and web servers. The following is a minimal
+quickstart guide for a new installation.
+
 Assuming there is no previous configuration in ``/etc/onionbalance``:
 
 .. code-block:: console
@@ -73,6 +77,7 @@ instructions.
    getting-started
    running-onionbalance
    use-cases
+   tutorial
 
 .. toctree::
    :maxdepth: 2

docs/installation.rst

@@ -23,8 +23,8 @@ or
     $ cd onionbalance
     # python setup.py install
 
-If you are running Debian unstable or testing, you can install OnionBalance
-with the following command:
+If you are running Debian Jessie (with backports enabled) or later you
+can install OnionBalance with the following command:
 
 .. code-block:: console
 

docs/running-onionbalance.rst

@@ -118,7 +118,7 @@ LOG_LEVEL
 
 REFRESH_INTERVAL
   How often to check for updated backend hidden service descriptors. This
-  value can be increased if your backend instance are under heavy loaded
+  value can be decreased if your backend instance are under heavy loaded
   causing them to rotate introduction points quickly.
   (default: 600 seconds).
 

docs/tutorial.rst

@@ -0,0 +1,102 @@
+.. _tutorial:
+
+Tutorial
+========
+
+This is a step-by-step tutorial to help you configure OnionBalance.
+
+OnionBalance implements `round-robin` like load balancing on top of Tor
+onion services. A typical OnionBalance deployment will incorporate one management
+servers and multiple backend application servers.
+
+Assumptions
+-----------
+
+You want to run:
+
+- one or more OnionBalance processes, to perform load balancing, on hosts
+  named ``obhost1``, ``obhost2``.
+- two or more Tor processes, to run the Onion Services, on hosts named
+  ``torhost1``, ``torhost2``.
+- two or more servers (e.g. web servers) or traditional load balancers on
+  hosts named ``webserver1``, ``webserver2``.
+
+Scaling up:
+
+- the number of ``obhostX`` can be increased but this will not help handling
+  more traffic.
+- the number of ``torhostX`` can be increased up to 60 instances to handle
+  more traffic.
+- the number of ``webserverX`` can be increased to handle more traffic until
+  the Tor daemons in front of them become the bottleneck.
+
+Scaling down:
+
+- the three type of services can be run on the same hosts. The number of hosts
+  can scale down to one.
+
+Reliability:
+
+Contrarily to traditional load balancers, the OnionBalance daemon does not
+receive and forward traffic. As such, ``obhostX`` does not need to be in
+proximity to ``torhostX`` and can be run from any location on the Internet.
+Failure of ``obhostX`` will not affect the service as long as either one
+``obhost`` is still up or or the failure is shorter than 30 minutes.
+
+Other assumptions:
+
+- the hosts run Debian or Ubuntu
+- there is no previous configuration
+
+Configuring the OnionBalance host
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+On ``obhost1``:
+
+.. code-block:: bash
+
+   sudo apt-get install onionbalance tor
+   mkdir -p /var/run/onionbalance
+   chown onionbalance:onionbalance /var/run/onionbalance
+   /usr/sbin/onionbalance-config -n <number_of_torhostX> --service-virtual-port <port> \
+       --service-target <ipaddr:port>  --output ~/onionbalance_master_conf
+   sudo cp ~/onionbalance_master_conf/master/*.key /etc/onionbalance/
+   sudo cp ~/onionbalance_master_conf/master/config.yaml /etc/onionbalance/
+   sudo chown onionbalance:onionbalance /etc/onionbalance/*.key
+   sudo service onionbalance restart
+   sudo tail -f /var/log/onionbalance/log
+
+Back up the files in ``~/onionbalance_master_conf``.
+
+If you have other ``obhostX``:
+
+.. code-block:: bash
+
+   sudo apt-get install onionbalance
+   mkdir -p /var/run/onionbalance
+   chown onionbalance:onionbalance /var/run/onionbalance
+
+Copy ``/etc/onionbalance/\*.key`` and ``/etc/onionbalance/config.yml``
+from ``obhost1`` to all hosts in ``obhostX``.
+
+Check the logs. The following warnings are expected:
+`"Error generating descriptor: No introduction points for service ..."`.
+
+Configuring the Tor services
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Copy the ``instance_torrc`` and ``private_key`` files from each of the
+directories named ``./config/srv1``, ``./config/srv2``,.. on ``obhost1``
+to ``torhostX`` - the contents of one directory for each ``torhostX``.
+
+Configure and start the services - the onion service on OnionBalance should
+be ready within 10 minutes.
+
+Monitoring
+~~~~~~~~~~
+
+On each ``obhostX``, run:
+
+.. code-block:: bash
+
+   sudo watch 'socat - unix-connect:/var/run/onionbalance/control'

onionbalance/__init__.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 
-__version__ = "0.1.7"
+__version__ = "0.1.8"
 __author__ = "Donncha O'Cearbhaill"
 __contact__ = "[email protected]"
 __url__ = "https://github.com/DonnchaC/onionbalance"

onionbalance/eventhandler.py

@@ -28,7 +28,12 @@ def new_status(status_event):
         if status_event.status_type == stem.StatusType.GENERAL:
             if status_event.action == "CONSENSUS_ARRIVED":
                 # Update the local view of the consensus in OnionBalance
-                consensus.refresh_consensus()
+                try:
+                    consensus.refresh_consensus()
+                except Exception:
+                    logger.exception("An unexpected exception occured in the "
+                                     "when processing the consensus update "
+                                     "callback.")
 
     @staticmethod
     def new_desc(desc_event):
@@ -59,7 +64,11 @@ def new_desc_content(desc_content_event):
             return None
 
         # Send content to callback function which will process the descriptor
-        descriptor.descriptor_received(descriptor_text)
+        try:
+            descriptor.descriptor_received(descriptor_text)
+        except Exception:
+            logger.exception("An unexpected exception occured in the "
+                             "new descriptor callback.")
 
         return None
 

onionbalance/instance.py

@@ -125,7 +125,7 @@ def update_descriptor(self, parsed_descriptor):
                          self.onion_address,
                          parsed_descriptor.published,
                          self.timestamp)
-            return
+            return False
         else:
             self.timestamp = parsed_descriptor.published