Incompatible with Tornado v5.0

[grantjenks]

I tried starting Circus and got the following:

$ circusd --log-level debug circus.ini 
2018-04-02 16:11:30 circus[15011] arbiter.py:525 [INFO] Starting master on pid 15011
2018-04-02 16:11:30 circus[15011] sockets.py:229 [DEBUG] Socket bound at 127.0.0.1:9999 - fd: 7
2018-04-02 16:11:30 circus[15011] sockets.py:229 [DEBUG] Socket bound at 127.0.0.1:8080 - fd: 12
2018-04-02 16:11:30 circus[15011] arbiter.py:493 [INFO] sockets started
2018-04-02 16:11:30 circus[15011] util.py:1089 [ERROR] exception __init__() takes 3 positional arguments but 4 were given caught

In a debug session you can see the problem:

> /Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/circus/util.py(1091)check_future_exception_and_log()
-> if hasattr(future, "exc_info"):
(Pdb) l
1086 	    if isinstance(future, concurrent.Future):
1087 	        exception = future.exception()
1088 	        if exception is not None:
1089 	            logger.error("exception %s caught" % exception)
1090 	            import pdb;pdb.set_trace()
1091 ->	            if hasattr(future, "exc_info"):
1092 	                exc_info = future.exc_info()
1093 	                traceback.print_tb(exc_info[2])
1094 	            return exception
[EOF]
(Pdb) p traceback.print_tb(exception.__traceback__)
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/tornado/gen.py", line 315, in wrapper
    yielded = next(result)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/types.py", line 204, in __next__
    return next(self.__wrapped)
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/circus/arbiter.py", line 529, in start
    self.ctrl.start()
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/circus/controller.py", line 116, in start
    self.check_delay, self.loop)

That shows that constructing the PeriodicCallback in controller.py is causing the problem. Here's the context:

    def start(self):
        self.initialize()
        if self.check_delay > 0:
            # The specific case (check_delay < 0)
            # so with no period callback to manage_watchers
            # is probably "unit tests only"
            self.caller = ioloop.PeriodicCallback(self.manage_watchers,
                                                  self.check_delay, self.loop)
            self.caller.start()
        self.started = True

We can inspect the source for Tornado and see that the signature has changed: http://www.tornadoweb.org/en/stable/_modules/tornado/ioloop.html#PeriodicCallback There's also a comment in the classdoc:

    .. versionchanged:: 5.0
       The ``io_loop`` argument (deprecated since version 4.1) has been removed.

And if you read the "What's New" for Tornado v5 at http://www.tornadoweb.org/en/stable/releases/v5.0.0.html you see they removed the argument.

I tried fixing the problem with:

$ pip uninstall tornado
$ pip install "tornado<5"

And got further but then saw:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/circusweb/circushttpd.py", line 25, in <module>
    import tornadio2
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/tornadio2/__init__.py", line 25, in <module>
    from tornadio2.router import TornadioRouter
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/tornadio2/router.py", line 27, in <module>
    from tornadio2 import persistent, polling, sessioncontainer, session, proto, preflight, stats
  File "/Users/grantj/repos/deleteme/env/lib/python3.6/site-packages/tornadio2/persistent.py", line 143
    except Exception, ex:

So I amended the example circus.ini like so:

[circus]
statsd = 1
# httpd = 1  # <-- Commented out this line. Don't run the web server.

[watcher:webapp]
cmd = /Users/grantj/repos/deleteme/env/bin/chaussette --fd $(circus.sockets.web)
numprocesses = 3
use_sockets = True

[socket:web]
host = 127.0.0.1
port = 9999

And then it worked!

So a few things broken here:

1. Update requirements to use "tornado<5"
2. Eventually fix the io_loop argument issue.
3. The tornadio2 dependency does not support Python 3 and seems abandoned?

[k4nar]

Hopefully #1062 should fix this, at least until we support Tornado >= 5 & PyZMQ >= 0.17.

[a-sevin]

Last pull request approved was in May 11, 2017....

[diorcety]

Yes ... and the travis is broken ...

[a-sevin]

I keep an eye on Circus but, a I switched to Monit

[k4nar]

Should be fixed by the release 0.15.0.

[NadavK]

I realize that you put a lot of effort into circus, but the dependency on older packages is forcing me to choose another solution

[csadorf]

I realize that you put a lot of effort into circus, but the dependency on older packages is forcing me to choose another solution

@NadavK Did you find a suitable alternative?