Skip to content
This repository has been archived by the owner on Oct 7, 2022. It is now read-only.

There is a random issue with async runners: concurrent.futures._base.TimeoutError #51

Closed
plumps opened this issue Jul 18, 2016 · 2 comments
Closed

There is a random issue with async runners: concurrent.futures._base.TimeoutError #51

plumps opened this issue Jul 18, 2016 · 2 comments
Milestone
1.3.0

Comments

@plumps
Copy link
Contributor

plumps commented Jul 18, 2016

This happened with the following requirements.txt on the latest macOS el capitan. Unfortunately this error is kind hard to re-produce. It may or may not happen on the subsequent run.

requirements.txt

ipython
pyyaml
click
nagiosplugin


pypi2nix -r requirements.txt -V "3" -E "libyaml"

Downloading wheels and creating wheelhouse ...
|-> nix-shell /nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/pypi2nix/pip.nix --arg extra_build_inputs [ "libyaml" ] --argstr cache_dir /var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/cache --argstr wheelhouse_dir /var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/wheelhouse --argstr python_version python3 --argstr project_tmp_dir /var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp --arg requirements_files [ "/Users/plumps/d/pkgs/ipython/requirements.txt" ] -I nixpkgs=/Users/plumps/.nix-defexpr/channels/nixpkgs --show-trace --pure --run exit
    Collecting ipython (from -r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting pyyaml (from -r /Users/plumps/d/pkgs/ipython/requirements.txt (line 2))
    Collecting click (from -r /Users/plumps/d/pkgs/ipython/requirements.txt (line 3))
      Downloading click-6.6.tar.gz (283kB)
    Collecting nagiosplugin (from -r /Users/plumps/d/pkgs/ipython/requirements.txt (line 4))
      Downloading nagiosplugin-1.2.4.tar.gz
    Collecting setuptools>=18.5 (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
      Using cached setuptools-24.0.3.tar.gz
    Collecting decorator (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting pickleshare (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
      Downloading pickleshare-0.7.3.tar.gz
    Collecting simplegeneric>0.8 (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting traitlets>=4.2 (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting prompt_toolkit<2.0.0,>=1.0.3 (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting pygments (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting appnope (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting pexpect (from ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting ipython_genutils (from traitlets>=4.2->ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting six>=1.9.0 (from prompt_toolkit<2.0.0,>=1.0.3->ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting wcwidth (from prompt_toolkit<2.0.0,>=1.0.3->ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Collecting ptyprocess>=0.5 (from pexpect->ipython->-r /Users/plumps/d/pkgs/ipython/requirements.txt (line 1))
    Building wheels for collected packages: ipython, pyyaml, click, nagiosplugin, setuptools, decorator, pickleshare, simplegeneric, traitlets, prompt-toolkit, pygments, appnope, pexpect, ipython-genutils, six, wcwidth, ptyprocess
      Running setup.py bdist_wheel for ipython: started
      Running setup.py bdist_wheel for ipython: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for pyyaml: started
      Running setup.py bdist_wheel for pyyaml: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for click: started
      Running setup.py bdist_wheel for click: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for nagiosplugin: started
      Running setup.py bdist_wheel for nagiosplugin: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for setuptools: started
      Running setup.py bdist_wheel for setuptools: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for decorator: started
      Running setup.py bdist_wheel for decorator: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for pickleshare: started
      Running setup.py bdist_wheel for pickleshare: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for simplegeneric: started
      Running setup.py bdist_wheel for simplegeneric: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for traitlets: started
      Running setup.py bdist_wheel for traitlets: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for prompt-toolkit: started
      Running setup.py bdist_wheel for prompt-toolkit: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for pygments: started
      Running setup.py bdist_wheel for pygments: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for appnope: started
      Running setup.py bdist_wheel for appnope: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for pexpect: started
      Running setup.py bdist_wheel for pexpect: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for ipython-genutils: started
      Running setup.py bdist_wheel for ipython-genutils: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for six: started
      Running setup.py bdist_wheel for six: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for wcwidth: started
      Running setup.py bdist_wheel for wcwidth: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
      Running setup.py bdist_wheel for ptyprocess: started
      Running setup.py bdist_wheel for ptyprocess: finished with status 'done'
      Stored in directory: /private/var/folders/r5/p283946j68v_9k03dcmhy3rm0000gn/T/pypi2nix/ba0deeeba33d462d422f1a6d9d2dfbd8/tmp
    Successfully built ipython pyyaml click nagiosplugin setuptools decorator pickleshare simplegeneric traitlets prompt-toolkit pygments appnope pexpect ipython-genutils six wcwidth ptyprocess
Extracting metadata ...
|-> from appnope-0.1.0.dist-info
|-> from click-6.6.dist-info
|-> from decorator-4.0.10.dist-info
|-> from ipython-5.0.0.dist-info
|-> from ipython_genutils-0.1.0.dist-info
|-> from nagiosplugin-1.2.4.dist-info
|-> from pexpect-4.2.0.dist-info
|-> from pickleshare-0.7.3.dist-info
|-> from prompt_toolkit-1.0.3.dist-info
|-> from ptyprocess-0.5.1.dist-info
|-> from Pygments-2.1.3.dist-info
|-> from PyYAML-3.11.dist-info
|-> from setuptools-24.0.3.dist-info
|-> from simplegeneric-0.8.1.dist-info
|-> from six-1.10.0.dist-info
|-> from traitlets-4.2.2.dist-info
|-> from wcwidth-0.1.7.dist-info
Traceback (most recent call last):
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/pypi2nix/stage2.py", line 91, in fetch_metadata
    async with session.get(url) as response:
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/aiohttp/client.py", line 539, in __aenter__
    self._resp = yield from self._coro
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/aiohttp/client.py", line 183, in _request
    conn = yield from self._connector.connect(req)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/aiohttp/connector.py", line 310, in connect
    transport, proto = yield from self._create_connection(req)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/aiohttp/connector.py", line 581, in _create_connection
    local_addr=self._local_addr)
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/base_events.py", line 663, in create_connection
    sock, protocol_factory, ssl, server_hostname)
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/base_events.py", line 686, in _create_connection_transport
    yield from waiter
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/futures.py", line 358, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/tasks.py", line 290, in _wakeup
    future.result()
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/futures.py", line 266, in result
    raise CancelledError
concurrent.futures._base.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/bin/.pypi2nix-wrapped", line 3, in <module>
    pypi2nix.cli.main()
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/pypi2nix/cli.py", line 161, in main
    packages_metadata = pypi2nix.stage2.main(wheels, cache_dir)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/pypi2nix/stage2.py", line 177, in main
    return list(fetch_all_metadata(cache_dir, wheels_metadata))
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/pypi2nix/stage2.py", line 105, in fetch_all_metadata
    yield combine_metadata(cache_dir, session, loop, package, loop.run_until_complete(fetch_metadata(session, url)))
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/base_events.py", line 337, in run_until_complete
    return future.result()
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/nix/store/msx8bs1894a6slk1i1dl6fxb9r1iqlkq-python3-3.5.1/lib/python3.5/asyncio/tasks.py", line 241, in _step
    result = coro.throw(exc)
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/pypi2nix/stage2.py", line 93, in fetch_metadata
    return await response.json()
  File "/nix/store/ak25602xa6sg6lk5mas2qy2dbwwy8djb-pypi2nix-1.2.0/pkgs/aiohttp/helpers.py", line 488, in __exit__
    raise asyncio.TimeoutError
concurrent.futures._base.TimeoutError
@garbas garbas mentioned this issue Jul 20, 2016
Closed
@garbas
Copy link
Contributor

garbas commented Jul 20, 2016

mrphlip/lrrbot#135 was also seeing same problem. i guess i just dont know how to use aiohttp :)

@garbas garbas modified the milestone: 1.3.0 Jul 23, 2016
garbas added a commit that referenced this issue Jul 24, 2016
@garbas
replace aiohttp with requests
c828fb9 
i guess i was to optimistic to just use aiohttp instead of
requests. i need to make more effort to learn all the new shiny
stuff in python. let make things furst work and then worry about
performance.

this should fix #48 and #51
@garbas
Copy link
Contributor

garbas commented Jul 24, 2016

fixed in master

@garbas garbas closed this as completed Jul 24, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
1.3.0
Development

No branches or pull requests
2 participants
@garbas @plumps