aiohttp request with proxy params in ccxt.async

[dhmk815]

Hello.
I want to fetch orderbooks from bittrex using ccxt.async through my proxy server.
I know that fetch_order_book uses aiohttp module in it, and I want to pass my proxy address to aiohttp.ClientSession().get() as a parameter, as shown below.

async with aiohttp.ClientSession() as session:
    async with session.get("http://python.org",
                           proxy="http://some.proxy.com") as resp:
        print(resp.status)

So, I think I need to override fetch_order_book().
How do I get through this?

[kroitor]

Hi! Thx for opening this issue! I'll add support for passing the existing proxy property to the session in a few minutes.

[kroitor]

This feature is available as of 1.9.214+. Cheers!

[dhmk815]

Thank you very much. If you don't mind, could you show me how to use it?

[kroitor]

Just like you would do with any other property of a class instance:

# initialization

import ccxt.async as ccxt
exchange = ccxt.bittrex ({
    'aiohttp_proxy': YOUR_PROXY,
})

# or 

exchange = ccxt.bittrex ()
exchange.aiohttp_proxy = YOUR_PROXY

# the rest of your code goes here, fetching tickers, orderbooks, etc...

[dhmk815]

@kroitor Thank you :)

[dhmk815]

@kroitor I think non-async ccxt module doesn't support proxy too. When I try it, it doesn't work... Can you check it please?

[kroitor]

@dhmk815 the non-async module currently only supports CORS proxies, which are URLs that are prepended to the target URL of the exchange, more about it here. Let me know if this answers your question.

[dhmk815]

@kroitor I got it. I didn't know about the concept of CORS proxies. When I tried it using non-async module again just before, I got an error message from bittrex, saying some reasons like DDos Protection or rate-limiting.
Do you have a plan to support other proxy concept to avoid it?

[kroitor]

Yes, see here ;) Basically, all your questions most likely have already been answered somewhere nearby ;) Also, I'd recommend reading the Manual and browsing through examples. Just take a look around ;) That is until we implement the improved support for proxies, and we will notify our users of it then. Stay tuned, more updates coming soon )

[dhmk815]

@kroitor Thank you. I always appreciate your effort.

[xgdgsc]

I want to use fetch_ohlcv with proxy in async. I have a use case that when setting both exchange.proxy and exchange.aiohttp_proxy it would raise errors regarding Invalid URL and doesn' t work. When I only set exchange.aiohttp_proxy it works. It makes me wonder whether it is using the proxy or not. For example, the exchange could be binance.

[kroitor]

@xgdgsc without seeing your instantiation code it's hard to tell the reason, but most likely you don't need to use both .proxy and .aiohttp_proxy. The .proxy is for CORS only, whereas the .aiohttp_proxy is for controlling the tunnelling agent (use with SOCKS and HTTP proxies). Hope it helps.

[xgdgsc]

I just want to make sure when I set exchange.aiohttp_proxy , exchange.fetch_ohlcv would use that proxy.

[kroitor]

@xgdgsc it should (we tested it earlier). Let us know if you have any difficulties with it.

[xgdgsc]

OK. Thanks!

[authurlan]

Traceback (most recent call last):
...
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/ccxt/async_support/base/exchange.py", line 177, in load_markets
markets = await self.fetch_markets(params)
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/ccxt/async_support/fcoin.py", line 149, in fetch_markets
response = await self.publicGetSymbols()
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/ccxt/async_support/base/exchange.py", line 116, in fetch2
return await self.fetch(request['url'], request['method'], request['headers'], request['body'])
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/ccxt/async_support/base/exchange.py", line 138, in fetch
proxy=self.aiohttp_proxy) as response:
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/aiohttp/client.py", line 1005, in aenter
self._resp = await self._coro
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/aiohttp/client.py", line 466, in _request
ssl=ssl, proxy_headers=proxy_headers, traces=traces)
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 292, in init
self.update_proxy(proxy, proxy_auth, proxy_headers)
File "/home/xxx/.local/share/virtualenvs/pytrader-r6KJuWFt/lib/python3.6/site-packages/aiohttp/client_reqrep.py", line 520, in update_proxy
raise ValueError("Only http proxies are supported")
ValueError: Only http proxies are supported

aiohttp_proxy seems not supporting socks5 proxy. ccxt version 1.18.195.

[owneroxxor]

Today, 19/06/2019, do we have support for SOCKS4 and SOCKS5 proxies for aiohttp_proxy param? If so, any exemple on using it?

[kroitor]

@owneroxxor using an external session + socks connector could work:

• https://github.com/romis2012/aiohttp-socks

pip install aiohttp_socks

The example code could look close to this:

import ccxt.async_support as ccxt
import aiohttp
import aiohttp_socks

connector = aiohttp_socks.SocksConnector.from_url('socks5://user:[email protected]:1080')
session = aiohttp.ClientSession(connector=connector)

exchange = ccxt.binance({
    'session': session,
    'enableRateLimit': True,
    # ...
})

# ...

Make sure you await session.close() when you don't need it anymore.

[owneroxxor]

Wow awsome! I can bet many out there were searching for this. Thanks a lot.

@owneroxxor using an external session + socks connector could work:

• https://github.com/romis2012/aiohttp-socks

pip install aiohttp_socks

The example code could look close to this:

import ccxt.async_support as ccxt
import aiohttp
import aiohttp_socks

connector = aiohttp_socks.SocksConnector.from_url('socks5://user:[email protected]:1080')
session = aiohttp.ClientSession(connector=connector)

exchange = ccxt.binance({
    'session': session,
    'enableRateLimit': True,
    # ...
})

# ...

Make sure you await session.close() when you don't need it anymore.

[donaldtrieuit]

Hi everyone, I am using to a proxy of smartproxy and this is not work.

proxy: http://username:[email protected]:7000

self.exchange = exchange_class({
                'apiKey': self.my_exchange.api_key,
                'secret': self.my_exchange.api_secret,
                'aiohttp_proxy': proxy,
                'aiohttp_trust_env': True,
                'verbose': True,
                'logger': loggerccxt,
            })

Thanks