Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for HTTP(S) proxies to connect() #364

Open
youngkylejan opened this issue Mar 19, 2018 · 29 comments · May be fixed by #422
Open

Add support for HTTP(S) proxies to connect() #364

youngkylejan opened this issue Mar 19, 2018 · 29 comments · May be fixed by #422

Comments

@youngkylejan
Copy link

how to make the websocket connection go through with http proxy?

@aaugustin
Copy link
Member

Are you making connections with connect or receiving connections with serve?

@youngkylejan
Copy link
Author

yep. for example, i want there exists function like
websockets.connect('host', http_proxy_host='proxy_host', http_proxy_port=port)

because my running environment is in the mainland of china where limits connections to lots of hosts overseas. I have to make websocket connections through my socks5 proxy which communicates with shadowsocksR server.

@aaugustin
Copy link
Member

OK. This isn't currently implemented.

@aaugustin aaugustin changed the title http_proxy Add support for proxies to connect() Mar 24, 2018
@aaugustin
Copy link
Member

This feature request is for implemting: https://tools.ietf.org/html/rfc7231#section-4.3.6

We must take care of handling TLS properly:

  • connect with or without TLS to the proxy
  • connect with or without TLS to the destination

This may require handling the TLS handshake manually where asyncio currently handles it.

@aaugustin aaugustin linked a pull request Jun 2, 2018 that will close this issue
@aaugustin
Copy link
Member

I have a patch that manages the four cases: TLS / not TLS between client and proxy / proxy and server.

The patch needs tests and I'm not sure how to write them.

@aaugustin
Copy link
Member

TLS between proxy and server is very messy due to https://bugs.python.org/issue23749

Also we're likely to run into issues like aio-libs/aiosmtpd#83 until we add code to handle that.

@bird
Copy link

bird commented Aug 20, 2018

According to https://bugs.python.org/issue23749 the bug was fixed as of 2018-05-28. Might be worth retaking a look at? @aaugustin

@aaugustin
Copy link
Member

I can consider that bug resolved after I drop support for all Python versions before 3.7, about five years from now.

@Yevgnen
Copy link

Yevgnen commented Sep 10, 2018

Any news?

@aaugustin
Copy link
Member

No.

@aaugustin aaugustin changed the title Add support for proxies to connect() Add support for HTTP(S) proxies to connect() Sep 23, 2018
@gpjt
Copy link

gpjt commented Dec 7, 2018

Would supporting only client proxies to which you connect over HTTP be any easier? From the discussion above, it looks like the difficulties were all when connecting to the proxy over HTTPS.

I ask because, at least as I understand it, an proxy to which you connect over HTTP is still pretty secure, even when you're using it to ultimately connect to a site over HTTPS. The pattern is:

Client makes an HTTP connection to the proxy and sends a message like

CONNECT something.com:443 HTTP/1.1

The proxy, if it's happy to proxy the request, and is able to connect to that host on that port, replies

HTTP/1.1 200 Connection established

...and then the proxy just forwards data blindly down the pipe. The data that is sent and received can be TLS (or indeed SSH or any other TCP-based protocol).

It's worth noting that everything is reasonably secure. The only information that is sent in the clear from the client to the proxy is the port number (which is effectively in the clear for all TCP connections anyway) and the hostname, which TLS handshakes already send in the clear via the SNI field.

It's possible (or perhaps likely) that I'm missing something important here, though.

@aaugustin
Copy link
Member

Partial support would be better than no support at all, indeed.

However, since there's a working PoC in #422 that does what you said and is merely missing error handling and tests, once it's complete for HTTP proxies, I don't think it would be very hard to also complete it for HTTP proxies.

@ghost
Copy link

ghost commented Oct 29, 2021

For anyone still looking for this, I recommend using the websocket-client library, which has supported proxies since 2014 and works perfectly. I honestly find the fact that this issue has been open for 3 years and has a "funding needed" label laughable. Why exactly do you need funding for such a simple feature? This is just an incredibly distasteful attempt at a cash grab. It would have been far better to say something like "I'm not willing to spend time on this, but PRs are welcome".

@lgrahl
Copy link
Collaborator

lgrahl commented Oct 29, 2021

@nyuszika7h Proxies are a pain. If you find this laughable easy, then just make a PR. PRs are always welcome. The only thing distasteful here is your comment.

@ghost
Copy link

ghost commented Oct 29, 2021

I didn't say HTTPS proxies are trivial, but definitely not the kind of thing you need to beg for funding for. There's plenty of prior art to look at.

@lgrahl
Copy link
Collaborator

lgrahl commented Oct 29, 2021

"The community house in this area could use some cleaning. Are you available tomorrow? Don't ask for payment though, cleaning has already been done to other community houses in the past."

@aaugustin
Copy link
Member

It would have been far better to say something like "I'm not willing to spend time on this, but PRs are welcome".

@nyuszika7h This is basically what the last comment before yours said — glad to see that we agree on this1

@weaming
Copy link

weaming commented Apr 29, 2022

watching this, any update?

@ghost
Copy link

ghost commented Jun 5, 2023

I can consider that bug resolved after I drop support for all Python versions before 3.7, about five years from now.

Hello from 2023! I see this comment was made in 2018 :)
And also noticed support for Python versions older than 3.7 was dropped in version 10.0, any plans to add this soon? I love the async features of this library and wouldn't want to switch to something else.

@aaugustin
Copy link
Member

Good job following up :-) No specific plans in this area at the time being, though.

@racinette
Copy link

@kyochikuto @weaming hey. You can check out my package: https://github.com/racinette/websockets_proxy . I subclassed the original connect and added to it the functionality by using a third party asyncio proxy connection package.

@weaming
Copy link

weaming commented Dec 22, 2023

Nic job! Although I'm not writing Python code now. You guys can try this.

@Maxhem2
Copy link

Maxhem2 commented Jan 5, 2024

I can consider that bug resolved after I drop support for all Python versions before 3.7, about five years from now.

Five years have passed.

@aaugustin
Copy link
Member

Yes, that obstacle no longer exists.

@ghost
Copy link

ghost commented Jan 8, 2024

You can check out my package: https://github.com/racinette/websockets_proxy . I subclassed the original connect and added to it the functionality by using a third party asyncio proxy connection package.

I just tested this and it's working flawlessly, really appreciate it! I love how it is only a wrapper around the original module and not a modified fork, this way you can easily keep it updated with the original module.

Keep up the good job!

@racinette
Copy link

@kyochikuto thanks man, appreciate it.

It can be used as drop-in replacement via monkey-patching, in case some other package using websockets needs proxy access. I didn't consider it for now, but I can extend the module, if the community needs it.

Also, please, do star the repo, so I can gain recognition.

@ghost
Copy link

ghost commented Jan 8, 2024

It can be used as drop-in replacement via monkey-patching, in case some other package using websockets needs proxy access. I didn't consider it for now, but I can extend the module, if the community needs it.

Drop-in replacements are always preferred, thanks!

Also, please, do star the repo, so I can gain recognition.

Done!

@Johnserf-Seed
Copy link

For anyone still looking for this, I recommend using the websocket-client library, which has supported proxies since 2014 and works perfectly. I honestly find the fact that this issue has been open for 3 years and has a "funding needed" label laughable. Why exactly do you need funding for such a simple feature? This is just an incredibly distasteful attempt at a cash grab. It would have been far better to say something like "I'm not willing to spend time on this, but PRs are welcome".

dude, it's been 6 years.

@aaugustin
Copy link
Member

dude, it's just a hobby, I have a real job :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants