Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix timeouterror on multihost config #1022

Closed
wants to merge 5 commits into from

Conversation

JMarkin
Copy link

@JMarkin JMarkin commented Apr 15, 2023

Hello,

Problem:
If I try to connect with dsn like "postgresql://postgres@host1,host2,host3:5432/postgres". If I have timeouterror on host1 asyncpg always skips host2 and host3.

For example:

  1. I try to connect to with dsn, timeout 60
  2. host1 doesn't break connection but doesn't resolve connection without oserror
  3. breaks me by asyncio.TimeoutError on 60 sec
  4. in code timeout=time.monotinic() - before in this case always be >=60
  5. result: host2 and host3 always skip

In this PR I try to fix with problem

@elprans
Copy link
Member

elprans commented Oct 9, 2023

Thanks for the PR! The fix in #1087 is more comprehensive, so closing this in favor of that.

@elprans elprans closed this Oct 9, 2023
@inikolaev
Copy link

inikolaev commented May 12, 2024

@elprans I think your fix does not really address the problem raised here. I had ran into a similar issue about a year ago and I described it in a discussion, but never followed up.

The problem is that the connection timeout configuration sets the overall connection timeout.

Consider this case: we have two hosts, we have configured the timeout of 10 seconds and first host does not respond within 10 seconds so the timeout context manager fires up. In such case we never make an attempt to connect with the second host.

I believe we might need a separate setting here, like one introduced in this PR, that would allow to configure both the overall connect timeout and per host connect timeout.

I have just tested the new version and I'm still running into the same issue. I found a peculiar case though: if I set timeout to 80 seconds, my example works, but it takes 75 seconds to succeed. It looks like 75 seconds is some internal timeout somewhere that kicks-in earlier and gives a chance to try to connect with another host.

I did the same for psycopg2 which handles this case and what they do is apply the same timeout out for each host. So if I set timeout to 2 seconds and both hosts unavailable the overall timeout will be 4 seconds. We could employ the same strategy with asyncpg - at least it would be consistent with psycopg2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants