Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve fault tolerance by handling leadership election and other metadata changes #55

Merged
merged 1 commit into from
Oct 7, 2013
Merged

Improve fault tolerance by handling leadership election and other metadata changes #55

merged 1 commit into from
Oct 7, 2013

Conversation

jimjh
Copy link
Contributor

@jimjh jimjh commented Oct 2, 2013

Inside client.py

  • when a send fails inside broker_aware, reset topics_to_brokers
  • when at least one message could not be sent, raise a FailedPayloadsException

Inside conn.py

  • when a send or recv fails, set a dirty flag
  • inside send, if the dirty flag is set, reinit connection

@jimjh
Copy link
Contributor Author

jimjh commented Oct 2, 2013

Any feedback is welcomed - however, please don't close this until the integration test suite is ready.

@mumrah
Copy link
Collaborator

mumrah commented Oct 3, 2013

@jimjh can you rebase from master to pick up @mrtheb's latest test fixes?

@jimjh
Copy link
Contributor Author

jimjh commented Oct 3, 2013

Just did a rebase - tests appear to be passing. Working on additional tests now.

(oops - I think GitHub deleted @mrtheb's comments.)

@mumrah
Copy link
Collaborator

mumrah commented Oct 3, 2013

@jimjh excellent!

Now that the tests are in a better place (thanks to @mrtheb) and we've got CI, could you add some tests hitting this functionality?

@jimjh
Copy link
Contributor Author

jimjh commented Oct 3, 2013

Yes, I am working on that right now. I will also make some of the changes recommended by @mrtheb.

@mrtheb mrtheb mentioned this pull request Oct 4, 2013
Closed
@mahendra
Copy link
Collaborator

mahendra commented Oct 4, 2013

I had opened a ticket for something similar - #40
do have a look.

@jimjh
Copy link
Contributor Author

jimjh commented Oct 4, 2013

hello travis ... knock knock

@mumrah
Copy link
Collaborator

mumrah commented Oct 4, 2013

@jimjh I'm guessing it's not building automatically since the PR can't be merged automatically. Try rebasing

@jimjh jimjh mentioned this pull request Oct 4, 2013
Closed
make changes to be more fault tolerant: clean up connections, brokers…
a6c99b2 
…, failed_messages

- add integration tests for sync producer
- add integration tests for async producer w. leadership election
- use log.exception
@jimjh
Copy link
Contributor Author

jimjh commented Oct 7, 2013

How does this look?

mumrah added a commit that referenced this pull request Oct 7, 2013
@mumrah
Merge pull request #55 from quixey/fault-tolerance
eb2c173 
Improve fault tolerance by handling leadership election and other metadata changes

Thanks, @jimjh!
@mumrah mumrah merged commit eb2c173 into dpkp:master Oct 7, 2013
@jimjh
Copy link
Contributor Author

jimjh commented Oct 8, 2013

OH YEAH.

@jimjh jimjh deleted the fault-tolerance branch April 9, 2014 18:30
@vshlapakov vshlapakov mentioned this pull request Feb 25, 2015
Merged
@huangcuiyang huangcuiyang mentioned this pull request Jan 18, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jimjh @mumrah @mahendra