2.x #17
Merged
2.x #17
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…quest header warc-prefix setting
…actor warc writer thread to use new WarcWriterPool class
… solve hanging connection issue (see comment in code)
…ails now, limits not implemented)
… informative text/plain body, in "420 Limit reached" response
…man-in-the-middle recording proxy code from warcproxy.py into mitmproxy.py
…eriodically in batches, instead of starting the timer in __init__
…orcing limits host limits using those stats, with tests
…into new start() and shutdown() methods, for easier integration into a separate python program
…t practice according to Python Packaging Authority (eases testing, etc)
… than regular (hard) limits (which result in a 420)
…ince rules apply to subdomains as well
… to the host and subdomains
…on mac however, let's see what happens on travis-ci)
…kdb tables if they don't already exist... warcprox normally creates them on demand at startup, but if multiple instances are starting up at the same time, you can end up with duplicate broken tables, so it's a good idea to use this utility when spinning up a cluster
…eed work (it can be configured per job or per site)
…rfere with other warcprox operations
…-length http response header (not chunked and server leaves connection open) -- reported by Alex Osborne
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.