Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Window size config #1975

Merged
merged 19 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ Unreleased
- :func:`.stream_generator` now accepts the ``continue_after_id`` parameter, which
starts the stream after a given item ID.
- Support for new share URL format created from Reddit's mobile apps.
- :class:`.Reddit` has a new configurable parameter, ``window_size``. This tells PRAW
how long reddit's rate limit window is. This defaults to 600 seconds and shouldn't
need to be changed unless reddit changes the size of their rate limit window.

**Fixed**

Expand Down
1 change: 1 addition & 0 deletions docs/getting_started/configuration/options.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,7 @@ These are options that do not belong in another category, but still play a part
throwing an exception.
:warn_comment_sort: When ``true``, log a warning when the ``comment_sort`` attribute of
a submission is updated after ``_fetch()`` has been called (default: ``true``).
:window_size: The number of seconds between rate limit resets (default: 600).

.. _custom_options:

Expand Down
23 changes: 21 additions & 2 deletions docs/getting_started/logging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,34 @@ Add the following to your code to log everything available:
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)

Or you can use the following to write the logs to a file for longer running bots or
scripts when you need to look back at what the bot did hours ago.

.. code-block:: python

import logging

stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.DEBUG)
file_handler = logging.handlers.RotatingFileHandler(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be too advanced. Maybe add a second example explaining why you recommend a rotating file handler.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

"praw_log.txt", maxBytes=1024 * 1024 * 16, backupCount=5
)
file_handler.setLevel(logging.DEBUG)
for logger_name in ("praw", "prawcore"):
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
logger.addHandler(stream_handler)
logger.addHandler(file_handler)

When properly configured, HTTP requests that are issued should produce output similar to
the following:

.. code-block:: text

Fetching: GET https://oauth.reddit.com/api/v1/me
Fetching: GET https://oauth.reddit.com/api/v1/me at 1691743155.4952002
Data: None
Params: {'raw_json': 1}
Response: 200 (876 bytes)
Response: 200 (876 bytes) (rst-45:rem-892:used-104 ratelimit) at 1691743156.3847592
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is from prawcore but what do you think about changing the epoch to a timestamp? It would only be 1 or 2 more characters and would be human readable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this to match the timestamp in the first line. But no real opinion whether they should both be timestamps or a human readable date.


Furthermore, any API ratelimits from POST actions that are handled will produce a log
entry with a message similar to the following message:
Expand Down
1 change: 1 addition & 0 deletions praw/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ def _initialize_attributes(self):
self.warn_additional_fetch_params = self._config_boolean(
self._fetch_default("warn_additional_fetch_params", default=True)
)
self.window_size = self._fetch_default("window_size", default=600)
self.kinds = {
x: self._fetch(f"{x}_kind")
for x in [
Expand Down
9 changes: 7 additions & 2 deletions praw/models/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,9 @@ def authorize(self, code: str) -> str | None:
authenticator = self._reddit._read_only_core._authorizer._authenticator
authorizer = Authorizer(authenticator)
authorizer.authorize(code)
authorized_session = session(authorizer)
authorized_session = session(
authorizer=authorizer, window_size=self._reddit.config.window_size
)
self._reddit._core = self._reddit._authorized_core = authorized_session
return authorizer.refresh_token

Expand All @@ -78,7 +80,10 @@ def implicit(self, *, access_token: str, expires_in: int, scope: str):
if not isinstance(authenticator, UntrustedAuthenticator):
raise InvalidImplicitAuth
implicit_session = session(
ImplicitAuthorizer(authenticator, access_token, expires_in, scope)
authorizer=ImplicitAuthorizer(
authenticator, access_token, expires_in, scope
),
window_size=self._reddit.config.window_size,
)
self._reddit._core = self._reddit._authorized_core = implicit_session

Expand Down
16 changes: 12 additions & 4 deletions praw/reddit.py
Original file line number Diff line number Diff line change
Expand Up @@ -549,7 +549,9 @@ def _prepare_common_authorizer(
else:
self._core = self._read_only_core
return
self._core = self._authorized_core = session(authorizer)
self._core = self._authorized_core = session(
authorizer=authorizer, window_size=self.config.window_size
)

def _prepare_objector(self):
mappings = {
Expand Down Expand Up @@ -628,13 +630,17 @@ def _prepare_trusted_prawcore(self, requestor: prawcore.requestor.Requestor):
self.config.redirect_uri,
)
read_only_authorizer = ReadOnlyAuthorizer(authenticator)
self._read_only_core = session(read_only_authorizer)
self._read_only_core = session(
authorizer=read_only_authorizer, window_size=self.config.window_size
)

if self.config.username and self.config.password:
script_authorizer = ScriptAuthorizer(
authenticator, self.config.username, self.config.password
)
self._core = self._authorized_core = session(script_authorizer)
self._core = self._authorized_core = session(
authorizer=script_authorizer, window_size=self.config.window_size
)
else:
self._prepare_common_authorizer(authenticator)

Expand All @@ -643,7 +649,9 @@ def _prepare_untrusted_prawcore(self, requestor: prawcore.requestor.Requestor):
requestor, self.config.client_id, self.config.redirect_uri
)
read_only_authorizer = DeviceIDAuthorizer(authenticator)
self._read_only_core = session(read_only_authorizer)
self._read_only_core = session(
authorizer=read_only_authorizer, window_size=self.config.window_size
)
self._prepare_common_authorizer(authenticator)

def _resolve_share_url(self, url: str) -> str:
Expand Down
Loading