Repository navigation
[WIP] feat: replace uwsgi with granian #1321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
b0c5bfd
4dcfa6f
8fb24fc
6cf5482
5ea2323
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| {% include "build/openedx/settings/granian-config" %} | ||
| {{ patch("granian-config") }} |
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| #! /bin/bash | ||
| echo "Reloading granian process..." | ||
| date +%s > /openedx/granian-reload-path |
This file was deleted.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| #!/usr/bin/env python3 | ||
| import os | ||
| from granian.cli import cli | ||
|
|
||
|
|
||
| def parse_args_file() -> list[str]: | ||
| args = [] | ||
| with open("/openedx/granian-config") as f: | ||
| for line in f: | ||
| line = line.strip() | ||
| if not line or line.startswith("#"): | ||
| # skip empty lines and comments | ||
| continue | ||
|
|
||
| # Expand environment variables like $VAR_NAME | ||
| line = os.path.expandvars(line) | ||
|
|
||
| # Split by first space (for options with values) | ||
| if " " in line: | ||
| option, value = line.split(" ", 1) | ||
| args.append(option) | ||
| args.append(value) | ||
| else: | ||
| # flags without values | ||
| args.append(line) | ||
| return args | ||
|
|
||
|
|
||
| def run(): | ||
| cli(args=parse_args_file(), standalone_mode=False) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Likewise, please comment what
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Resolved in 5ea2323 |
||
|
|
||
| if __name__ == "__main__": | ||
| run() | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In general, please add a trailing new line in your files.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Resolved in 5ea2323 |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| --interface wsgi | ||
| --host 0.0.0.0 | ||
| --port 8000 | ||
| --workers $GRANIAN_WORKERS | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not blocking, but I'm curious if you folks are planning to experiment with different threading values?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We landed on --blocking-threads 8. Granian's default for WSGI is backpressure // 2, which with our setup works out to 256 threads per worker. The problem is Django holds a MySQL connection open per thread for the whole request (we run with ATOMIC_REQUESTS), so at 256 threads two workers alone could try to open way more connections than MySQL's default limit of 151. 8 keeps us well under that while still giving us enough overlap for the usual I/O waits (DB, Mongo, Redis, etc). If anyone wants uWSGI's old one-request-per-worker behavior, they can just set --blocking-threads 1 via the granian-config patch. |
||
| --static-path-route /static | ||
| --static-path-mount /openedx/staticfiles/ | ||
| --static-path-route /media | ||
| --static-path-mount /openedx/media/ | ||
| --reload | ||
| --reload-paths /openedx/granian-reload-path | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is for watching a directory and reloading if there are file changes detected, right? Just out of curiosity, why enable this in a prod environment?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We had a |
||
| --http1-buffer-size 8192 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems really low? Is this because we're not actually expecting to get HTTP 1.1 requests?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Basically it turns out 8192 is actually the lowest value granian will accept for that flag, so we were already at the floor. It wasn't about HTTP/1.1 support (Caddy does proxy HTTP/1.1 to us), it was there to recreate uWSGI's old 8KB header limit, which used to blow up on large cookies from Open edX sessions/MFEs. We just removed the line entirely, so granian's normal (much bigger) default kicks in instead. No memory cost either, since it only allocates that buffer as needed per connection. |
||
| $SERVICE_VARIANT.wsgi:application | ||
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add a docstring explaining why this is necessary (I realize it's because granian doesn't support a config file format, but this might be confusing to people who don't know that.)
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Resolved in 5ea2323