Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Capture Nginx and Gunicorn logs in stdout #434

Merged
merged 3 commits into from
May 6, 2024

Conversation

microamp
Copy link
Contributor

@microamp microamp commented Apr 23, 2024

Fixes #432 and #433

I can see both Nginx and Gunicorn logs when I run kubectl logs -f locally now.

127.0.0.1 - - [23/Apr/2024:04:23:57 +0100] "GET /healthz HTTP/1.0" 200 2 "-" "kube-probe/1.28" "10.244.0.1"
127.0.0.1 - - [23/Apr/2024:04:23:57 +0100] "GET /healthz HTTP/1.0" 200 2 "-" "kube-probe/1.28" "10.244.0.1"
10.244.0.1 - - [23/Apr/2024:03:23:57 +0000] "GET /healthz HTTP/1.1" 200 2 "-" "kube-probe/1.28"
10.244.0.1 - - [23/Apr/2024:03:23:57 +0000] "GET /healthz HTTP/1.1" 200 2 "-" "kube-probe/1.28"
127.0.0.1 - - [23/Apr/2024:04:24:06 +0100] "GET / HTTP/1.0" 200 30624 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" "127.0.0.1"
127.0.0.1 - - [23/Apr/2024:03:24:06 +0000] "GET / HTTP/1.1" 200 5552 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0"
127.0.0.1 - - [23/Apr/2024:03:24:06 +0000] "GET /static/dist/main.9949552e4686.css HTTP/1.1" 200 324461 "http://localhost:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0"
127.0.0.1 - - [23/Apr/2024:03:24:06 +0000] "GET /static/img/youtube-icon.c20ebf0e90c8.svg HTTP/1.1" 200 930 "http://localhost:8080/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0"

However, I'm not sure about the second commit. Configuring the Grafana agent to read from a file path may be cleaner (e.g. /var/log/nginx/access.log). To be discussed further.

Update: Fixing #433 in the same PR now.

@microamp microamp requested review from kesara and NGPixel April 23, 2024 04:11
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.80%. Comparing base (9a3f9f6) to head (3a0acad).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #434   +/-   ##
=======================================
  Coverage   82.80%   82.80%           
=======================================
  Files         126      126           
  Lines        2611     2611           
=======================================
  Hits         2162     2162           
  Misses        449      449           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@NGPixel NGPixel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Output should be in json format for easier tagging

@microamp
Copy link
Contributor Author

Output should be in json format for easier tagging

Currently as a separate ticket here, #433, but happy to combine them

@microamp
Copy link
Contributor Author

Application logs are now in JSON format. e.g.

{"remote_host": "127.0.0.1", "remote_logname": "-", "remote_user": "-", "timestamp": "[24/Apr/2024:10:04:55 +0100]", "request": "GET /announcements/ HTTP/1.0", "status_code": "200", "response_size": "37152", "referrer": "http://localhost:8080/announcements/reappointment-of-eliot-lear-as-independent-submission-editor/", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36", "x_forwarded_for": "127.0.0.1"}

Pretty-printed:

{
  "remote_host": "127.0.0.1",
  "remote_logname": "-",
  "remote_user": "-",
  "timestamp": "[24/Apr/2024:10:04:55 +0100]",
  "request": "GET /announcements/ HTTP/1.0",
  "status_code": "200",
  "response_size": "37152",
  "referrer": "http://localhost:8080/announcements/reappointment-of-eliot-lear-as-independent-submission-editor/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36",
  "x_forwarded_for": "127.0.0.1"
}

"response_size": "%(b)s",
"referrer": "%(f)s",
"user_agent": "%(a)s",
"x_forwarded_for": "%({x-forwarded-for}i)s",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"%({X-Forwarded-For}i)s"

is now all lowercase, i.e.

"%({x-forwarded-for}i)s"

as per the official documentation

Use lowercase for header and environment variable names, and put {...}x names inside %(...)s. For example:

%({x-forwarded-for}i)s

https://docs.gunicorn.org/en/stable/settings.html#access-log-format

@microamp microamp requested a review from NGPixel April 24, 2024 10:02
@microamp
Copy link
Contributor Author

microamp commented May 5, 2024

@NGPixel Mind having another look when you can please?

@microamp microamp merged commit cd14fde into ietf-tools:main May 6, 2024
5 checks passed
@microamp microamp deleted the chore/logging-to-stdout branch May 6, 2024 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Logging: Redirect stdout from child processes to Supervisor
4 participants