Skip to content

chore: Make sure each container / pod has its own stdout log files#36205

Merged
nidhi-nair merged 3 commits intoreleasefrom
chore/clean-stale-file-error
Sep 10, 2024
Merged

chore: Make sure each container / pod has its own stdout log files#36205
nidhi-nair merged 3 commits intoreleasefrom
chore/clean-stale-file-error

Conversation

@nidhi-nair
Copy link
Contributor

@nidhi-nair nidhi-nair commented Sep 9, 2024

Description

Log files from stdout, when undefined were defaulting to the supervisord child dir random fd paths. These files got deleted on supervisor process restarts, which meant that successive rollouts would always end up with stale file handles. We're now providing a deterministic path for these logs as well, rolled over and backed up similar to other files.

Warning

Tests have not run on the HEAD 5f99008 yet


Mon, 09 Sep 2024 16:07:19 UTC

Summary by CodeRabbit

  • New Features
    • Enhanced logging capabilities for better monitoring and management of log files.
    • Introduced dynamic paths for log files based on environment variables.
    • Added log file size management and backup retention settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Sep 9, 2024

Walkthrough

The pull request introduces enhancements to the supervisord.conf file for the Appsmith supervisor, specifically focusing on logging configurations. New parameters have been added to define log file paths, manage log sizes, and control the number of backup files. These changes enable dynamic resolution of log directories and hostnames, thereby improving the overall logging capabilities of the supervisor.

Changes

File Path Change Summary
deploy/docker/fs/etc/supervisor/supervisord.conf Added logging parameters, including paths for stdout and stderr logs, max byte limits, and backup retention settings.

Possibly related PRs

Poem

In the land of logs where data flows,
New paths for output and errors arose.
With backups aplenty, and sizes just right,
The supervisor's logging shines ever so bright.
Monitoring made easy, a sight to behold,
In the world of Appsmith, new stories unfold! 🌟


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the skip-changelog Adding this label to a PR prevents it from being listed in the changelog label Sep 9, 2024
@nidhi-nair nidhi-nair changed the title chore: Make sure each container / pod have their own stdout log files chore: Make sure each container / pod has its own stdout log files Sep 9, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 205ba07 and 5f99008.

Files selected for processing (1)
  • deploy/docker/fs/etc/supervisor/supervisord.conf (1 hunks)
Additional context used
Learnings (2)
Common learnings
Learnt from: sharat87
PR: appsmithorg/appsmith#30252
File: deploy/docker/fs/usr/lib/python3/dist-packages/supervisor/appsmith_supervisor_stdout.py:21-29
Timestamp: 2024-01-12T01:41:30.885Z
Learning: The user has confirmed that the suggested changes to handle potential exceptions and improve the robustness of the `main` function in `appsmith_supervisor_stdout.py` are acceptable.
deploy/docker/fs/etc/supervisor/supervisord.conf (1)
Learnt from: sharat87
PR: appsmithorg/appsmith#30252
File: deploy/docker/fs/usr/lib/python3/dist-packages/supervisor/appsmith_supervisor_stdout.py:21-29
Timestamp: 2024-01-12T01:41:30.885Z
Learning: The user has confirmed that the suggested changes to handle potential exceptions and improve the robustness of the `main` function in `appsmith_supervisor_stdout.py` are acceptable.
Additional comments not posted (6)
deploy/docker/fs/etc/supervisor/supervisord.conf (6)

42-42: Ensure environment variable usage is secure and validated.

The use of environment variables in the log file path (%(ENV_APPSMITH_LOG_DIR)s/supervisor/access-supervisor-%(ENV_HOSTNAME)s.log) is a common practice to allow dynamic configuration. However, ensure that these environment variables are properly validated and sanitized to prevent potential security issues, such as directory traversal attacks.


43-43: Confirm environment variable consistency and security.

Similar to the stdout log file, the stderr log file path uses environment variables (%(ENV_APPSMITH_LOG_DIR)s/supervisor/error-supervisor-%(ENV_HOSTNAME)s.log). It's crucial to ensure that these variables are consistently set across different deployment environments and are secured against unauthorized modifications.


44-44: Review the log file size limit configuration.

Setting stdout_logfile_maxbytes=10MB is intended to manage the size of log files effectively. Confirm that this size limit aligns with the operational requirements and storage capabilities of the deployment environment. Additionally, consider the implications of this setting on log rotation and archival strategies.


45-45: Check stderr log file size configuration.

The configuration for stderr_logfile_maxbytes=10MB mirrors that of stdout. It's important to ensure that this setting is appropriate for the expected volume of error logs and does not lead to unexpected disk space issues.


46-46: Validate the backup configuration for stdout logs.

The setting stdout_logfile_backups=10 specifies the number of backup files to retain. Verify that this number is suitable for your logging needs and storage limits, and consider how it integrates with your overall log management policy.


47-47: Ensure stderr log backups are appropriately configured.

Similarly, stderr_logfile_backups=10 needs to be assessed to ensure it meets the operational and storage requirements of the system. This setting should be consistent with the policies for managing stdout log backups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Adding this label to a PR prevents it from being listed in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants