Skip to content

feat(server): harden deployment and add log management - #195

Merged
YaoYinYing merged 9 commits into
mainfrom
codex/require-server-env-settings
Jul 29, 2026
Merged

feat(server): harden deployment and add log management#195
YaoYinYing merged 9 commits into
mainfrom
codex/require-server-env-settings

Conversation

@YaoYinYing

@YaoYinYing YaoYinYing commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • require explicit server, UniRef database, and administrator settings before startup touches SQLite
  • generate bootstrap administrator passwords transiently and use one ephemeral signing key per preloaded web launch
  • make the Gist manifest build path work for both initial upload and later updates
  • add scheduled, line-count, and total-size log rotation with ZIP compression and archive pruning
  • add a standalone admin log viewer that lazily streams the four active service logs
  • group rotated logs in a lazy file tree with individual ZIP downloads
  • restrict log access to fixed active-log identifiers and exact rotator-generated archive names, with traversal and symlink regression coverage

Why

Deployment previously retained unsafe implicit defaults and configurable signing-secret infrastructure, while server logs could grow without bounds and were not inspectable from the admin web interface. This change fails closed on required deployment settings, keeps bootstrap credentials and signing material transient, and adds bounded, administrator-only log operations.

Operational impact

  • SERVER_DIR, DB_UNIREF30, DB_UNIREF90, and ADMIN_USERS are mandatory
  • web restarts intentionally invalidate existing login, verification, and password-reset tokens
  • log rotation remains disabled when ROTATE_LOG_MAX_LINENO, ROTATE_LOG_PERIOD, and MAX_LOG_SIZE are all unset
  • ROTATE_LOG_PERIOD accepts a quoted five-field crontab expression such as "0 0 * * *"
  • administrators receive a Server Logs entry in the dashboard header; arbitrary filesystem browsing is not exposed

Validation

  • make -C server test — 259 passed
  • focused admin log streaming, archive listing/download, and path-traversal tests
  • JavaScript syntax validation for the log viewer
  • Python byte-compilation for changed server modules
  • Docker Compose production configuration validation
  • git diff --check

Summary by CodeRabbit

  • New Features

    • Added an admin-only server log viewer with live streaming for key services and downloads of rotated log archives.
    • Added automatic log rotation with optional scheduling, line thresholds, and total archive-size limits.
    • Added support for configuring multiple administrators and securely displaying one-time bootstrap credentials.
  • Bug Fixes

    • Tokens now expire when the server restarts, improving security.
    • Startup and restart now fail early when required deployment settings are missing.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@YaoYinYing, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 31 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 85694c6d-9496-4db9-be6e-defa0b481052

📥 Commits

Reviewing files that changed from the base of the PR and between f8da9f6 and 53c46d9.

📒 Files selected for processing (10)
  • CHANGELOG.md
  • Makefile
  • docs/dev-guide/server.md
  • server/.env.example
  • server/README.md
  • server/pssm_gremlin_server/maintenance/tasks/log_rotation.py
  • server/pssm_gremlin_server/static/js/log-viewer.js
  • server/run/restart_pssm_flask.sh
  • server/tests/test_log_rotation.py
  • server/tests/test_process_isolation.py
📝 Walkthrough

Walkthrough

The change makes GREMLIN deployment settings mandatory, replaces persistent token signing with an ephemeral key, adds scheduled log rotation and archive limits, introduces an admin-only log viewer, updates bootstrap behavior and documentation, and extracts Gist manifest generation into a standalone tool.

Changes

GREMLIN server operations

Layer / File(s) Summary
Required deployment settings and ephemeral authentication
server/pssm_gremlin_server/config.py, server/pssm_gremlin_server/auth.py, server/pssm_gremlin_server/pssm_gremlin.py, server/run/restart_pssm_flask.sh, server/docker-compose.yml, server/tests/*, server/README.md, docs/dev-guide/server.md
Required paths and ADMIN_USERS now fail before database setup; configured administrators receive bootstrap credentials, and token signing keys are regenerated on startup.
Scheduled log rotation and archive limits
server/pssm_gremlin_server/maintenance/*, server/.env.example, server/docker-compose.yml, server/tests/test_log_rotation.py, server/tests/test_maintenance_manager.py
Maintenance archives non-empty logs, truncates active files, schedules threshold or periodic jobs, and prunes archives under configured size limits.
Admin log viewer and protected archive access
server/pssm_gremlin_server/routes.py, server/pssm_gremlin_server/templates/*, server/pssm_gremlin_server/static/*, server/tests/test_admin.py, server/tests/test_security_advanced.py
Administrators can stream four managed active logs and lazily list or download validated rotated ZIP archives through the new viewer interface and endpoints.

Gist manifest publishing

Layer / File(s) Summary
Standalone Gist manifest generation
tools/generate_gist_manifest.py, Makefile
A dedicated script generates HMAC manifests, while upload-gists updates an existing manifest or adds one when absent.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Admin
  participant LogViewer
  participant GREMLINRoutes
  participant LogDirectory
  Admin->>LogViewer: Open Server Logs
  LogViewer->>GREMLINRoutes: Request active log
  GREMLINRoutes->>LogDirectory: Validate and stream managed file
  LogDirectory-->>GREMLINRoutes: Log chunks
  GREMLINRoutes-->>LogViewer: Stream response
  Admin->>LogViewer: Expand rotated archives
  LogViewer->>GREMLINRoutes: Request archive metadata
  GREMLINRoutes->>LogDirectory: List validated ZIP archives
  LogDirectory-->>GREMLINRoutes: Grouped archive metadata
  GREMLINRoutes-->>LogViewer: Render archive tree and download links
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.40% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the two main themes: stricter deployment requirements and new server log management features.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Fix failing CI checks
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/require-server-env-settings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@YaoYinYing
YaoYinYing marked this pull request as ready for review July 29, 2026 02:38
@deepsource-io

deepsource-io Bot commented Jul 29, 2026

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 4f71fac...53c46d9 on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade   Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
Python Jul 29, 2026 3:06a.m. Review ↗

Important

AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.

@codacy-production

codacy-production Bot commented Jul 29, 2026

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 1 medium

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
BestPractice 1 medium

View in Codacy

🟢 Metrics 161 complexity · 1 duplication

Metric Results
Complexity 161
Duplication 1

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8da9f61b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +96 to 100
if set(_bootstrap_passwords) != ADMIN_USERS:
raise RuntimeError(
"Bootstrap credentials for every ADMIN_USERS entry are required for an empty "
"user database; start the deployment with restart_pssm_flask.sh"
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bootstrap admins when starting with up

On a fresh installation started with the documented restart_pssm_flask.sh up subcommand or the equivalent direct Docker Compose command, ADMIN_BOOTSTRAP_CREDENTIALS is unset because only cmd_restart generates it. The newly created user database is empty, so this check raises during application import and Gunicorn repeatedly fails to start. Generate bootstrap credentials for every supported first-start path, or reject up before launching when the database is empty.

Useful? React with 👍 / 👎.

Comment on lines +181 to +185
id=f"{self.id}-thresholds",
replace_existing=True,
coalesce=True,
max_instances=self.max_instances,
**self._threshold_args,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Serialize cron and threshold log rotations

When both a scheduled period and a line/size threshold are configured, this registers the threshold invocation under a separate job ID from the cron invocation. APScheduler's max_instances=1 therefore applies independently to each job, so a long threshold rotation can overlap the cron rotation (especially after misfires or while compressing large logs). Both executions can ZIP and truncate the same active file concurrently, producing incomplete archives and discarding writes; guard the rotation method with a shared lock or combine the triggers into one serialized job.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/dev-guide/server.md (1)

390-393: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Stale "First run" description contradicts the new multi-admin bootstrap.

This step still describes a single default admin user, but the PR changes bootstrap to create every configured ADMIN_USERS entry with a distinct, transient password (see README.md's "First run" section and pssm_gremlin.py's bootstrap loop). This section wasn't updated alongside the rest of this file's environment-variable table changes.

📝 Suggested fix
 3. **Configure authentication**:
-   - On first run, a default admin user is created automatically (username: `admin`, password auto-generated and displayed by `restart_pssm_flask.sh`).
-   - Change the admin password immediately via the Profile page.
+   - On first run, every username listed in `ADMIN_USERS` is created automatically, each with a distinct, transient password printed once by `restart_pssm_flask.sh`.
+   - Change each admin password immediately via the Profile page.
    - Optionally enable self-registration with `ENABLE_REGISTER=true` and either SMTP or Resend settings.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/dev-guide/server.md` around lines 390 - 393, Update the “Configure
authentication” section to describe the multi-admin bootstrap: every configured
ADMIN_USERS entry is created with a distinct transient password, and the
generated credentials are displayed by restart_pssm_flask.sh. Remove the
outdated single default admin/password wording while retaining the instruction
to change credentials and the optional ENABLE_REGISTER guidance.
🧹 Nitpick comments (2)
server/pssm_gremlin_server/pssm_gremlin.py (1)

37-38: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Flask session secret now reuses auth.py's private token-signing key.

app.secret_key is set from auth._SECRET_KEY (imported here as a private, underscore-prefixed symbol), so the same ephemeral secret now signs both Flask's session cookie and the bearer/verification/reset tokens produced by auth._serializer. Previously these were independent secrets. itsdangerous's per-purpose salting mitigates cross-use risk, but merging two distinct security domains onto one secret is worth a second look, and importing a leading-underscore name across module boundaries is fragile.

Please confirm this reuse is intentional (and that no other Flask feature relying on secret_key, e.g. CSRF/flash, assumes an independently-rotatable secret). Consider generating app.secret_key independently (e.g. its own secrets.token_hex(32) at import time) if separation is desired, and exporting a non-underscored accessor from auth.py if the shared value is intentional.

♻️ Optional: decouple the two secrets
-app.secret_key = app.secret_key or _TOKEN_SIGNING_KEY
+app.secret_key = app.secret_key or secrets.token_hex(32)

Also applies to: 86-87

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/pssm_gremlin_server/pssm_gremlin.py` around lines 37 - 38, Decouple
Flask’s session secret from auth.py’s private _SECRET_KEY by generating an
independent app.secret_key with the module’s existing secrets support, and
remove the cross-module import alias _TOKEN_SIGNING_KEY. Keep auth._SECRET_KEY
dedicated to auth._serializer token signing and preserve the existing Flask
configuration behavior.
server/pssm_gremlin_server/templates/log_viewer.html (1)

32-40: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Incomplete ARIA tab pattern on the log toolbar.

The toolbar declares role="tablist" but its children are plain <button>s without role="tab"/aria-selected, and #logOutput lacks role="tabpanel". Buttons remain keyboard-operable natively, so this is polish rather than a blocker.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/pssm_gremlin_server/templates/log_viewer.html` around lines 32 - 40,
The log toolbar’s ARIA tab pattern is incomplete. Update the log-select buttons
in the log-toolbar to use role="tab" with appropriate aria-selected states, and
assign role="tabpanel" to `#logOutput`; keep the refreshLog button outside the tab
semantics.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Makefile`:
- Around line 84-90: Update the Gist target’s trap command to use the
POSIX-compatible exit signal 0 instead of the environment-specific EXIT,
preserving the existing cleanup command and recipe flow.
- Around line 83-90: Replace the predictable /tmp/manifest.json usage in the
Makefile target with a unique directory created via mktemp -d, ensure it
contains only manifest.json, and generate the manifest at that path. Update the
cleanup trap to remove the temporary directory and pass the generated manifest
file from that directory to gh gist edit in both the existing-file and add-file
branches.

In `@server/pssm_gremlin_server/maintenance/tasks/log_rotation.py`:
- Around line 60-72: Update the log-rotation flow around _prune_oldest_archives
so archives created during the current invocation are excluded from deletion.
Track and pass the newly rotated archive(s) as an exemption when pruning,
allowing the size cap to be temporarily exceeded rather than deleting the only
surviving copy of the log.
- Around line 88-97: Update the size-based rotation logic in the loop over
`directory.glob("*.log")` to re-evaluate `_managed_log_size(directory) >
max_size` for each log instead of relying on the upfront `rotate_for_size`
boolean. Preserve the existing line-count, periodic, and empty-file checks so
size rotation stops once the aggregate total is back under `max_size`.

In `@server/run/restart_pssm_flask.sh`:
- Around line 356-374: Update the ADMIN_USERS bootstrap loop around
_configured_admins and _admin_username to track usernames already encountered
and reject duplicates before generating or printing credentials. On a duplicate,
emit an error and exit nonzero immediately; preserve the existing whitespace
trimming, empty-entry skipping, and credential generation for unique usernames.

In `@server/tests/test_process_isolation.py`:
- Line 149: Remove the unnecessary f-string prefix from the assertion’s literal
error message in the process-isolation test, leaving the expected text and
assertion behavior unchanged.

---

Outside diff comments:
In `@docs/dev-guide/server.md`:
- Around line 390-393: Update the “Configure authentication” section to describe
the multi-admin bootstrap: every configured ADMIN_USERS entry is created with a
distinct transient password, and the generated credentials are displayed by
restart_pssm_flask.sh. Remove the outdated single default admin/password wording
while retaining the instruction to change credentials and the optional
ENABLE_REGISTER guidance.

---

Nitpick comments:
In `@server/pssm_gremlin_server/pssm_gremlin.py`:
- Around line 37-38: Decouple Flask’s session secret from auth.py’s private
_SECRET_KEY by generating an independent app.secret_key with the module’s
existing secrets support, and remove the cross-module import alias
_TOKEN_SIGNING_KEY. Keep auth._SECRET_KEY dedicated to auth._serializer token
signing and preserve the existing Flask configuration behavior.

In `@server/pssm_gremlin_server/templates/log_viewer.html`:
- Around line 32-40: The log toolbar’s ARIA tab pattern is incomplete. Update
the log-select buttons in the log-toolbar to use role="tab" with appropriate
aria-selected states, and assign role="tabpanel" to `#logOutput`; keep the
refreshLog button outside the tab semantics.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3b091c15-fdd3-4392-bfb4-1b2a3ec36bf5

📥 Commits

Reviewing files that changed from the base of the PR and between 4f71fac and f8da9f6.

📒 Files selected for processing (26)
  • CHANGELOG.md
  • Makefile
  • docs/dev-guide/server.md
  • server/.env.example
  • server/README.md
  • server/docker-compose.yml
  • server/pssm_gremlin_server/auth.py
  • server/pssm_gremlin_server/config.py
  • server/pssm_gremlin_server/maintenance/manager.py
  • server/pssm_gremlin_server/maintenance/tasks/log_rotation.py
  • server/pssm_gremlin_server/pssm_gremlin.py
  • server/pssm_gremlin_server/routes.py
  • server/pssm_gremlin_server/static/css/log-viewer.css
  • server/pssm_gremlin_server/static/js/log-viewer.js
  • server/pssm_gremlin_server/templates/log_viewer.html
  • server/pssm_gremlin_server/templates/pssm_gremlin_dashboard.html
  • server/run/restart_pssm_flask.sh
  • server/tests/conftest.py
  • server/tests/test_admin.py
  • server/tests/test_config.py
  • server/tests/test_database_backup.py
  • server/tests/test_log_rotation.py
  • server/tests/test_maintenance_manager.py
  • server/tests/test_process_isolation.py
  • server/tests/test_security_advanced.py
  • tools/generate_gist_manifest.py

Comment thread Makefile Outdated
Comment thread Makefile Outdated
Comment thread server/pssm_gremlin_server/maintenance/tasks/log_rotation.py Outdated
Comment thread server/pssm_gremlin_server/maintenance/tasks/log_rotation.py
Comment thread server/run/restart_pssm_flask.sh Outdated
Comment thread server/tests/test_process_isolation.py Outdated
)

assert result.returncode != 0
assert f"Missing required setting(s)" in result.stderr

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove extraneous f prefix (Ruff F541).

No placeholders in this f-string.

🧹 Fix
-    assert f"Missing required setting(s)" in result.stderr
+    assert "Missing required setting(s)" in result.stderr
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
assert f"Missing required setting(s)" in result.stderr
assert "Missing required setting(s)" in result.stderr
🧰 Tools
🪛 Ruff (0.16.0)

[error] 149-149: f-string without any placeholders

Remove extraneous f prefix

(F541)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/tests/test_process_isolation.py` at line 149, Remove the unnecessary
f-string prefix from the assertion’s literal error message in the
process-isolation test, leaving the expected text and assertion behavior
unchanged.

Source: Linters/SAST tools

@YaoYinYing
YaoYinYing merged commit d33b737 into main Jul 29, 2026
6 of 13 checks passed
@YaoYinYing
YaoYinYing deleted the codex/require-server-env-settings branch July 29, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant