Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .github/workflows/server-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,24 @@ jobs:
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0

- name: Setup Python
uses: actions/setup-python@2f17f13ef37957a9bc7461aa1a61b211e3c1c3ae # v7.0.0
uses: actions/setup-python@v6
with:
python-version: "3.12"

- name: Install Server and Test Dependencies
run: |
python -m pip install --upgrade pip
pip install -e "server/[test]"
pip install pytest-cov

- name: Run Server Tests (non-Docker)
run: |
python -m pytest server/tests/ -v \
python -m pytest server/tests/ -v --cov=pssm_gremlin_server --cov-report=xml \
-k "not Docker and not docker and not runner_image and not server_image and not test_server_image and not test_runner_image and not test_server_rejects and not test_server_reports and not running_gremlin"

- name: Upload coverage to Codecov
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./coverage.xml
flags: server
if: always()
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,4 @@ src/REvoDesign/UI/Ui_REvoDesign.py
users.sqlite3
testerror.txt
/server/server_state
/server/pssm_gremlin_server.egg-info
50 changes: 50 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,56 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Admin password**: the bootstrap-generated admin password was logged to
gunicorn stderr (inaccessible to the operator). The restart script now
generates and displays it on the console before bringing services up.
- **Host header spoofing in email links**: `_public_base_url()` now always
uses the configured `SERVER_BASE_URL` for verification / password-reset
links, never the request `Host` header (which an attacker can spoof to
place valid tokens in links pointing to an attacker-controlled domain).
- **Role demotion doesn't clear `is_admin`**: updating a user role from
`admin` to `user`/`guest` now also sets `is_admin = False`, preventing
privilege-leak bugs where `require_admin()` checks `is_admin` instead of
`role`.
- **Cancelled tasks resurrected by late worker writes**: cancelled tasks are
now treated as terminal — the DB update guard prevents late `running`/
`finished` status writes from overwriting a `cancelled` task.
- **Legacy NULL columns not backfilled**: `_ensure_columns()` now backfills
`deleted`, `registration_status`, `user_status`, and `role` for rows that
predate those columns.
- **Admin digest silently drops users on email failure**: `_send_email()`
returns `False` on failure (doesn't raise), so the try/except never caught
transient errors. The digest loop now checks return values and unmarks users
as notified when all recipient deliveries fail.
- **Failed-task archives recreated after deletion**: the three exception
handlers in `run_gremlin_task` now guard `_pack_failed_results_archive`
with `_task_is_deleted()` so deleted tasks don't get artifacts recreated.
- **Celery submission failure leaves orphaned pending task**:
`apply_async` failures now mark the task as failed and return a 503,
preventing the pending task from consuming the user's quota forever.
- **Account enumeration via resend-verification**: the unauthenticated
endpoint now returns the same generic response for unknown, deleted, banned,
and already-verified accounts.
- **Batch-enable doesn't verify email**: batch-enable (`enable` action)
now also sets `email_verified = True` so admin-approved users can
immediately use features that require a verified email.
- **Rate limiter unbounded growth**: the in-memory rate-limit state dict
now periodically prunes expired IP entries, preventing unbounded memory
growth across the process lifetime.
- **Pydantic email validators crash on non-string input**:
`normalize_email()` now raises a clean `ValueError` instead of
`AttributeError`.
- **Test DB path targets wrong database**: `_inject_admin_password` now
writes to the user DB (`users.sqlite3`) instead of the task DB
(`pssm_gremlin_server.sqlite3`).
- **Missing columns in SQLAlchemy metadata**: `admin_notified`,
`verification_resend_count`, and `verification_resend_at` are now
declared in the `_users_table` Table definition.
- **Server test env now self-contained**: the server test suite no longer
imports from the root `tests/conftest.py`. The server conftest defines
`REPO_DIR` and `has_docker_daemon` locally, enabling a lightweight
`REvoDesignServerDev` conda env.
- **CI workflow fix**: replaced unresolvable `actions/setup-python` pinned
SHA with `@v6` in `server-test.yml`.


- **Chrome file picker not opening**: native ``<input type="file">`` click
fails to open the file dialog in Chrome (works in Safari). Added
drag-and-drop file upload as a browser-agnostic workaround — drop a
Expand Down
3 changes: 3 additions & 0 deletions server/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,9 @@ RUNNER_GROUP=revodesign_appgroup
# ============================================================================
# Public HTTP port.
PORT=8080
# Number of days to retain task result files before automatic cleanup.
# Default is 30 days. Set to 0 to disable auto-cleanup.
# RESULT_RETENTION_DAYS=30
# Scope task dashboard to the logged-in user (false, default) or show all
# tasks to everyone (true, for public-facing deployments without auth).
PUBLIC_DASHBOARD=false
Expand Down
42 changes: 30 additions & 12 deletions server/pssm_gremlin_server/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,8 @@ def _env_int(var: str, default: int) -> int:
sa.Column("deleted", sa.Boolean, nullable=False, default=False),
sa.Column("role", sa.String(32), nullable=False, default="user"),
sa.Column("admin_notified", sa.Boolean, nullable=False, default=False),
sa.Column("verification_resend_count", sa.Integer, nullable=False, default=0),
sa.Column("verification_resend_at", sa.Float, nullable=True),
sa.Column("registration_ip", sa.String(45), nullable=True),
sa.Column("registration_country", sa.String(8), nullable=True),
)
Expand Down Expand Up @@ -170,6 +172,24 @@ def _ensure_columns(conn) -> None:
conn.exec_driver_sql(
"UPDATE users SET admin_notified = 1 WHERE is_admin = 0"
)
# Backfill legacy rows that predate the column — ensures no NULLs
# slip through for columns added after initial schema creation.
if "deleted" not in existing:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Backfill NULLs even when columns already exist

On databases that already ran the previous migration, these columns are already present in existing but legacy rows can still contain NULL, so this new backfill is skipped exactly where it is needed. Those rows remain broken after upgrade: for example list_users() filters deleted == False, excluding NULL rows, and UserResponse expects non-null registration_status/user_status/role, so admin user listing can still hide users or fail validation.

Useful? React with 👍 / 👎.

conn.exec_driver_sql("UPDATE users SET deleted = 0 WHERE deleted IS NULL")
if "registration_status" not in existing:
conn.exec_driver_sql(
"UPDATE users SET registration_status = 'approved' "
"WHERE registration_status IS NULL"
)
if "user_status" not in existing:
conn.exec_driver_sql(
"UPDATE users SET user_status = 'active' WHERE user_status IS NULL"
)
if "role" not in existing:
conn.exec_driver_sql(
"UPDATE users SET role = CASE WHEN is_admin THEN 'admin' ELSE 'user' END "
"WHERE role IS NULL"
)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

# -- write helpers -------------------------------------------------------

Expand Down Expand Up @@ -646,15 +666,10 @@ def validate_captcha(token: str, answer: str) -> bool:
def _public_base_url() -> str:
"""Return the public-facing base URL for email links.

Uses the current request's ``Host`` header so email links point to the
same domain the user is accessing. Falls back to ``SERVER_BASE_URL``
env var, then ``http://localhost:8080`` (dev).
Always uses the configured ``SERVER_BASE_URL`` — never the request
``Host`` header, which an attacker can spoof to place valid tokens
into links pointing at an attacker-controlled domain.
"""
# ponytail: request.host_url already has scheme+host from the Host header
try:
return request.host_url.rstrip("/")
except RuntimeError:
pass # outside request context (tests, scripts)
return _env_str("SERVER_BASE_URL", "http://localhost:8080").rstrip("/")


Expand Down Expand Up @@ -902,18 +917,21 @@ def send_admin_digest() -> bool:
f"text-decoration:none;border-radius:6px;font-weight:600;\">"
f"Review Registrations</a></p>"
)
any_sent = False
try:
for email in recipients:
_send_email(
if _send_email(
to=email,
subject=f"{len(new_users)} new registration(s) — REvoDesign GREMLIN",
text=text,
html=_email_html(html_body),
)
):
any_sent = True
except Exception:
any_sent = False
if not any_sent:
db.unmark_users_notified(user_ids)
raise
return True
return any_sent
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved


def validate_reset_token(token: str) -> int | None:
Expand Down
8 changes: 5 additions & 3 deletions server/pssm_gremlin_server/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ class TaskDatabase:
"""Minimal SQLite-based task tracker for GREMLIN jobs."""

DELETED_STATUSES = {"deleted:finshed", "deleted:cancel"}
TERMINAL_STATUSES = {"deleted:finshed", "deleted:cancel", "cancelled"}

VALID_STATUSES = {
"pending",
Expand Down Expand Up @@ -160,11 +161,12 @@ def update_task(self, md5sum: str, **fields) -> None:
if status:
self._ensure_status(status)
stmt = update(self.tasks_table).where(self.tasks_table.c.md5sum == md5sum).values(**fields)
# Deleted tasks are terminal in the runtime state machine.
# Terminal tasks (deleted / cancelled) must stay terminal.
# Ignore late worker writes (running/packing/finished/run_stage, etc.)
# that would otherwise resurrect tasks after user deletion.
# that would otherwise resurrect tasks after user deletion or
# cancellation.
if status is None or (not self._is_deleted_status(status)):
stmt = stmt.where(self.tasks_table.c.status.notin_(tuple(self.DELETED_STATUSES)))
stmt = stmt.where(self.tasks_table.c.status.notin_(tuple(self.TERMINAL_STATUSES)))
with self.engine.begin() as conn:
conn.execute(stmt)

Expand Down
57 changes: 30 additions & 27 deletions server/pssm_gremlin_server/pssm_gremlin.py
Original file line number Diff line number Diff line change
Expand Up @@ -897,40 +897,43 @@ def _on_stage_change(stage: str) -> None:
except docker.errors.ContainerError as exc:
finish_time = time.time()
error_message = f"docker: {exc}"
_pack_failed_results_archive(task, error_message)
task_store.update_task(
md5sum,
status="failed",
finished_at=finish_time,
walltime=finish_time - start_time,
error=error_message,
run_stage=stage_state["current"],
)
if not _task_is_deleted(md5sum):
_pack_failed_results_archive(task, error_message)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat cancelled tasks as terminal before repacking failures

When a user cancels a running task, cancel_task() deletes the artifacts and records status='cancelled', but a terminated Docker run commonly reaches this exception path afterward. Since this new guard only checks deleted statuses, the worker still calls _pack_failed_results_archive() for a cancelled task, recreating the result directory/zip that cancellation just removed while the DB guard prevents the status from changing away from cancelled.

Useful? React with 👍 / 👎.

task_store.update_task(
md5sum,
status="failed",
finished_at=finish_time,
walltime=finish_time - start_time,
error=error_message,
run_stage=stage_state["current"],
)
except docker.errors.DockerException as exc:
finish_time = time.time()
error_message = f"docker: {exc}"
_pack_failed_results_archive(task, error_message)
task_store.update_task(
md5sum,
status="failed",
finished_at=finish_time,
walltime=finish_time - start_time,
error=error_message,
run_stage=stage_state["current"],
)
if not _task_is_deleted(md5sum):
_pack_failed_results_archive(task, error_message)
task_store.update_task(
md5sum,
status="failed",
finished_at=finish_time,
walltime=finish_time - start_time,
error=error_message,
run_stage=stage_state["current"],
)
logging.error("Docker daemon unavailable for GREMLIN task %s: %s", md5sum, exc)
except Exception as exc: # pylint: disable=broad-except
finish_time = time.time()
error_message = str(exc)
_pack_failed_results_archive(task, error_message)
task_store.update_task(
md5sum,
status="failed",
finished_at=finish_time,
walltime=finish_time - start_time,
error=error_message,
run_stage=stage_state["current"],
)
if not _task_is_deleted(md5sum):
_pack_failed_results_archive(task, error_message)
task_store.update_task(
md5sum,
status="failed",
finished_at=finish_time,
walltime=finish_time - start_time,
error=error_message,
run_stage=stage_state["current"],
)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
logging.exception("Unexpected failure while running GREMLIN task %s", md5sum)


Expand Down
14 changes: 14 additions & 0 deletions server/pssm_gremlin_server/ratelimit.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,29 @@ def login():
"""
state: dict[str, list[float]] = {}
_lock = threading.Lock()
_last_cleanup: float = 0.0

def _prune_expired(now: float, cutoff: float) -> None:
"""Drop per-IP entries whose most recent timestamp is expired."""
empty = [ip for ip, ts in state.items() if not ts or ts[-1] <= cutoff]
for ip in empty:
del state[ip]

def decorator(f: Callable) -> Callable:
@wraps(f)
def decorated(*args: Any, **kwargs: Any) -> Any:
nonlocal _last_cleanup
ip = request.remote_addr or "unknown"
now = time.time()
cutoff = now - window_seconds

with _lock:
# Periodic cleanup of expired entries — prevents unbounded
# growth of the state dict across process lifetime.
if now - _last_cleanup > max(window_seconds, 600):
_prune_expired(now, cutoff)
_last_cleanup = now

timestamps = [t for t in state.get(ip, []) if t > cutoff]
if len(timestamps) >= max_requests:
retry_after = int(timestamps[0] - cutoff)
Expand Down
34 changes: 23 additions & 11 deletions server/pssm_gremlin_server/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,16 @@ def upload_file():
error=None,
)

async_result = run_gremlin_task.apply_async(args=[md5sum])
try:
async_result = run_gremlin_task.apply_async(args=[md5sum])
except Exception:
logging.exception("Failed to submit GREMLIN task %s to Celery", md5sum)
task_store.update_task(
md5sum,
status="failed",
error="Task queue unavailable — please try again later",
)
return jsonify({"error": "Task queue unavailable — please try again"}), 503
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
task_store.update_task(md5sum, celery_task_id=async_result.id)

return redirect(f"/PSSM_GREMLIN/api/running/{md5sum}", code=302)
Expand Down Expand Up @@ -841,16 +850,19 @@ def auth_resend_verification():

db = _get_user_db()
user = db.get_user_by_email(email)
if user is None:
return jsonify({"error": "No account found with this email address"}), 404

if user.get("deleted"):
return jsonify({"error": "Account has been deleted"}), 403
if user.get("user_status") == "banned":
return jsonify({"error": "Account has been suspended"}), 403

# Return a generic response for all non-actionable cases to prevent
# account enumeration (unknown, deleted, banned, already verified).
_generic = (
jsonify({"message": "If that email is registered and unverified, a new verification email has been sent."}),
200,
)
if user is None:
return _generic
if user.get("deleted") or user.get("user_status") == "banned":
return _generic
if user.get("email_verified"):
return jsonify({"message": "This email is already verified. You can log in."}), 200
return _generic

# Per-email backoff: 10×n minutes since last resend
count = user.get("verification_resend_count") or 0
Expand Down Expand Up @@ -1140,8 +1152,7 @@ def admin_manage_user(user_id):
if is_self:
return jsonify({"error": "Administrators cannot change their own role"}), 400
update_fields["role"] = req.role
if req.role == "admin":
update_fields["is_admin"] = True
update_fields["is_admin"] = req.role == "admin"

# Only set approved_by / approved_at when the admin is actually approving.
new_reg = update_fields.get("registration_status")
Expand Down Expand Up @@ -1188,6 +1199,7 @@ def admin_batch_users():
updates = {
"user_status": "active",
"registration_status": "approved",
"email_verified": True,
"deleted": False,
"approved_by": admin_id,
"approved_at": now,
Expand Down
2 changes: 2 additions & 0 deletions server/pssm_gremlin_server/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ def normalize_email(email: str) -> str:
``user+tag@domain.com`` → ``user@domain.com`` — prevents one person
from creating multiple accounts via plus-aliased addresses.
"""
if not isinstance(email, str):
raise ValueError("email must be a string")
email = email.strip().lower()
local, at, domain = email.partition("@")
local = local.split("+")[0]
Expand Down
1 change: 1 addition & 0 deletions server/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ resend = [
test = [
"pytest>=8",
"pytest-mock>=3.14",
"pytest-cov>=6",
]

[tool.setuptools.packages.find]
Expand Down
Loading
Loading