Skip to content

feat: add system task runner - #5680

Merged
Calcium-Ion merged 4 commits into
mainfrom
feat/system-tasks
Jun 24, 2026
Merged

feat: add system task runner#5680
Calcium-Ion merged 4 commits into
mainfrom
feat/system-tasks

Conversation

@Calcium-Ion

@Calcium-Ion Calcium-Ion commented Jun 23, 2026

Copy link
Copy Markdown
Member

⚠️ 提交说明 / PR Notice

Important

  • 请提供人工撰写的简洁摘要,避免直接粘贴未经整理的 AI 输出。

📝 变更描述 / Description

  • 新增数据库驱动的 system_tasks 后台任务框架,用于记录任务运行状态、进度、执行实例与结果。
  • 新增 system_task_locks 按任务类型做数据库租约锁,保证多 master 下同类型任务同一时刻只由一个实例执行;节点失联后由其他 runner 在租约过期后标记旧 run 失败。
  • 将日志清理、批量渠道测试、上游模型更新、Midjourney 轮询、异步任务轮询接入统一 runner,并移除旧的常驻轮询 goroutine。
  • 新增默认前端“系统信息”页面与系统任务面板,支持手动刷新、自动刷新提示、进行中任务与历史任务分区展示。
  • 优化 runner 空闲查询:批量查询每类 pending/latest 任务,移除历史任务自动删除逻辑,保留任务历史由列表 limit 控制展示范围。

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
  • Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

已通过本地验证:

go test ./model/... ./service/...
go build ./...
bun run typecheck
bun run i18n:sync
git diff --check

Summary by CodeRabbit

  • New Features

    • Added an authenticated System Info page with a System Tasks panel showing active tasks and recent history (executor, progress, and error details) with live auto-refresh.
    • Introduced a system task listing API (supports limit) to view recent task runs.
  • Refactor

    • Moved channel tests, upstream model detection/apply, Midjourney polling, and async polling onto a unified scheduled system-task framework with progress reporting and cancellation-aware execution.
  • Bug Fixes

    • Manual batch actions now return 409 Conflict when the related system task is already running or queued.

@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 67c32d51-1fbb-417d-85c8-9ea99c0a0dac

📥 Commits

Reviewing files that changed from the base of the PR and between d36d404 and eae66d5.

📒 Files selected for processing (4)
  • common/constants.go
  • common/init.go
  • common/node_identity.go
  • web/default/src/hooks/use-sidebar-data.ts

Walkthrough

Adds a database-backed system task framework, converts several batch jobs to single-pass task handlers, and introduces a System Info page for listing system tasks. It also changes node identity startup to record whether the name came from the environment or hostname.

Changes

System Task Framework & System Info UI

Layer / File(s) Summary
Task schema and migrations
model/system_task.go, model/main.go, model/midjourney.go, model/task.go
Updates system task models, adds task-type constants, introduces a separate lock table, changes task creation and response shapes, and migrates the new table in both migration paths. Adds unfinished-task existence helpers for Midjourney and sync tasks.
Claim, lease, finish, and query helpers
model/system_task.go, model/system_task_test.go
Reworks task claiming and completion around the lock table, adds lease renewal and expiry helpers, and expands task lookup queries for pending and latest tasks. Tests cover lifecycle creation, duplicate active runs, claim contention, expired leases, stale-lock expiration, query helpers, renewal ownership, executor retention, and lock-owner enforcement.
System task runner and scheduler
service/system_task.go, service/system_task_test.go
Refactors the runner to use idle and wakeup loops, introduces a handler registry, scheduled-task creation, claim-pass dispatch, lease heartbeats, on-demand enqueueing, and throttled progress reporting. Updates log-cleanup progress persistence to the new state-update signature and adds tests for scheduler deduplication, disabled handlers, claim dispatch, and enqueue behavior.
Single-pass background job refactors
controller/channel-test.go, controller/channel_upstream_update.go, controller/midjourney.go, service/task_polling.go, controller/task.go, controller/channel_test_internal_test.go, controller/channel_upstream_update_test.go
Converts channel testing, upstream model detection, Midjourney polling, and async task polling from inline or looping execution into context-aware single-pass functions returning summaries. Manual batch endpoints now enqueue system tasks and reject duplicate active runs with HTTP 409.
Controller registration, API endpoints, and startup wiring
controller/system_task_handlers.go, controller/system_task.go, router/api-router.go, main.go, model/task_cas_test.go, service/task_billing_test.go
Registers four scheduled handlers, adds the system-task list endpoint, updates startup to register handlers before starting the runner, and includes the lock table in test migrations and cleanup.
System Info page, task list panel, and route wiring
web/default/src/features/system-info/..., web/default/src/features/system-settings/..., web/default/src/hooks/..., web/default/src/routes/_authenticated/system-info/index.tsx, web/default/src/routeTree.gen.ts, web/default/src/components/layout/types.ts
Adds the System Info page, task list panel, route guard, sidebar entry, route tree updates, API helper, response type, and sidebar role filtering support. The panel lists active and historical system tasks and auto-refreshes while tasks are pending or running.
Translation updates
web/default/src/i18n/locales/*.json
Adds translations across all locales for task states, system-task labels, empty states, task-history labels, auto-refresh text, system info labels, and upstream detection progress messaging.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Controller
  participant EnqueueSystemTask
  participant SystemTaskRunner
  participant DB

  Client->>Controller: trigger batch test or upstream detection
  Controller->>EnqueueSystemTask: create pending system task
  EnqueueSystemTask->>DB: CreateSystemTask
  EnqueueSystemTask->>SystemTaskRunner: wake runner
  SystemTaskRunner->>DB: claim task and lease lock
  SystemTaskRunner->>DB: run handler once
  SystemTaskRunner->>DB: FinishSystemTask
  SystemTaskRunner->>DB: ReleaseSystemTaskLock

  Client->>Controller: GET /api/system-task/list
  Controller->>DB: ListSystemTasks
  DB-->>Controller: task rows
  Controller-->>Client: system task list
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • QuantumNous/new-api#1244: Both PRs touch controller/channel-test.go; the retrieved PR adds a guard in testChannel, while this PR refactors the same path into context-aware system-task execution.
  • QuantumNous/new-api#2002: Both PRs modify channel-test scheduling and execution flow in controller/channel-test.go, so they overlap in the same automatic test path.
  • QuantumNous/new-api#2985: Both PRs affect controller/midjourney.go task update handling, including the persistence/update flow around Midjourney polling.

Suggested reviewers

  • seefs001

Poem

🐇 I hop through tasks with a tidy little grin,
Locks in one table, and the races grow thin.
System Info glows with a queue row or two,
I nibble on progress and refresh the view.
One pass, one heartbeat, one hop at a time—
This bunny approves of the orderly rhyme.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 24.75% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately captures the main change: introducing a system task runner.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/system-tasks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
controller/channel_upstream_update.go (1)

580-626: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Stop the per-channel scan as soon as the task context is canceled.

Line 558 checks cancellation only before fetching a batch. Once a batch is loaded, lease loss can still run every channel in that batch and sleep between checks.

Make the inner loop cancellation-aware
+scanLoop:
 	for {
 		if ctx != nil && ctx.Err() != nil {
 			break
 		}
...
 		for _, channel := range channels {
+			if ctx != nil && ctx.Err() != nil {
+				break scanLoop
+			}
 			if channel == nil {
 				continue
 			}
...
 			if common.RequestInterval > 0 {
-				time.Sleep(common.RequestInterval)
+				if ctx != nil {
+					select {
+					case <-ctx.Done():
+						break scanLoop
+					case <-time.After(common.RequestInterval):
+					}
+				} else {
+					time.Sleep(common.RequestInterval)
+				}
 			}
 		}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@controller/channel_upstream_update.go` around lines 580 - 626, The inner loop
iterating over channels (starting with `for _, channel := range channels`) does
not check for context cancellation, meaning once a batch is loaded, all channels
in that batch will be processed even if the task context is canceled. Add a
context cancellation check at the beginning of the channel iteration loop (after
the nil check for channel) to immediately break out of the loop when the context
is canceled, preventing unnecessary processing of remaining channels in the
batch.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@controller/channel_upstream_update.go`:
- Line 949: The EnqueueSystemTask call in the channel_upstream_update.go file
dedupes tasks only by type, not payload, which means it can return an
already-active scheduled model_update task with Manual=false instead of creating
a new one with Manual=true. This violates the manual detect-all contract since
the existing task may skip forced checks and auto-apply changes. Fix this by
either checking the returned task and returning a conflict error if its Manual
field is false, or by implementing payload-aware task queueing/deduplication
logic that ensures manual detect-all operations with Manual=true either run
independently or clearly report failure when a conflicting non-manual task is
active. Reference the modelUpdateTaskPayload struct and its Manual field when
implementing this check.

In `@controller/channel-test.go`:
- Around line 906-927: The context cancellation is only checked between loop
iterations in performChannelTests, but the actual channel testing operations
ignore the cancellation signal. Pass the context ctx to the testChannel function
call so it can respect cancellation during provider requests, and ensure any
RequestInterval sleep or wait operations also check for context cancellation
before proceeding. Additionally, verify that the notify operation handling
(referenced in lines 966-1000) also respects context cancellation to prevent
"completed" notifications after a partially canceled run.

In `@controller/midjourney.go`:
- Around line 111-118: The context timeout in the Midjourney HTTP client call is
incorrectly using context.Background() as the parent context, which breaks
cancellation propagation from the task runner. Replace the
context.WithTimeout(context.Background(), timeout) call with
context.WithTimeout(ctx, timeout) where ctx should be the runner context derived
from the incoming request (typically obtained from req.Context() or passed as a
function parameter). This ensures that when the task is cancelled, the in-flight
HTTP request will be cancelled immediately instead of waiting for the 15-second
timeout to expire.
- Around line 149-152: The code retrieves a task from the taskM map using
responseItem.MjId without checking if the result is nil, and then immediately
dereferences task.SubmitTime on the following line, which will panic if the MjId
is unknown. Add a nil guard check immediately after the task assignment from
taskM[responseItem.MjId] to verify that task is not nil before attempting to
access task.SubmitTime or any other task fields. If task is nil, skip processing
that responseItem or handle it appropriately.

In `@model/system_task.go`:
- Around line 306-327: The lock validation in the ensureSystemTaskLockHeld call
at the beginning of UpdateSystemTaskState and the subsequent database write are
not atomic, allowing a lease to expire between the check and the write. Refactor
UpdateSystemTaskState to combine the lock ownership validation and state update
into a single atomic operation, either by wrapping both in a transaction with
lock validation rechecked during the update or by incorporating the lock check
directly into the WHERE clause in a way that prevents race conditions. Apply
this same atomic pattern to FinishSystemTask as well to ensure consistency
across all system task state management functions.
- Line 29: The ID field in the SystemTask model in system_task.go contains an
explicit AUTO_INCREMENT directive in the gorm tag which is database-dialect
specific and reduces portability. Remove the `;AUTO_INCREMENT` portion from the
gorm tag on the ID field, leaving only `gorm:"primary_key"` to allow GORM to
handle primary key generation automatically in a cross-dialect compatible
manner.

In `@service/system_task_test.go`:
- Around line 112-115: The onRun callback at line 112-115 uses require.NoError
inside an async worker goroutine, which causes test timeouts when the assertion
fails because only the worker goroutine terminates while the main test goroutine
remains blocked on the ran channel. Define a result type that carries both the
task type and any error from FinishSystemTask, modify the onRun callback to
capture the error from FinishSystemTask instead of asserting immediately, send
the result (task type and error) through the ran channel, and then in the main
test goroutine after receiving from ran, assert on the error value to catch
actual failures. Apply this same pattern to all other onRun callbacks mentioned
in the file including those at lines 145-157.

In `@service/system_task.go`:
- Around line 18-26: The systemTaskSchedulerInterval constant is set to 30
seconds, which throttles the scheduler evaluation and prevents handlers with
shorter configured intervals (such as the 15-second Midjourney and async polling
handlers) from running at their intended cadence. Lower the
systemTaskSchedulerInterval value to match or be shorter than the minimum
handler interval (15 seconds) so that all configured handlers can execute within
their specified intervals, or alternatively derive the scheduler wake cadence
dynamically from the shortest enabled handler interval defined in
controller/system_task_handlers.go. Also review the related code mentioned in
lines 132-146 to ensure consistency in the scheduler timing logic.
- Around line 286-288: The error returned from the CreateSystemTask call in the
block starting with the model.CreateSystemTask function is being silently
ignored when it fails, which prevents visibility into why scheduled tasks are
not being created. Add logging before the continue statement to capture and log
the actual error information along with context about which scheduled task
failed (include details like scheduled.Type() to identify which background job
failed). This will help with debugging when recurring background jobs disappear.

In `@service/task_polling.go`:
- Line 164: The DispatchPlatformUpdate function call does not receive the
polling context, causing it to use context.Background() internally for
Suno/video polling operations, which ignores lease cancellation. Pass the
polling context as a parameter to the DispatchPlatformUpdate function call, then
update the DispatchPlatformUpdate function signature to accept this context
parameter and use it instead of context.Background() for any platform polling or
dispatch operations that need to respect cancellation.

In `@web/default/src/features/system-info/components/system-tasks-panel.tsx`:
- Around line 196-199: The queryFn for listSystemTasks currently silently
converts unsuccessful responses to an empty array, causing failures to appear as
"No system tasks yet" instead of displaying an error state. Throw an error in
the queryFn when res.success is false or res.data is not an array, then update
the component's render logic to add an isError branch that displays a friendly
error message with a retry button before the empty-state fallback. Apply this
same error handling pattern to the other listSystemTasks queryFn occurrence
(around lines 269-288) and use handleServerError for consistent error handling
across the component.

In `@web/default/src/i18n/locales/ja.json`:
- Around line 4132-4137: The translation for the "Task Logs" key is incorrectly
set to "タスク履歴" which is identical to the "Task History" translation, creating
ambiguity for users. Change the value of the "Task Logs" key in the ja.json file
from "タスク履歴" to "タスクログ" to match the correct translation used for the "Task
logs" key on the preceding line, ensuring users can properly distinguish between
logs and history.

---

Outside diff comments:
In `@controller/channel_upstream_update.go`:
- Around line 580-626: The inner loop iterating over channels (starting with
`for _, channel := range channels`) does not check for context cancellation,
meaning once a batch is loaded, all channels in that batch will be processed
even if the task context is canceled. Add a context cancellation check at the
beginning of the channel iteration loop (after the nil check for channel) to
immediately break out of the loop when the context is canceled, preventing
unnecessary processing of remaining channels in the batch.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: faa48ecf-2047-4724-a0e9-9cff04211f7c

📥 Commits

Reviewing files that changed from the base of the PR and between 9fc9c8f and 61f4fdf.

📒 Files selected for processing (34)
  • controller/channel-test.go
  • controller/channel_upstream_update.go
  • controller/midjourney.go
  • controller/system_task.go
  • controller/system_task_handlers.go
  • controller/task.go
  • main.go
  • model/main.go
  • model/midjourney.go
  • model/system_task.go
  • model/system_task_test.go
  • model/task.go
  • model/task_cas_test.go
  • router/api-router.go
  • service/system_task.go
  • service/system_task_test.go
  • service/task_billing_test.go
  • service/task_polling.go
  • web/default/src/components/layout/types.ts
  • web/default/src/features/channels/hooks/use-channel-upstream-updates.ts
  • web/default/src/features/system-info/components/system-tasks-panel.tsx
  • web/default/src/features/system-info/index.tsx
  • web/default/src/features/system-settings/api.ts
  • web/default/src/features/system-settings/types.ts
  • web/default/src/hooks/use-sidebar-data.ts
  • web/default/src/hooks/use-sidebar-view.ts
  • web/default/src/i18n/locales/en.json
  • web/default/src/i18n/locales/fr.json
  • web/default/src/i18n/locales/ja.json
  • web/default/src/i18n/locales/ru.json
  • web/default/src/i18n/locales/vi.json
  • web/default/src/i18n/locales/zh.json
  • web/default/src/routeTree.gen.ts
  • web/default/src/routes/_authenticated/system-info/index.tsx
💤 Files with no reviewable changes (1)
  • controller/task.go

Comment thread controller/channel_upstream_update.go Outdated
Comment thread controller/channel-test.go
Comment thread controller/midjourney.go Outdated
Comment thread controller/midjourney.go
Comment thread model/system_task.go Outdated
Comment thread service/system_task.go Outdated
Comment thread service/system_task.go
Comment thread service/task_polling.go Outdated
Comment thread web/default/src/i18n/locales/ja.json

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
model/system_task.go (1)

383-405: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Make task finalization and lease release atomic.

Line 392 commits the terminal task state before Line 405 deletes the lease row. If ReleaseSystemTaskLock fails after the update succeeds, the task is finished and active_key is cleared, but the stale system_task_locks row still prevents the next same-type claim until the lease expires. Wrap the update and lock delete in one transaction so a transient delete failure cannot stall the runner for that task type.

Suggested direction
 func FinishSystemTask(taskID string, lockedBy string, status SystemTaskStatus, resultPayload any, errorMessage string) error {
 	resultText, err := marshalSystemTaskJSON(resultPayload)
 	if err != nil {
 		return err
 	}
 	now := common.GetTimestamp()
-	result := DB.Model(&SystemTask{}).
-		Where("task_id = ? AND status = ? AND locked_by = ?", taskID, SystemTaskStatusRunning, lockedBy).
-		Where("EXISTS (SELECT 1 FROM system_task_locks WHERE system_task_locks.task_id = system_tasks.task_id AND system_task_locks.locked_by = ? AND system_task_locks.locked_until >= ?)", lockedBy, now).
-		Updates(map[string]any{
-			"status":     status,
-			"active_key": nil,
-			"result":     resultText,
-			"error":      errorMessage,
-			"updated_at": now,
-		})
-	if result.Error != nil {
-		return result.Error
-	}
-	if result.RowsAffected == 0 {
-		return ErrSystemTaskLockLost
-	}
-	return ReleaseSystemTaskLock(taskID, lockedBy)
+	return DB.Transaction(func(tx *gorm.DB) error {
+		result := tx.Model(&SystemTask{}).
+			Where("task_id = ? AND status = ? AND locked_by = ?", taskID, SystemTaskStatusRunning, lockedBy).
+			Where("EXISTS (SELECT 1 FROM system_task_locks WHERE system_task_locks.task_id = system_tasks.task_id AND system_task_locks.locked_by = ? AND system_task_locks.locked_until >= ?)", lockedBy, now).
+			Updates(map[string]any{
+				"status":     status,
+				"active_key": nil,
+				"result":     resultText,
+				"error":      errorMessage,
+				"updated_at": now,
+			})
+		if result.Error != nil {
+			return result.Error
+		}
+		if result.RowsAffected == 0 {
+			return ErrSystemTaskLockLost
+		}
+		return tx.Where("task_id = ? AND locked_by = ?", taskID, lockedBy).Delete(&SystemTaskLock{}).Error
+	})
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@model/system_task.go` around lines 383 - 405, Make FinishSystemTask atomic by
wrapping the terminal status update and ReleaseSystemTaskLock in a single
transaction: the current flow in FinishSystemTask updates the SystemTask row
first and only then deletes the lease row, so a failure in ReleaseSystemTaskLock
can leave a stale system_task_locks record behind. Use the existing
FinishSystemTask and ReleaseSystemTaskLock symbols to locate the logic, and
ensure both the DB.Model(&SystemTask{}).Updates call and the lock deletion
either commit together or roll back together so task finalization cannot succeed
without releasing the lease.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@model/system_task.go`:
- Around line 383-405: Make FinishSystemTask atomic by wrapping the terminal
status update and ReleaseSystemTaskLock in a single transaction: the current
flow in FinishSystemTask updates the SystemTask row first and only then deletes
the lease row, so a failure in ReleaseSystemTaskLock can leave a stale
system_task_locks record behind. Use the existing FinishSystemTask and
ReleaseSystemTaskLock symbols to locate the logic, and ensure both the
DB.Model(&SystemTask{}).Updates call and the lock deletion either commit
together or roll back together so task finalization cannot succeed without
releasing the lease.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9fe0f9d4-a8ea-41f0-bdbd-09ce4acb8ef3

📥 Commits

Reviewing files that changed from the base of the PR and between cea3530 and d36d404.

📒 Files selected for processing (4)
  • model/system_task.go
  • model/system_task_test.go
  • service/system_task.go
  • service/system_task_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • service/system_task.go

@Calcium-Ion
Calcium-Ion merged commit 5377192 into main Jun 24, 2026
1 check was pending
@Calcium-Ion
Calcium-Ion deleted the feat/system-tasks branch June 24, 2026 10:42
shudonglin added a commit to rayward-external/new-api that referenced this pull request Jun 27, 2026
* chore: avoid duplicate shadcn skill exposure

* fix: support SMTP STARTTLS mode and NTLM auth (QuantumNous#5426)

* fix: support SMTP STARTTLS mode and NTLM auth

Add explicit SMTP STARTTLS configuration for 587-style connections and keep SSL/TLS as the implicit TLS mode.

Prefer PLAIN when advertised, keep LOGIN compatibility, and add NTLM as a fallback for Exchange SMTP servers that require it after STARTTLS.

* fix: respect explicit SMTP encryption mode

* fix: preserve SMTP TLS compatibility

* fix: preserve SMTP PLAIN auth TLS guard

* chore(deps): bump github.com/ClickHouse/ch-go from 0.58.2 to 0.65.0 (QuantumNous#5664)

Bumps [github.com/ClickHouse/ch-go](https://github.com/ClickHouse/ch-go) from 0.58.2 to 0.65.0.
- [Release notes](https://github.com/ClickHouse/ch-go/releases)
- [Commits](ClickHouse/ch-go@v0.58.2...v0.65.0)

---
updated-dependencies:
- dependency-name: github.com/ClickHouse/ch-go
  dependency-version: 0.65.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: update agent skills and project config

- add vercel-react-best-practices skill (SKILL.md + full-guide.md)
- slim CLAUDE.md to import shared AGENTS.md conventions
- promote go-ntlmssp to a direct dependency in go.mod

* fix: date-fns-tz classic theme build error (QuantumNous#5676)

* chore(deps): update clickhouse-go and orb dependencies

* feat: add system task runner (QuantumNous#5680)

* feat: add system instance info panel (QuantumNous#5716)

* feat: add system instance reporting

* feat: show system instance resources

* fix: update translations for heartbeat messages in Russian and Vietnamese

* fix(web): replace default markdown renderer and expand syntax support (QuantumNous#5689)

* fix(markdown): render default markdown with marked

- switch default frontend markdown rendering from react-markdown/remark-gfm to marked to avoid old WebKit parse failures from lookbehind regex literals
- sanitize marked HTML output with DOMPurify and preserve external link target and rel behavior
- remove default direct dependencies on react-markdown, remark-gfm, and rehype-raw while leaving classic unchanged

* fix(markdown): expand default markdown rendering support

- render default markdown with marked extensions for KaTeX formulas, page breaks, and common emoji shortcodes.
- sanitize KaTeX output with an explicit DOMPurify allowlist while preserving external link behavior.
- avoid overriding marked text rendering so lists and inline parsing keep their internal parser context.

* fix(markdown): render diagram code blocks in default UI

- add sanitized SVG rendering for flow and sequence diagram code blocks.
- size flow nodes from their labels and route edges from node anchors to prevent clipping.
- style diagram nodes, arrows, labels, and notes with theme-aware classes.

* fix(web): sync channel card selection state (QuantumNous#5700)

* fix(web): hide wallet entry in profile dropdown when wallet module disabled (QuantumNous#5708)

The profile dropdown rendered the wallet item unconditionally, so it
still showed after an admin disabled the personal/topup (wallet) sidebar
module. Reuse the sidebar module visibility check so the dropdown honours
the same toggle as the sidebar.

Fixes QuantumNous#5696

* feat(system-settings): add user token limit configuration section (QuantumNous#5678)

* feat: add channel async polling delay toggle

Fixes QuantumNous#5717
Fixes QuantumNous#4244

* fix: add token limit save label translations

* feat: enhance i18n-translate skill

* feat: add date-fns and date-fns-tz dependencies

* feat: add date-fns and date-fns-tz paths to build configuration

* chore(deps): bump dompurify from 3.4.5 to 3.4.11 in /web/default (QuantumNous#5718)

Bumps [dompurify](https://github.com/cure53/DOMPurify) from 3.4.5 to 3.4.11.
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](cure53/DOMPurify@3.4.5...3.4.11)

---
updated-dependencies:
- dependency-name: dompurify
  dependency-version: 3.4.11
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(ci): install classic workspace dependencies for releases (QuantumNous#5719)

* fix: use neutral drawing task labels

* perf(web): streamline table actions and destructive dialogs (QuantumNous#5645)

* perf(data-table): autosize action columns

- exclude actions columns from shared table width calculations so action cells size to their content.
- remove fixed size and w-* width overrides from feature action columns to preserve content-based layout.

* perf(data-table): streamline row action controls

- expose common edit and status actions directly while moving secondary actions into overflow menus.
- add shared row action menu helpers so static and table rows use consistent action controls.
- let action columns size to their content instead of relying on fixed widths.

* fix(web): localize destructive dialog copy

- route delete, reset, and batch update confirmation text through i18n.
- add locale entries for affected channel, model, system settings, and user dialogs.

* perf(web): unify destructive dialog actions

- align delete and cleanup confirmation buttons with the shared destructive variant.
- replace custom destructive color overrides with semantic button variants.
- clean up lint errors in touched dialog files before committing.

* fix(web): add user action success translations

- add localized success messages for user delete, status, and role changes.
- keep user management toast copy available across all frontend locales.

* fix(data-table): prevent mobile badge clipping

- expose badge cell slots so mobile card styles can target nested badge wrappers.
- reset badge margins in card rows to keep provider icons fully visible on small screens.

* fix: add Waffo goods info and webhook SDK update (QuantumNous#5704)

* fix: add Waffo goods info and webhook SDK update

* chore: remove Waffo test code from PR

* fix(model-pricing): refresh tiered expression editor when switching models (QuantumNous#5752)

Switching models in the pricing editor kept the previous model's tiers and prices in the expression panel: TieredPricingEditor seeds its internal visual/raw state only on mount, and the initRef guard never re-ran on prop changes, so only the model name updated.

Bump a reload token in the same effect that seeds billingExpr and use it as the editor's key, so a freshly loaded model remounts the editor and re-parses its expression. The token changes in lockstep with billingExpr, and user edits (which only touch state) do not trigger it.

Closes QuantumNous#5750

* chore(deps): sync bun.lock for dompurify 3.4.11 (QuantumNous#5738)

* fix(theme): 切换前端主题后重置到首页,避免路由 404 (QuantumNous#5612)

* fix(theme): 切换前端主题后重置到首页,避免路由 404
经典前端与新版前端的路由路径不同,切换主题后停留在原路径会导致 404:
- 经典前端切换到新版前端时跳转首页,不再原地刷新当前路径
- 新版前端保存时若前端主题发生变化,保存成功后跳转首页

Fixes QuantumNous#4947

* fix: 更新前端切换提示信息,修正页面跳转逻辑

* fix(task): attribute async task usage log to the initiating node (QuantumNous#5684)

Async task usage logs (LogQuotaData node dimension) were recorded
under whichever node happened to poll the task to completion, not the
node that submitted it. For token/adaptor-billed video tasks the
pre-deduction is often 0, so the entire quota landed on the last
polling node.

Snapshot common.NodeName into TaskPrivateData at submit time and use
it when writing the settlement consume log; fall back to the current
node when empty so existing tasks stay compatible.

* chore: update i18n skill

* feat: better admin permissions (QuantumNous#5755)

* feat: add casbin admin permissions

* feat: improve audit logging to associate logs with actual operators and target users

* feat: enhance admin permissions and UI interactions for sensitive actions

* Refactor authz RBAC and tighten channel permissions

* Split channel authz field policy

* Address channel authz review findings

* fix: adapt ClickHouse log LIKE filters

* feat(playground): improve Playground chat experience and Markdown rendering (QuantumNous#5217)

* refactor(playground): streamline chat request state

- extract conversation actions from the page component to keep message flow logic reusable.
- unify streaming and non-streaming generation state, including abort support for non-stream requests.
- simplify message rendering and payload construction while localizing Playground prompts.

* fix(playground): validate persisted chat state

- wrap saved Playground state with a storage version while still reading legacy values.

- validate config, parameter toggles, and messages before restoring them from localStorage.

- cap stored chat history to the latest messages to avoid oversized or stale state.

* refactor(playground): centralize message content access

- route chat rendering, copy actions, and error display through shared message helpers.

- reuse the current-version update helper for non-streaming assistant responses.

- keep message version details behind utility functions to reduce future model churn.

* refactor(playground): split storage schemas

- move Playground storage validation schemas into a dedicated module.

- keep storage read and write logic focused on migration, trimming, and persistence.

- preserve the existing storage envelope and validation behavior.

* refactor(playground): extract options loading hook

- move model and group queries into a dedicated hook so the page component stays focused on layout wiring.
- preserve existing fallback selection and error toast behavior while reusing the hook through the playground barrel export.

* refactor(playground): extract prompt suggestions

- move static prompt suggestion rendering into a focused component so the input stays centered on compose controls.
- preserve translated suggestion submission behavior while isolating icon metadata from the input form.

* refactor(playground): extract input tools

- move attachment and search controls into a dedicated component so the prompt input stays focused on compose state.
- keep existing development toast behavior and disabled handling while centralizing tool metadata.

* refactor(playground): extract input controls

- move model, group, send, and stop controls into a focused component so the input only manages compose state.
- preserve existing disabled states and generation button behavior while isolating control rendering.

* refactor(playground): extract message content display

- move sources, reasoning, loading, error, and response rendering into a dedicated message content component.
- keep the chat list focused on message iteration, edit state, and action wiring without changing display behavior.

* refactor(playground): extract message editor

- move inline message editing controls into a dedicated editor component so the chat list stays focused on rendering flow.
- preserve save, save-and-submit, cancel, and disabled-state behavior for edited messages.

* refactor(playground): extract stream error parsing

- move SSE error payload parsing into a reusable stream utility so the request hook stays focused on lifecycle handling.
- preserve existing error message, error code, and fallback behavior for raw or empty stream errors.

* refactor(playground): extract request error parsing

- move non-stream request error extraction into a shared utility so the chat handler stays focused on request flow.
- preserve the existing response message, error code, and fallback priority for failed chat completions.

* refactor(playground): extract streaming chunk updates

- move reasoning and content chunk application into a message utility so the chat handler only wires stream events.
- preserve error-state skipping, reasoning accumulation, and content streaming behavior for assistant messages.

* refactor(playground): extract message reasoning parser

- move think tag parsing into a dedicated playground message utility.
- export the parser through the shared playground lib barrel for consistent imports.

* refactor(playground): extract message streaming utilities

- move stream chunk application and message finalization into a dedicated utility.
- keep stored message sanitization with the streaming lifecycle helpers.

* refactor(playground): extract message update utilities

- move assistant message update helpers into a focused playground utility.
- keep error-state message updates separate from core message construction helpers.

* refactor(playground): extract completion choice handling

- move non-streaming choice application into the message streaming utilities.
- keep the chat handler focused on request orchestration and message updates.

* refactor(playground): centralize assistant completion state

- add a helper for finalizing assistant messages with complete status.
- reuse the helper in stream completion and stop-generation paths.

* refactor(playground): extract stream message parsing

- move SSE delta parsing into a shared stream utility.
- keep the stream request hook focused on lifecycle handling and update dispatch.

* refactor(playground): extract stream ready state checks

- move SSE ready-state status handling into stream utilities.
- keep weak source status typing outside the stream request hook.

* refactor(playground): extract conversation message helpers

- move send, regenerate, and edit message list construction into focused utilities.
- keep the conversation hook focused on edit state and update dispatch.

* refactor(playground): extract state initialization helpers

- move playground initial state loading into focused utility helpers.
- centralize message state updater resolution outside the React state hook.

* refactor(playground): extract option fallback helpers

- move model and group fallback selection into focused playground utilities.
- keep the options hook focused on query results, toasts, and config updates.

* refactor(playground): extract message action helpers

- move message action state derivation into focused utilities.

- keep the action component focused on guarded handlers and rendering.

* refactor(playground): extract input control state

- move submit, stop, and selector state derivation into a pure helper.

- keep input controls focused on rendering model selectors and action buttons.

* refactor(playground): extract message content state

- move source, reasoning, loader, and body visibility checks into a pure helper.

- use a discriminated state shape so rendered reasoning content stays type-safe.

* refactor(playground): extract message editor state

- move save eligibility and submit visibility checks into a pure helper.

- keep the editor component focused on textarea and button rendering.

* refactor(playground): extract message error state

- move error kind, fallback content, and admin visibility checks into a pure helper.

- centralize the model pricing settings path used by the error action.

* refactor(playground): extract chat render state

- move editing content lookup and per-message render flags into conversation helpers.

- keep the chat component focused on mapping messages to editor and content views.

* refactor(playground): extract suggestion display state

- move suggestion class selection into a pure helper.

- keep the suggestions component focused on translation and rendering.

* refactor(playground): extract assistant message state checks

- move final and pending assistant status checks into streaming utilities.

- keep the chat handler focused on request lifecycle updates.

* refactor(playground): extract input tool state

- move attachment action metadata and development notices into input tool utilities.

- keep the input tools component focused on menu and button rendering.

* refactor(playground): extract stream protocol checks

- move SSE done-message and closed-ready-state checks into stream utilities.

- keep the stream request hook focused on event handling flow.

* refactor(playground): extract message removal helper

- move delete-message filtering into conversation message utilities.

- keep the conversation hook focused on action orchestration.

* refactor(playground): extract option error messages

- move option load error message selection into playground option utilities
- keep the options hook focused on query effects and fallback updates

* refactor(playground): extract input submit text helper

- move prompt submit text validation into input control utilities
- let the input component submit only when a concrete text value is available

* refactor(playground): centralize error message checks

- add a shared helper for identifying error messages
- remove direct status string checks from message content rendering

* refactor(playground): extract message content display checks

- move loader and content visibility decisions into local helper functions
- keep message content state assembly focused on composing render state

* refactor(playground): replace raw message role checks

- use shared message role constants in conversation edit handling
- avoid raw assistant role literals when validating API messages

* refactor(playground): extract non-stream response handling

- move chat completion response choice handling into message streaming utilities
- keep the chat handler focused on request lifecycle and error routing

* refactor(playground): centralize stream cleanup

- reuse one stream cleanup path for completion, errors, startup failures, and manual stops
- preserve the current-source guard when closing SSE streams

* refactor(playground): extract pending assistant check

- centralize pending assistant message detection in streaming utilities
- reuse the helper when sanitizing stored playground messages

* perf(playground): improve mobile input controls

- split mobile input controls into selector and action rows
- keep the desktop input footer compact while reducing mobile control crowding

* perf(playground): add starter empty state

- show starter prompts in the empty playground chat area
- wire empty-state prompt selection into the existing send flow
- add localized copy for the new empty state

* perf(playground): improve mobile message actions

- collapse mobile message actions into a touch-friendly dropdown menu
- keep the desktop hover action strip unchanged for pointer workflows
- share one action list between desktop buttons and the mobile menu

* perf(playground): add error recovery actions

- show retry, edit, and delete actions inside error message alerts
- route edit recovery to the previous user prompt when available
- keep recovery controls touch-friendly on mobile layouts

* perf(playground): refine message editing experience

- present message edits in a focused bordered editor panel
- add unsaved-change state, reset, and cancel confirmation flows
- improve mobile touch targets and keyboard shortcuts for editing

* perf(playground): improve markdown code blocks

- render fenced markdown code with syntax highlighting, line numbers, and fallback plain text
- add copy, download, and collapse controls for playground AI responses
- tighten code block layout and theme token styles for responsive markdown rendering

* fix(playground): constrain markdown code block height

- collapse long playground code blocks after a short preview instead of waiting for very large snippets
- cap expanded code blocks so long responses scroll inside the code block
- keep generic code block usage unconstrained unless a caller opts in

* feat(playground): add chat history clearing

- add a toolbar action that is enabled only when saved playground messages exist.
- confirm destructive clears before removing browser-stored conversation state.
- add localized strings for the action, dialog, and completion toast.

* perf(playground): improve chat markdown rendering

- refine assistant and user message surfaces so chat content matches the app UI.
- normalize markdown typography, tables, images, lists, blockquotes, and details rendering.
- add indentation cues for collapsible reasoning and source sections.

* style: format code block component

* style: format playground frontend files

* feat(playground): render markdown with stream parser

- replace Streamdown with stream-markdown-parser for project-owned markdown rendering and styling.
- split response rendering into focused block, inline, table, alert, details, and footnote modules.
- pass message final state into response parsing so streaming content can be parsed incrementally.

* fix(playground): localize reasoning and chat feedback

- translate reasoning status, message actions, playground errors, and response renderer fallbacks across supported locales.
- keep reasoning duration numeric and tighten the collapsible layout to prevent trigger jitter.
- register dynamic keys so i18n sync keeps runtime labels covered.

* refactor(playground): group files by functional area

- move chat, input, and message components into focused subdirectories to make the UI structure easier to scan.
- split playground helpers into input, message, streaming, storage, options, state, and suggestions modules.
- update barrel exports and imports so existing feature entry points continue to work.

* fix(playground): prevent history replay from freezing page

- defer saved conversation loading so route entry no longer blocks on localStorage parsing and markdown rendering.
- limit initial history rendering and skip expensive markdown parsing for oversized responses.
- normalize corrupted streaming snapshots and cumulative chunks to keep saved playground history bounded.
- add message timing metadata and layout alignment groundwork without introducing live timers.

* feat(playground): allow regenerating from user messages

- show regenerate actions on user messages with saved content.
- truncate following conversation state before starting a fresh assistant response.

* feat(playground): add raw response source view

- add a per-message source toggle for assistant responses.
- render raw response content with the existing code block viewer.
- localize the new source and preview action labels.

* feat(playground): render code with unified editor

- replace Shiki HTML rendering with a read-only CodeMirror view for code blocks and raw responses.
- reuse the same CodeMirror frame for message editing so source and edit modes stay visually aligned.
- add lightweight CodeMirror dependencies while keeping language support scoped to Markdown.

* perf(playground): streamline chat input controls

- combine model and group selection into one compact picker for faster context switching.
- switch playground action buttons to icon-first controls with tooltips to reduce toolbar width.
- refresh input footer styling and submit states so active and destructive actions are clearer.
- bump dompurify lockfile entry to keep the frontend dependency current.

* fix(playground): filter models by selected group

- query user models by the selected playground group instead of reusing the cross-group model union.
- clear unavailable model selections and block sending when the active group has no models.
- align model selector and error action controls with the existing playground interaction style.

* perf(playground): remove input suggestion chips

- remove the prompt suggestion row below the playground input to reduce visual noise.
- delete the now-unused suggestion component and display helper.

* perf(playground): stabilize reasoning trigger layout

- use fixed icon slots around the reasoning label so the left content stays still when toggling.
- limit the open state animation to the chevron rotation for a smoother collapse interaction.

* perf(playground): smooth reasoning expansion

- use the collapsible panel height animation for vertical reasoning reveals.
- sync inner content opacity and position with the panel state.

* fix(auth): align password validation copy (QuantumNous#5759)

* fix(i18n): add missing frontend translations

- add missing locale entries for API key loading, channel model empty states, auth, playground, and model configuration copy.
- correct inaccurate Russian and Vietnamese model empty-state translations to avoid fallback or misleading copy.

* fix(auth): align password validation copy

- remove the login password length gate so existing shorter passwords are not blocked before reaching the server.
- reuse distinct minimum-length and 8-20 character messages based on the actual validation rule.
- drop unused duplicate password locale keys and align the user creation placeholder with the 8-20 character constraint.

* fix(i18n): add auth validation message translations

- cover schema-driven auth form errors that are translated through FormMessage.
- keep password, username, confirmation, and OTP validation messages available in every locale.

* fix(web): render custom HTML and Markdown content consistently (QuantumNous#5760)

* fix(markdown): render announcement markdown consistently

- support soft line breaks for announcement markdown without changing the default parser behavior.
- add explicit markdown element styles so lists, tables, code blocks, and quotes render correctly when typography styles are unavailable.
- apply the announcement markdown mode in both the popover and detail dialog for consistent display.

* refactor(markdown): simplify fallback markdown styles

- remove duplicate typography utility classes now covered by explicit markdown element fallbacks.
- keep the markdown renderer behavior unchanged while reducing class noise.
- modernize small helper expressions to satisfy targeted lint checks.

* fix(content): render custom HTML consistently

- add shared rich content rendering so custom HTML and Markdown use the same path across public pages and announcements.
- reuse common URL and HTML detection instead of duplicating content format checks per page.
- keep custom home content inside the standard public layout while preserving full-page iframe rendering for external URLs.

* fix(security): pin patched frontend transitive dependencies

* fix(web): secure rich content rendering

* fix(web): harden iframe sandboxing

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: CaIon <i@caion.me>
Co-authored-by: Benson Yan <fuxin04@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Seefs <40468931+seefs001@users.noreply.github.com>
Co-authored-by: QuentinHsu <xuquentinyang@gmail.com>
Co-authored-by: yyhhyyyyyy <yyhhyyyyyy8@gmail.com>
Co-authored-by: feitianbubu <feitianbubu@qq.com>
Co-authored-by: RedwindA <128586631+RedwindA@users.noreply.github.com>
Co-authored-by: zhongyuanzhao-alt <zhongyuan.zhao@waffo.com>
Co-authored-by: peakchao <zhangzhichaolove@vip.qq.com>
ruanhangjian pushed a commit to ruanhangjian/new-api that referenced this pull request Jul 11, 2026
noah-wung pushed a commit to noah-wung/new-api that referenced this pull request Jul 17, 2026
zhaodechao2008 pushed a commit to zhaodechao2008/new-api that referenced this pull request Jul 27, 2026
330079598 pushed a commit to 330079598/new-api that referenced this pull request Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant