Skip to content

fix: surface kanban completion evidence and validate skills - #28752

Open
jiayuxiaochaoren wants to merge 3 commits into
NousResearch:mainfrom
jiayuxiaochaoren:fix/kanban-completion-evidence-invalid-skills
Open

fix: surface kanban completion evidence and validate skills#28752
jiayuxiaochaoren wants to merge 3 commits into
NousResearch:mainfrom
jiayuxiaochaoren:fix/kanban-completion-evidence-invalid-skills

Conversation

@jiayuxiaochaoren

Copy link
Copy Markdown

Summary

  • fix Kanban legacy DB initialization when tasks.session_id is missing by creating the session_id index after additive migrations
  • validate per-task skill identifiers so natural-language labels like ppt skill fail fast instead of causing worker startup crash loops
  • add dashboard completion evidence rollups for task details: run counts, suspicious completion reasons, dependency completion, deliverables, and invalid skills

Context

A legacy Kanban board upgraded to the newer schema can fail loading the dashboard with:

sqlite3.OperationalError: no such column: session_id

The root cause is that SCHEMA_SQL created idx_tasks_session_id before _migrate_add_optional_columns() had a chance to add tasks.session_id to older DBs.

A separate but related operational issue: if a task stores a natural-language per-task skill such as ppt skill, workers are spawned with --skills "ppt skill" and exit before doing work with:

Error: Unknown skill(s): ppt skill

The dashboard had the raw run history/logs, but completion evidence was not summarized above the fold, making it hard to tell whether dependency tasks completed and why the root task kept failing.

Test Plan

  • ./venv/bin/python -m py_compile hermes_cli/kanban_db.py plugins/kanban/dashboard/plugin_api.py
  • ./venv/bin/python -m pytest tests/hermes_cli/test_kanban_db_init.py -q
  • ./venv/bin/python -m pytest tests/hermes_cli/test_kanban_db.py::test_session_id_index_exists tests/hermes_cli/test_kanban_db.py::test_create_task_rejects_natural_language_skill_names tests/hermes_cli/test_kanban_db.py::test_create_task_accepts_slug_like_skill_names -q
  • ./venv/bin/python -m pytest tests/plugins/test_kanban_dashboard_plugin.py::test_task_detail_completion_evidence_rolls_up_dependencies tests/plugins/test_kanban_dashboard_plugin.py::test_dashboard_bundle_renders_completion_evidence_section -q

Note: I also ran the broader Kanban DB/dashboard plugin tests locally. Three unrelated home-channel tests failed because my local real ~/.hermes has a Weixin home channel configured, which leaks into those tests' expectations.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins labels May 19, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

The index migration fix in this PR overlaps with #28461, #28602, #28741, and #28754 (all fixing #28464). Note: this PR only moves the session_id index, while #28754 moves all 3 optional-column indexes (session_id, tenant, idempotency_key). The skill validation and dashboard evidence features are unique to this PR.

@jiayuxiaochaoren

Copy link
Copy Markdown
Author

Thanks for the context — that matches what I saw locally.

I’m happy to treat the index migration hunk in this PR as incidental/overlapping with #28461/#28602/#28741/#28754. If #28754 is the preferred migration fix since it covers all three optional-column indexes (session_id, tenant, idempotency_key), I can rebase/drop my session_id index change after it lands.

The parts I’d like to keep unique in this PR are:

  • per-task skill validation so natural-language labels like ppt skill fail fast instead of causing worker startup crash loops with Unknown skill(s)
  • dashboard task-detail completion evidence rollup showing suspicious completions, run counts, dependency completion, deliverables, and invalid skills

I can also split those unique pieces into a smaller follow-up PR if that would be easier to review.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the cleanup here. The unique parts are worth salvaging, but I found two correctness issues to fix before this should be carried forward.

Problems

  • plugins/kanban/dashboard/plugin_api.py:669-681 builds completion_evidence from the same runs list returned to the drawer. That list is already filtered by run_state_type/run_state_name, so filtering the drawer to failed runs also makes the evidence rollup lose completed-run counts and latest completion data. Evidence should use an unfiltered all-runs query; only the visible run-history list should be filtered.
  • hermes_cli/kanban_db.py:1426 rejects skill names that contain /, but current main's loader accepts categorized skill paths via skill_view() direct-path lookup (tools/skills_tool.py:869-870, tools/skills_tool.py:1020-1024). That means valid per-task skill identifiers like devops/kanban-worker would be rejected along with natural-language labels.

Suggested changes

  • Keep the completion rollup, but compute it from all runs for the task and pass the filtered runs only to the existing runs response field.
  • Validate per-task skills against the existing skill-loading grammar/resolution path, or at minimum allow categorized path identifiers while still rejecting whitespace/comma labels like ppt skill.
  • Drop the index-migration hunk during salvage; current main already moved additive indexes after column migrations in hermes_cli/kanban_db.py:1696-1709.

Automated hermes-sweeper review.

)
],
"runs": [_run_dict(r) for r in runs],
"completion_evidence": _completion_evidence(conn, task, runs),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This passes the run-filtered runs list into the evidence rollup, so ?run_state_type=outcome&run_state_name=failed makes completion evidence report zero completed runs even if the task has completed attempts. Fetch an unfiltered run list for _completion_evidence and keep this filtered list only for the drawer's run history.

Comment thread hermes_cli/kanban_db.py
f"skill name cannot contain comma: {name!r} "
f"(pass a list of separate names instead of a comma-joined string)"
)
if not _VALID_TASK_SKILL_RE.match(name):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This rejects /, but the existing skill loader accepts categorized skill paths such as devops/kanban-worker through skill_view() direct-path lookup. The validation should match the loader's accepted identifier grammar, otherwise valid per-task skills get blocked.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants