Skip to content

feat(skills): Psyche network monitor skill-only rework with multi-run support - #183

Closed
eren-karakus0 wants to merge 5 commits into
NousResearch:mainfrom
eren-karakus0:feat/psyche-skill-rework
Closed

feat(skills): Psyche network monitor skill-only rework with multi-run support#183
eren-karakus0 wants to merge 5 commits into
NousResearch:mainfrom
eren-karakus0:feat/psyche-skill-rework

Conversation

@eren-karakus0

@eren-karakus0 eren-karakus0 commented Feb 28, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

Reworked the Psyche network monitor skill to address reviewer feedback about HuggingFace-based run discovery.

Key Changes

  • Primary discovery source changed: psyche.network/api/runs is now the primary run discovery source instead of HuggingFace
  • HuggingFace demoted to checkpoint-only: Two-pass search (org-scoped then broad) for checkpoint lookup, not run discovery
  • Correct API parsing: Fixed response shape ({"runs":[...]} not array) and status field ({"type":"active"} not string)
  • Source labels: Every data point tagged with [tracked], [on-chain], [best-effort], or [unverified]
  • Context-dependent fallback: Separate chains for discovery (API → web) vs detail (API → CLI → web)
  • Allowlist-aware: Handles "API 200 but run not listed" scenario gracefully
  • Solana RPC as reference only: Borsh deserialization not feasible for agent, documented but not in practical fallback

Files Changed

  • skills/psyche/network-monitor/SKILL.md — Complete rewrite
  • skills/psyche/network-monitor/references/commands.md — Complete rewrite (Psyche API > CLI > HF > RPC ordering)
  • skills/psyche/network-monitor/references/run-states.md — Updated Finished state (no longer assumes HF)

What This Does NOT Do

  • No files added to tools/ (skill-only, as requested)
  • No new dependencies

Test Plan

  • psyche.network/api/runs response correctly parsed (verified against live API: 15 runs, status objects)
  • Run found on API but not on HF → reports [tracked] with "checkpoints may be P2P/GCS/ephemeral"
  • Broad HF search finds non-PsycheFoundation checkpoints
  • "API 200 but run not in list" → [tracked:not-found] with CLI suggestion
  • No encoding issues in files

@eren-karakus0
feat(skills): add Psyche network monitor skill with multi-run support
2597b66 
Rework of NousResearch#35 based on maintainer feedback — removes the tool entirely
and ships only as a skill. No changes to tools/, toolsets.py,
model_tools.py, or tests/.

- Dynamic multi-run discovery via HuggingFace API (no hardcoded runs)
- Fallback strategy: HF API → Solana RPC → Psyche CLI
- Run state reference aligned with official Psyche documentation
- Command reference with curl + python3 parsing examples
- Participation guide for mining pool, compute, and code contributions
@eren-karakus0 eren-karakus0 changed the title feat(skills): Psyche network monitor — skill-only rework with multi-run support feat(skills): Psyche network monitor skill-only rework with multi-run support Feb 28, 2026
@eren-karakus0
fix(skills): use createdAt instead of lastModified in model list endp…
edca3d7 
…oint

The HuggingFace /models list endpoint returns createdAt but not
lastModified. The lastModified field is only available on the
individual model detail endpoint. Updated parsing examples accordingly.
@eren-karakus0
fix(skills): fix syntax error, add missing states, clarify CLI invoca…
65e3f8c 
…tion

- Fix Python f-string quote collision in commands.md parse example
- Add Uninitialized and Finished states to run-states.md (from Psyche glossary)
- Clarify psyche-solana-client vs run-manager invocation pattern
@teknium1

teknium1 commented Mar 1, 2026

Copy link
Copy Markdown
Collaborator

Still, checking huggingface shows near none of the runs that are active or completed - many don't ever get on HuggingFace and many will not be placed on PsycheFoundations' HuggingFace (i.e. other users' runs, etc)

@eren-karakus0
fix(skills): use Psyche API as primary run discovery, HF only for che…
c392dc9 
…ckpoint lookup

- Replace HuggingFace-based run discovery with psyche.network/api/runs as primary source
- HuggingFace now used only for checkpoint search (best-effort, two-pass: org-scoped then broad)
- Fix /api/runs parsing: response is {"runs":[...]} not a direct array
- Fix status field handling: status is an object {"type":"active"} not a string
- Add source labels ([tracked], [on-chain], [best-effort], [unverified]) for data provenance
- Add context-dependent fallback chains (discovery vs detail)
- Handle "API 200 but run not in list" scenario (allowlist filtering)
- Document that Solana RPC is reference-only (Borsh parse not feasible for agent)
- Remove incorrect "Finished → check HuggingFace" guidance from run-states
- Add disambiguation rules for multiple HF matches in broad search
@eren-karakus0

eren-karakus0 commented Mar 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

@teknium1 You're absolutely right I've reworked the skill based on your feedback.

What changed:

  1. Run discovery now uses psyche.network/api/runs as the primary source (not HuggingFace). The API returns 15 tracked runs with status, arch, tokens, etc.

  2. HuggingFace is demoted to checkpoint lookup only (best-effort). It does a two-pass search:

    • First: org-scoped (author=PsycheFoundation)
    • Then: broad search (catches checkpoints uploaded to other users' HF accounts)

  3. The skill explicitly states that not all runs have HF checkpoints they may be P2P, GCS, or ephemeral. Only ~33% of tracked runs match HF models.

  4. Fixed actual API parsing the response is {"runs":[...]} (not a flat array) and status is an object like {"type":"active"} (not a string). Previous parsing examples would have errored on the live API.

  5. Every data point is now tagged with source labels ([tracked], [on-chain], [best-effort], [unverified]) so users know the reliability of what they're seeing.

Still skill-only, no tools/ changes.

@teknium1

teknium1 commented Mar 3, 2026

Copy link
Copy Markdown
Collaborator

Will send to psyche team to review sometime soon to see if they like!

@teknium1 teknium1 closed this Mar 13, 2026
This was referenced Jun 2, 2026
Closed
Merged
Closed
Closed
This was referenced Aug 9, 2026
Closed
Merged
sweetrb added a commit to sweetrb/hermes-agent that referenced this pull request Sep 23, 2026
@sweetrb
fix(optional-mcps): re-pin apple-notes-mcp 2.8.13 -> 2.8.49
a863320 
2.8.49 fixes defects in default_enabled tools since 2.8.13
(sweetrb/apple-notes-mcp): search-notes body search timing out on broad
terms (NousResearch#100, now a read-only database path when Full Disk Access is
granted); list-notes returning Recently Deleted notes as live (NousResearch#207, now
excluded by default -- a behaviour change); list-shared-notes (NousResearch#183) and
list-attachments (NousResearch#197) returning duplicates; get-checklist-state and every
body reader failing on subscript text (NousResearch#188); and doctor's Full Disk Access
remediation naming Claude.app instead of the Node binary (NousResearch#220).

Validated on macOS per catalog policy: npx -y apple-notes-mcp@2.8.49
completes the initialize handshake reporting serverInfo.version 2.8.49,
tools/list returns 66 tools all with descriptions (15 added, none removed),
and all 20 default_enabled entries resolve. Nothing added to
default_enabled. tests/hermes_cli/test_mcp_catalog.py: 39 passed.

Host: robs-work-mbpromax
@sweetrb sweetrb mentioned this pull request Sep 23, 2026
Open
1 task done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@eren-karakus0 @teknium1