Rank skills by installs and harden install telemetry by jesse-merhi · Pull Request #2581 · openclaw/clawhub

jesse-merhi · 2026-06-10T04:00:07Z

Summary

This PR keeps the skill install telemetry and search-ranking work that remains after #2586 and #2633.

Adds skill install telemetry dedupe by user, skill, and UTC day.
Makes telemetry clearing batch-safe for account deletion and ban cleanup.
Adds daily pruning for old skill install telemetry dedupe rows.
Cleans skill install telemetry dedupe rows when a skill is hard-deleted.
Changes skill search popularity to use stars plus all-time installs instead of downloads.

What Changed

Net diff from origin/main to this PR:
11 files changed, 800 insertions, 50 deletions

Current net changed files:

convex/crons.test.ts
convex/crons.ts
convex/schema.ts
convex/search.test.ts
convex/search.ts
convex/skillStatEvents.test.ts
convex/skills.banBatches.test.ts
convex/skills.exportList.test.ts
convex/skills.ts
convex/telemetry.test.ts
convex/telemetry.ts

Branch-only churn with no net diff:

convex/skills.publicListCursor.test.ts
specs/install-telemetry-ranking-abuse.md

Those files are intentionally not described as current PR behavior.

Flow

sequenceDiagram
  actor CLI
  participant Telemetry as telemetry:reportCliInstallInternal
  participant Dedupe as installTelemetryDedupes
  participant Installs as userSkillInstalls
  participant Stats as skillStatEvents

  CLI->>Telemetry: install event with slug and version
  Telemetry->>Dedupe: lookup user + skill + UTC day
  alt already seen today
    Telemetry-->>CLI: return without new stat event
  else first report for day
    Telemetry->>Dedupe: insert dedupe row
    Telemetry->>Installs: upsert rootless install state
    Telemetry->>Stats: insert install_new or install_reactivate
  end

flowchart TD
  A[Account deletion or ban cleanup starts] --> B[Use clearStartedAt cutoff]
  B --> C[Delete old aggregate install rows in bounded batches]
  C --> D[Delete old root rows in bounded batches]
  D --> E[Delete old root install rows in bounded batches]
  E --> F[Delete installTelemetryDedupes for that user]
  F --> G{More rows remain?}
  G -- yes --> H[Schedule next clear batch with same cutoff]
  H --> C
  G -- no --> I[Clear complete]
  J[Daily cron] --> K[Prune stale dedupe day buckets]

Proof

Evidence	Claim Proved	How To Check
Net diff pass	PR body describes the current base-to-head diff, not branch churn	`python3 /Users/jmerhi/repos/AGENT.MDs/skills/pr-proof-pack/scripts/pr_net_diff.py --markdown`
Dedupe flow diagram	Install telemetry is counted once per user/skill/day	`convex/telemetry.ts`, `convex/telemetry.test.ts`
Clear/prune flow diagram	Clear and prune paths are bounded without a persisted clear-state table; legacy root tables are cleanup-only	`convex/telemetry.ts`, `convex/crons.ts`, focused tests
Search popularity tests	Search popularity fallback uses stars plus all-time installs; scoring callers no longer pass downloads	`convex/search.test.ts`

Screenshots

None. The current net diff has no UI files. For this backend/system PR, Mermaid diagrams, tests, and command proof are clearer than route screenshots.

How To Test

Focused local verification:

npm_config_registry=https://registry.npmjs.org BUN_CONFIG_REGISTRY=https://registry.npmjs.org bunx convex codegen
bunx vitest run convex/search.test.ts convex/telemetry.test.ts convex/crons.test.ts convex/skillStatEvents.test.ts convex/skills.exportList.test.ts convex/skills.banBatches.test.ts
git diff --check origin/main...HEAD

Useful read-only spot checks after deploy:

curl 'https://clawhub.ai/api/v1/search?q=swarm&limit=5'
curl 'https://clawhub.ai/api/v1/skills?sort=installs&limit=5'

Expected signals:

search still prioritizes text relevance;
stars and all-time installs are only the search popularity prior;
install-sorted skill listing continues to return high-install rows;
duplicate same-day install telemetry from the same user and skill does not double-count.

Verification

npm_config_registry=https://registry.npmjs.org BUN_CONFIG_REGISTRY=https://registry.npmjs.org bunx convex codegen
# passed

bunx vitest run convex/search.test.ts convex/telemetry.test.ts convex/crons.test.ts convex/skillStatEvents.test.ts convex/skills.exportList.test.ts convex/skills.banBatches.test.ts
# 6 files passed, 82 tests passed

git diff --check origin/main...HEAD
# passed

bunx tsc --noEmit
# passed

vercel · 2026-06-10T04:00:13Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
clawhub	Ready	Preview, Comment	Jun 16, 2026 3:49am

clawsweeper · 2026-06-10T04:01:31Z

Codex review: found issues before merge. Reviewed June 15, 2026, 9:40 PM ET / 01:40 UTC.

Summary
This PR adds Convex install telemetry dedupe and clear/prune state, cleans dedupe rows on skill hard delete, changes skill search popularity to stars plus all-time installs, and adds focused tests plus an internal spec.

Reproducibility: not applicable. this is a feature and compatibility PR rather than a bug report. The observable checks are the linked prod-backed screenshots and the live GitHub mergeability state.

Review metrics: 3 noteworthy metrics.

Merge state: CONFLICTING/DIRTY at 24ec574. The branch needs conflict resolution before ordinary CI or code review can settle merge readiness.
Stored schema surface: 2 tables added, 9 indexes added. New persisted telemetry state makes upgrade compatibility and Convex runtime validation material before merge.
Changed surface: 12 files, +1594/-76. The diff spans Convex functions, schema, tests, hard-delete cleanup, search ranking, and specs.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster ✨ media proof bonus
Patch quality: 🦐 gold shrimp
Result: needs maintainer review before merge.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Resolve conflicts against current main while preserving fix: accept legacy install telemetry snapshots #2656 and feat(cli)!: remove sync command #2669 behavior.
Run a real Convex validation path such as bunx convex dev --once --typecheck=disable plus the targeted Vitest files after rebase.
Refresh PR proof or the PR body if conflict resolution changes visible search or browse behavior.

Risk before merge

[P1] The branch cannot merge as-is because GitHub reports the latest head as CONFLICTING/DIRTY against current main.
[P1] Conflict resolution must preserve current-main legacy install snapshot compatibility from fix: accept legacy install telemetry snapshots #2656 and CLI sync-removal behavior from feat(cli)!: remove sync command #2669 while adding the new dedupe/clear/prune state.
[P1] The PR adds persisted telemetry tables and indexes, so maintainers need Convex runtime validation for fresh installs, legacy snapshots, clear batching, prune scheduling, and reinstall-during-clear after the rebase.
[P1] Changing skill search popularity from downloads to all-time installs is a product/ranking choice that should be intentionally accepted after conflicts are resolved.

Maintainer options:

Rebase with telemetry compatibility validation (recommended)
Resolve the conflicts onto current main while preserving legacy snapshot compatibility, CLI sync-removal behavior, package install metric schema, and this PR's dedupe/clear/prune behavior, then run focused Convex runtime validation.
Accept installs as the search popularity source
Maintainers can intentionally land the ranking change after reviewing the post-rebase search behavior and confirming downloads should no longer drive the search popularity boost.
Narrow or replace the branch
If current main's install-signal direction is now the intended endpoint, close or replace this PR with a smaller telemetry-dedupe follow-up that avoids the ranking decision.

Next step before merge

[P2] The open action is human conflict resolution and compatibility/product judgment, not a safe narrow automated repair on the dirty branch.

Security
Cleared: No concrete security or supply-chain regression was found in the diff; the material concern is compatibility and runtime validation of telemetry behavior.

Review findings

[P1] Rebase and preserve current-main telemetry compatibility — convex/schema.ts:2401

Review details

Best possible solution:

Rebase onto current main, preserve the current telemetry compatibility behavior, keep this PR's dedupe/clear/prune implementation, run Convex runtime validation, then have maintainers explicitly accept the installs-based search-ranking change.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this is a feature and compatibility PR rather than a bug report. The observable checks are the linked prod-backed screenshots and the live GitHub mergeability state.

Is this the best way to solve the issue?

Unclear until the branch is rebased; the implementation direction is plausible, but the current-main compatibility conflict and Convex runtime-validation gap must be settled before this is the best merge path.

Full review comments:

[P1] Rebase and preserve current-main telemetry compatibility — convex/schema.ts:2401
GitHub reports the latest head as CONFLICTING/DIRTY against current main. Before this new persisted telemetry state can land, the rebase needs to keep the current-main legacy snapshot compatibility from fix: accept legacy install telemetry snapshots #2656 and the CLI sync-removal behavior from feat(cli)!: remove sync command #2669 alongside these new tables and indexes.
Confidence: 0.97

Overall correctness: patch is incorrect
Overall confidence: 0.86

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against e3e5705d8969.

Label changes

Label justifications:

P2: This is a normal-priority telemetry and ranking improvement with meaningful but bounded backend and search-ordering impact.
merge-risk: 🚨 compatibility: The PR changes persisted telemetry storage and search-ranking behavior while conflicting with current-main install telemetry compatibility paths.
rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (screenshot): Prod-backed screenshots show the install-sorted skills page, search results, and skill detail behavior; backend telemetry still needs post-rebase runtime validation, but the attached visible proof is useful.
proof: sufficient: Contributor real behavior proof is sufficient. Prod-backed screenshots show the install-sorted skills page, search results, and skill detail behavior; backend telemetry still needs post-rebase runtime validation, but the attached visible proof is useful.
proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. Prod-backed screenshots show the install-sorted skills page, search results, and skill detail behavior; backend telemetry still needs post-rebase runtime validation, but the attached visible proof is useful.

Evidence reviewed

What I checked:

Repository policy read: AGENTS.md was read fully and applied to the Convex/stat/spec/runtime-validation parts of this review where compatible with read-only constraints. (AGENTS.md:1, e3e5705d8969)
Convex guidelines read: The repository-required Convex AI guidelines were read before reviewing the Convex query, mutation, cron, and schema changes. (convex/_generated/ai/guidelines.md:1, e3e5705d8969)
Live PR state: GitHub reports the latest head 24ec57438689dcf52c9bce8fe15308f83e90feed as mergeable=CONFLICTING and mergeStateStatus=DIRTY against base e3e5705. (24ec57438689)
Current main does not implement the central change: Current main has no installTelemetryDedupes/userTelemetryClearState/pruneInstallTelemetryDedupes symbols and still scores search popularity with downloads. (convex/search.ts:131, e3e5705d8969)
PR schema implementation: The PR head adds installTelemetryDedupes and userTelemetryClearState tables plus indexes used by the clear/prune paths. (convex/schema.ts:2401, 24ec57438689)
PR telemetry implementation: The PR head adds bounded prune and phased clear behavior for install telemetry dedupe rows and user install state. (convex/telemetry.ts:139, 24ec57438689)

Likely related people:

Patrick-Erichsen: Recent current-main telemetry and CLI sync-removal work owns the compatibility behavior this PR must preserve while resolving conflicts. (role: recent adjacent owner; confidence: high; commits: e3e5705d8969, ab3c708cc993; files: convex/telemetry.ts, convex/schema.ts, packages/clawhub/src/schema/schemas.ts)
jesse-merhi: Recent merged browse/install-ranking work and this branch both touch the adoption, ranking, and schema surfaces involved here. (role: recent area contributor; confidence: medium; commits: 078425f074da, b2447d750a2c; files: convex/skills.ts, convex/schema.ts, convex/search.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Copilot

Pull request overview

Updates Clawhub’s popularity/ranking model to use all-time installs (with stars as the primary tie-breaker), and adds install-telemetry deduplication plus retention-based pruning to reduce duplicate install counting and abuse signal noise.

Changes:

Switch recommended/search popularity ranking inputs from downloads → all-time installs (stars remain first tie-breaker).
Add per-(user, skill, root, UTC day) install telemetry dedupe table + daily pruning cron.
Update docs/specs and expand regression coverage for install-based ranking and cursor/index compatibility.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
specs/install-telemetry-ranking-abuse.md	Updates the spec to reflect installs as a ranking input and documents dedupe behavior.
docs/http-api.md	Updates API docs for `recommended` sorting to use all-time installs.
docs/api.md	Updates API docs for `recommended` sorting to use all-time installs.
convex/telemetry.ts	Implements install telemetry dedupe + adds internal prune mutation and day-bucketing helper.
convex/telemetry.test.ts	Adds/updates tests for deduped install telemetry behavior.
convex/skills.ts	Switches recommended indexing and fallback logic from downloads-based to installs-based ranking.
convex/skills.publicListCursor.test.ts	Adds regression coverage for stale recommended cursors after index rename.
convex/skills.listPublicPageV4.test.ts	Updates index-field contract and highlighted sorting expectations for installs-based ranking.
convex/search.ts	Switches popularity boost and tie-breaking from downloads → installsAllTime (downloads retained for compatibility).
convex/search.test.ts	Updates and adds tests validating installs-based popularity scoring and ordering.
convex/schema.ts	Renames recommended rank indexes and adds `installTelemetryDedupes` table + indexes.
convex/crons.ts	Schedules daily pruning for install telemetry dedupe rows.
convex/crons.test.ts	Extends internal API mock for the new telemetry prune cron reference.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+export const pruneInstallTelemetryDedupesInternal = internalMutation({
+  args: {},
+  handler: async (ctx) => {
+    const cutoffDayStart = getDayStart(Date.now() - INSTALL_TELEMETRY_DEDUPE_RETENTION_MS);
+    const stale = await ctx.db
+      .query("installTelemetryDedupes")
+      .withIndex("by_day", (q) => q.lt("dayStart", cutoffDayStart))
+      .take(PRUNE_BATCH_SIZE);
+
+    for (const entry of stale) {
+      await ctx.db.delete(entry._id);
+    }
+
+    const hasMore = stale.length === PRUNE_BATCH_SIZE;
+    if (hasMore) {
+      await ctx.scheduler.runAfter(0, internal.telemetry.pruneInstallTelemetryDedupesInternal, {});
+    }
+
+    return { deleted: stale.length, hasMore };
+  },
+});


+    telemetry: {
+      pruneInstallTelemetryDedupesInternal: Symbol("install-telemetry-dedupe-prune"),
+    },


jesse-merhi · 2026-06-11T06:59:42Z

@clawsweeper re-review

clawsweeper · 2026-06-11T06:59:47Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

State: Complete
Detail: The targeted re-review finished, the durable review comment was updated, and the synced verdict was routed.
Run: https://github.com/openclaw/clawsweeper/actions/runs/27329760438
Updated: 2026-06-11T07:04:39.664Z

Copilot AI review requested due to automatic review settings June 10, 2026 04:00

jesse-merhi requested review from a team and Patrick-Erichsen as code owners June 10, 2026 04:00

Copilot started reviewing on behalf of jesse-merhi June 10, 2026 04:00 View session

Copilot AI reviewed Jun 10, 2026

View reviewed changes

This was referenced Jun 10, 2026

Clarify downloads vs installs across ClawHub and OpenClaw CLIs #2405

Closed

Update install telemetry and ranking signals #2535

Merged

vercel Bot deployed to Preview – clawhub June 10, 2026 14:54 View deployment

jesse-merhi changed the title ~~Rank skills by installs and dedupe install telemetry~~ Rank skills by installs and harden install telemetry Jun 10, 2026

vercel Bot deployed to Preview – clawhub June 10, 2026 15:06 View deployment

jesse-merhi force-pushed the jesse/install-metrics-abuse-cleanup branch from a808e16 to eaf0e3a Compare June 11, 2026 04:11

Base automatically changed from jesse/install-metrics-abuse-cleanup to main June 11, 2026 04:22

jesse-merhi force-pushed the jesse/install-telemetry-dedupe-ranking branch from f960445 to 526f32e Compare June 11, 2026 06:59

vercel Bot deployed to Preview – clawhub June 11, 2026 06:59 View deployment

clawsweeper Bot removed the rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. label Jun 11, 2026

jesse-merhi force-pushed the jesse/install-telemetry-dedupe-ranking branch from f43b8ed to 24ec574 Compare June 16, 2026 01:34

vercel Bot deployed to Preview – clawhub June 16, 2026 01:34 View deployment

jesse-merhi force-pushed the jesse/install-telemetry-dedupe-ranking branch from 24ec574 to 7bc81fd Compare June 16, 2026 01:38

vercel Bot deployed to Preview – clawhub June 16, 2026 01:39 View deployment

vercel Bot deployed to Preview – clawhub June 16, 2026 02:14 View deployment

vercel Bot deployed to Preview – clawhub June 16, 2026 02:25 View deployment

vercel Bot deployed to Preview – clawhub June 16, 2026 03:14 View deployment

jesse-merhi added 12 commits June 16, 2026 13:35

docs: add install telemetry ranking spec

62e1717

feat: rank skills by installs and dedupe install telemetry

0ecf6bf

test: cover install telemetry dedupe cleanup

aa339e3

fix: clean install dedupes on skill hard delete

ebbbb87

fix: drain telemetry clear batches

92d3b69

fix: phase telemetry clear batches

bcb69e7

test: cover install stat and export cursor regressions

615cbe8

chore: drop obsolete recommendation ranking changes

4d24f27

test: update skill stat event batch harness

dd3bbe9

docs: remove install telemetry spec

6537c85

chore: remove downloads from search scoring input

081874e

fix: keep install telemetry prune cron guarded

6a047f0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Rank skills by installs and harden install telemetry#2581

Rank skills by installs and harden install telemetry#2581
jesse-merhi merged 12 commits into
mainfrom
jesse/install-telemetry-dedupe-ranking

jesse-merhi commented Jun 10, 2026 •

edited

Loading

Uh oh!

vercel Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

jesse-merhi commented Jun 11, 2026

Uh oh!

clawsweeper Bot commented Jun 11, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

jesse-merhi commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Changed

Flow

Proof

Screenshots

How To Test

Verification

Uh oh!

vercel Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clawsweeper Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

jesse-merhi commented Jun 11, 2026

Uh oh!

clawsweeper Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jesse-merhi commented Jun 10, 2026 •

edited

Loading

vercel Bot commented Jun 10, 2026 •

edited

Loading

clawsweeper Bot commented Jun 10, 2026 •

edited

Loading

clawsweeper Bot commented Jun 11, 2026 •

edited

Loading