Skip to content

Update the kilo enkrypt integration for new models - #6808

Merged
intentionally-left-nil merged 2 commits into
mainfrom
update-enkrypt-models
Sep 28, 2026
Merged

intentionally-left-nil merged 2 commits into
mainfrom
update-enkrypt-models

Conversation

@intentionally-left-nil

Copy link
Copy Markdown
Contributor

Summary

We are tracking more models (see https://anaconda.atlassian.net/browse/AISAGE-35 )
and want to have this data shown on the kilo dashboard as well.

To do so, we need to update the mapping of enkrypt models to the model id.

Verification

Will double check the cron job after it runs, but the existing mappings were pulled from both enkrypt as well as the kilo leaderboard
curl -sS --max-time 30 'https://api.enkryptai.com/leaderboard/v2/scores'

and
curl -sS --max-time 20 'https://api.kilo.ai/api/gateway/models'

modelId: 'x-ai/grok-4.6',
},
{
identity: { model_name: 'laguna-s-2.1', provider: 'openai_compatible', source: 'xAI' },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: laguna-s-2.1 is mapped with source: 'xAI' while its canonical target is poolside/laguna-s-2.1.

source is part of the exact identity key used by matchEnkryptScores (identity.source === score.source), and Laguna models are published by Poolside — the sibling entry laguna-xs-2.1 below correctly uses source: 'poolside'. If Enkrypt reports poolside for this record, the mapping never matches and the model's stats are dropped as unreviewed_identity. The fixture at apps/web/src/tests/fixtures/enkrypt-scores.ts:154 mirrors the same value, so the test suite cannot catch it.

Suggested change
identity: { model_name: 'laguna-s-2.1', provider: 'openai_compatible', source: 'xAI' },
identity: { model_name: 'laguna-s-2.1', provider: 'openai_compatible', source: 'poolside' },

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The feedback is not accurate. Fresh live API call (seconds ago) confirms Enkrypt reports:
laguna-s-2.1 | provider: openai_compatible | source: xAI
laguna-xs-2.1 | provider: openai_compatible | source: poolside
Our mapping's source: 'xAI' for laguna-s-2.1 is exactly what the real leaderboard returns — captured from the live response when I added the mappings, not assumed. Enkrypt's source field reflects where they served the model for evaluation, not the publisher: they ran Laguna S 2.1 through xAI's endpoint and Laguna XS 2.1 through Poolside's, hence the sibling entries differ.
If we "corrected" it to 'poolside' per the feedback, the identity (laguna-s-2.1, openai_compatible, poolside) would never match any real Enkrypt record — creating exactly the silent unreviewed_identity drop the reviewer's warning describes. The fixture mirroring the mapping is by design (it encodes reviewed identities), so the test suite enforcing mapping↔fixture parity is working as intended; live API validation like this is the actual safety check.
The canonical target poolside/laguna-s-2.1 is right — only the identity triple must match Enkrypt's record, and it does. No change made.

@kilo-code-bot

kilo-code-bot Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (4 files)
  • apps/web/src/app/api/cron/sync-model-stats/route.test.ts
  • apps/web/src/lib/model-stats/enkrypt-identity.test.ts
  • apps/web/src/lib/model-stats/enkrypt-identity.ts
  • apps/web/src/tests/fixtures/enkrypt-scores.ts
Previous Review Summary (commit 7c07454)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 7c07454)

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 1
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/lib/model-stats/enkrypt-identity.ts 392 laguna-s-2.1 identity uses source: 'xAI' but maps to poolside/laguna-s-2.1; sibling laguna-xs-2.1 uses source: 'poolside'
Files Reviewed (4 files)
  • apps/web/src/lib/model-stats/enkrypt-identity.ts - 1 issue
  • apps/web/src/tests/fixtures/enkrypt-scores.ts - mirrors the same mapping (fixture confirms tests cannot catch it)
  • apps/web/src/lib/model-stats/enkrypt-identity.test.ts - 0 issues
  • apps/web/src/app/api/cron/sync-model-stats/route.test.ts - 0 issues

Fix these issues in Kilo Cloud


Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0

Review guidance: REVIEW.md from base branch main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants