Update the kilo enkrypt integration for new models - #6808
Conversation
| modelId: 'x-ai/grok-4.6', | ||
| }, | ||
| { | ||
| identity: { model_name: 'laguna-s-2.1', provider: 'openai_compatible', source: 'xAI' }, |
There was a problem hiding this comment.
WARNING: laguna-s-2.1 is mapped with source: 'xAI' while its canonical target is poolside/laguna-s-2.1.
source is part of the exact identity key used by matchEnkryptScores (identity.source === score.source), and Laguna models are published by Poolside — the sibling entry laguna-xs-2.1 below correctly uses source: 'poolside'. If Enkrypt reports poolside for this record, the mapping never matches and the model's stats are dropped as unreviewed_identity. The fixture at apps/web/src/tests/fixtures/enkrypt-scores.ts:154 mirrors the same value, so the test suite cannot catch it.
| identity: { model_name: 'laguna-s-2.1', provider: 'openai_compatible', source: 'xAI' }, | |
| identity: { model_name: 'laguna-s-2.1', provider: 'openai_compatible', source: 'poolside' }, |
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
There was a problem hiding this comment.
The feedback is not accurate. Fresh live API call (seconds ago) confirms Enkrypt reports:
laguna-s-2.1 | provider: openai_compatible | source: xAI
laguna-xs-2.1 | provider: openai_compatible | source: poolside
Our mapping's source: 'xAI' for laguna-s-2.1 is exactly what the real leaderboard returns — captured from the live response when I added the mappings, not assumed. Enkrypt's source field reflects where they served the model for evaluation, not the publisher: they ran Laguna S 2.1 through xAI's endpoint and Laguna XS 2.1 through Poolside's, hence the sibling entries differ.
If we "corrected" it to 'poolside' per the feedback, the identity (laguna-s-2.1, openai_compatible, poolside) would never match any real Enkrypt record — creating exactly the silent unreviewed_identity drop the reviewer's warning describes. The fixture mirroring the mapping is by design (it encodes reviewed identities), so the test suite enforcing mapping↔fixture parity is working as intended; live API validation like this is the actual safety check.
The canonical target poolside/laguna-s-2.1 is right — only the identity triple must match Enkrypt's record, and it does. No change made.
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Files Reviewed (4 files)
Previous Review Summary (commit 7c07454)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 7c07454)Status: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Files Reviewed (4 files)
Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0 Review guidance: REVIEW.md from base branch |
7c07454 to
d6258c1
Compare
Summary
We are tracking more models (see https://anaconda.atlassian.net/browse/AISAGE-35 )
and want to have this data shown on the kilo dashboard as well.
To do so, we need to update the mapping of enkrypt models to the model id.
Verification
Will double check the cron job after it runs, but the existing mappings were pulled from both enkrypt as well as the kilo leaderboard
curl -sS --max-time 30 'https://api.enkryptai.com/leaderboard/v2/scores'and
curl -sS --max-time 20 'https://api.kilo.ai/api/gateway/models'