test: realign two stale assertions with product behavior (#13313) - #13315
diegosouzapw merged 3 commits into
Conversation
anhtahaylove
changed the title
|
@diegosouzapw — still open and MERGEABLE, test-only ( Short version: #13187 has merged, so the concurrency blocker is gone; the red checks here are the repo-wide ones that also fail on |
The custom-model assertions expected ids prefixed `jina-ai/`, but the catalog prefixes model ids with the provider alias, which is `jina`. The synced-model test directly above asserts `jina/` and passes, so the two cases contradicted each other within the same file. The expectation predates the alias: the test was written in v3.7.9 (2026-05-04) and `alias: "jina"` was added in v3.8.36 (2026-06-25). Aligns the two assertions with the sibling test and with the product. The file now passes 44/44 (was 43 with 1 failure). Confirmed the assertions still bite: renaming the alias to `jina-XX` fails exactly these two cases.
Both tests assert the Portuguese output-style string but configured languageConfig with enabled:false and defaultLanguage:"en". resolveOutputStyleLanguage returns "en" on its first line when enabled is not true, so the English pack was injected and the assertion could never hold. autoDetect:true would not have helped either: the user turns in these fixtures are English, so the detector resolves back to "en". The pack under test has to be pinned, hence autoDetect:false with an explicit defaultLanguage. This restores coverage rather than just turning the suite green. Mutating the pt-BR pack string in outputMode.ts now fails exactly these two tests; with the old fixture the file reported 9 pass / 2 fail whether the pack was intact or mutated, so it detected nothing. File is 11/11 (was 9 + 2 failures). The third languageConfig fixture in this file belongs to an rtk test that makes no language assertion and is left untouched.
7a89889 to
1894077
Compare
|
Correction to my previous comment. I called this PR test-only and backed that with Re-checked properly, this PR really is test-only — both files are tests:
So the conclusion holds here, but it was reached by a broken check, and the same sentence was wrong on #13289 — that one changes Everything else stands: this branch is rebased onto current |
|
Thanks for digging into this. I reproduced both failures on the release tip, and I think the pt-BR half of the diagnosis stops one step short. Evidence below. The fixture's global The user turn is Setting the global config to |
|
Nice, focused split from #13292 — both stale-assertion fixes check out. Ran |
Reverts 8d371d4. The catalog file holds two Jina cases that are not the same scenario: the custom-model/alias case legitimately expects the `jina/` alias prefix, while the specialty-model case expects the canonical provider id `jina-ai/`. Aligning the second to the first reads like a fix but changes a passing assertion into a failing one. Verified against the current release tip by mutation, both directions: the specialty case passes as `jina-ai/...` and fails as `jina/...`, with the runner reporting `actual: 'jina-ai/jina-embeddings-v5-text-small'`. The canonical ids are what the source declares — `embeddingRegistry.ts` and `rerankRegistry.ts` both key the provider as `jina-ai`, as does EMBEDDING_RERANK_PROVIDER_IDS in src/shared/constants/providers.ts. The pt-BR language-pack commit on this branch is untouched: that one is a real fix and repairs two genuinely failing assertions.
1603c86
into
diegosouzapw:release/v3.8.51
…w#13313) (diegosouzapw#13315) * test: expect the jina alias prefix in the custom-model catalog case The custom-model assertions expected ids prefixed `jina-ai/`, but the catalog prefixes model ids with the provider alias, which is `jina`. The synced-model test directly above asserts `jina/` and passes, so the two cases contradicted each other within the same file. The expectation predates the alias: the test was written in v3.7.9 (2026-05-04) and `alias: "jina"` was added in v3.8.36 (2026-06-25). Aligns the two assertions with the sibling test and with the product. The file now passes 44/44 (was 43 with 1 failure). Confirmed the assertions still bite: renaming the alias to `jina-XX` fails exactly these two cases. * test: pin the pt-BR pack in the two language-pack fixtures Both tests assert the Portuguese output-style string but configured languageConfig with enabled:false and defaultLanguage:"en". resolveOutputStyleLanguage returns "en" on its first line when enabled is not true, so the English pack was injected and the assertion could never hold. autoDetect:true would not have helped either: the user turns in these fixtures are English, so the detector resolves back to "en". The pack under test has to be pinned, hence autoDetect:false with an explicit defaultLanguage. This restores coverage rather than just turning the suite green. Mutating the pt-BR pack string in outputMode.ts now fails exactly these two tests; with the old fixture the file reported 9 pass / 2 fail whether the pack was intact or mutated, so it detected nothing. File is 11/11 (was 9 + 2 failures). The third languageConfig fixture in this file belongs to an rtk test that makes no language assertion and is left untouched. * test: keep the canonical jina-ai prefix in the catalog case Reverts 8d371d4. The catalog file holds two Jina cases that are not the same scenario: the custom-model/alias case legitimately expects the `jina/` alias prefix, while the specialty-model case expects the canonical provider id `jina-ai/`. Aligning the second to the first reads like a fix but changes a passing assertion into a failing one. Verified against the current release tip by mutation, both directions: the specialty case passes as `jina-ai/...` and fails as `jina/...`, with the runner reporting `actual: 'jina-ai/jina-embeddings-v5-text-small'`. The canonical ids are what the source declares — `embeddingRegistry.ts` and `rerankRegistry.ts` both key the provider as `jina-ai`, as does EMBEDDING_RERANK_PROVIDER_IDS in src/shared/constants/providers.ts. The pt-BR language-pack commit on this branch is untouched: that one is a real fix and repairs two genuinely failing assertions. --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
2 participants
Two test files assert things the product does not do. Both fail on
release/v3.8.51before this branch, and neither is a product bug — theexpectations are simply stale. Split out of #13292 so the teardown work can be
reviewed sepa...[truncated]