feat: add embeddings support to integration tests#212
Conversation
Summary by CodeRabbit
WalkthroughThis update introduces OpenAI embeddings support across the Bifrost HTTP transport integration. It adds new embedding request/response types, endpoint routing, and request handling logic. The integration test suite and documentation are expanded to cover embeddings, with new test data, validation utilities, and configuration entries for embedding models and capabilities. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant OpenAI Router
participant Bifrost
participant ModelBackend
Client->>OpenAI Router: POST /openai/v1/embeddings (OpenAIEmbeddingRequest)
OpenAI Router->>OpenAI Router: Convert to BifrostRequest
OpenAI Router->>Bifrost: EmbeddingRequest(BifrostRequest)
Bifrost->>ModelBackend: Generate Embeddings
ModelBackend-->>Bifrost: Embedding Vectors
Bifrost-->>OpenAI Router: BifrostResponse (Embeddings)
OpenAI Router->>OpenAI Router: Convert to OpenAIEmbeddingResponse
OpenAI Router-->>Client: OpenAIEmbeddingResponse
Estimated code review effort🎯 4 (Complex) | ⏱️ ~40 minutes Possibly related PRs
Suggested reviewers
Poem
✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (8)
tests/transports-integrations/README.md(6 hunks)tests/transports-integrations/config.yml(4 hunks)tests/transports-integrations/tests/integrations/test_openai.py(4 hunks)tests/transports-integrations/tests/utils/common.py(2 hunks)transports/bifrost-http/integrations/openai/router.go(1 hunks)transports/bifrost-http/integrations/openai/types.go(6 hunks)transports/bifrost-http/integrations/utils.go(1 hunks)transports/bifrost-http/lib/store.go(1 hunks)
🧰 Additional context used
🧠 Learnings (9)
📚 Learning: in transports/bifrost-http/lib/store.go, the loadfromconfig method already handles missing config fi...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#149
File: transports/bifrost-http/main.go:259-261
Timestamp: 2025-07-08T18:20:24.086Z
Learning: In transports/bifrost-http/lib/store.go, the LoadFromConfig method already handles missing config files gracefully by initializing defaults and auto-detecting providers from environment variables. It only returns errors for actual parsing/processing failures, not missing files, making log.Fatalf appropriate in main.go since real errors should terminate the program.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: the `lib.converttobifrostcontext` function in the bifrost http transport never returns nil and handl...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/router.go:62-64
Timestamp: 2025-06-09T11:27:00.925Z
Learning: The `lib.ConvertToBifrostContext` function in the bifrost HTTP transport never returns nil and handles the conversion internally, so nil checks are not needed when calling this function.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: the `lib.converttobifrostcontext` function in the bifrost http transport never returns nil and alway...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/router.go:62-64
Timestamp: 2025-06-09T11:27:00.925Z
Learning: The `lib.ConvertToBifrostContext` function in the bifrost HTTP transport never returns nil and always returns a pointer to a valid context. It starts with context.Background() and only adds values to it, so nil checks are not needed when calling this function.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: the generic router in transports/bifrost-http/integrations/utils.go already handles nil pointers fro...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#67
File: transports/bifrost-http/integrations/anthropic/router.go:26-34
Timestamp: 2025-06-10T11:19:29.604Z
Learning: The Generic router in transports/bifrost-http/integrations/utils.go already handles nil pointers from RequestConverter functions. When a RequestConverter returns a nil *schemas.BifrostRequest, the Generic router automatically returns an HTTP 400 error with "Invalid request" message, making additional nil checks in individual router implementations redundant.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost codebase, errors returned from client methods like chatcompletionrequest are of type ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/end_to_end_tool_calling.go:43-45
Timestamp: 2025-06-16T04:13:55.437Z
Learning: In the Bifrost codebase, errors returned from client methods like ChatCompletionRequest are of type BifrostError, not the standard error interface. For testing these errors, use require.Nilf instead of require.NoErrorf since BifrostError doesn't work with the standard error assertion methods.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrosterror struct, the error field is a struct (not a pointer), so accessing bifrosterr.err...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#80
File: plugins/maxim/main.go:249-264
Timestamp: 2025-06-14T06:17:54.426Z
Learning: In the BifrostError struct, the Error field is a struct (not a pointer), so accessing bifrostErr.Error.Message, bifrostErr.Error.Code, and bifrostErr.Error.Type is safe without nil checks on the Error field itself. The Code and Type fields are of type *string.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost codebase, errors returned from methods like chatcompletionrequest are of type bifrost...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/simple_chat.go:39-41
Timestamp: 2025-06-16T04:13:42.755Z
Learning: In the Bifrost codebase, errors returned from methods like ChatCompletionRequest are of type BifrostError (a custom error type) rather than the standard Go error interface. Therefore, require.Nilf should be used for error assertions instead of require.NoErrorf.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: the openai `name` field on messages cannot be preserved when converting to bifrost format because th...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/types.go:223-231
Timestamp: 2025-06-10T11:06:06.670Z
Learning: The OpenAI `name` field on messages cannot be preserved when converting to Bifrost format because the `schemas.BifrostMessage` struct in bifrost/core does not support a Name field. This is a known limitation of the Bifrost core schema design.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost system, error returns are of type `bifrosterror` rather than the standard go `error` ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/complete_end_to_end.go:39-41
Timestamp: 2025-06-16T04:12:05.427Z
Learning: In the Bifrost system, error returns are of type `BifrostError` rather than the standard Go `error` interface. Therefore, use `require.Nilf(t, err, ...)` instead of `require.NoError(t, err)` when checking for errors in Bifrost function calls.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
🪛 Ruff (0.12.2)
tests/transports-integrations/tests/integrations/test_openai.py
97-97: ..utils.common.EMBEDDINGS_EMPTY_TEXTS imported but unused
Remove unused import
(F401)
102-102: ..utils.common.assert_embeddings_similarity imported but unused
Remove unused import
(F401)
103-103: ..utils.common.assert_embeddings_dissimilarity imported but unused
Remove unused import
(F401)
759-759: Missing return type annotation for public function test_21_single_text_embedding
Add return type annotation: None
(ANN201)
759-759: Missing type annotation for function argument openai_client
(ANN001)
759-759: Missing type annotation for function argument test_config
(ANN001)
759-759: Unused method argument: test_config
(ARG002)
762-762: Trailing comma missing
Add trailing comma
(COM812)
779-779: Missing return type annotation for public function test_22_batch_text_embeddings
Add return type annotation: None
(ANN201)
779-779: Missing type annotation for function argument openai_client
(ANN001)
779-779: Missing type annotation for function argument test_config
(ANN001)
779-779: Unused method argument: test_config
(ARG002)
782-782: Trailing comma missing
Add trailing comma
(COM812)
787-787: Trailing comma missing
Add trailing comma
(COM812)
798-798: Missing return type annotation for public function test_23_embedding_similarity_analysis
Add return type annotation: None
(ANN201)
798-798: Missing type annotation for function argument openai_client
(ANN001)
798-798: Missing type annotation for function argument test_config
(ANN001)
798-798: Unused method argument: test_config
(ARG002)
801-801: Trailing comma missing
Add trailing comma
(COM812)
805-805: Trailing comma missing
Add trailing comma
(COM812)
827-827: Missing return type annotation for public function test_24_embedding_dissimilarity_analysis
Add return type annotation: None
(ANN201)
827-827: Missing type annotation for function argument openai_client
(ANN001)
827-827: Missing type annotation for function argument test_config
(ANN001)
827-827: Unused method argument: test_config
(ARG002)
830-830: Trailing comma missing
Add trailing comma
(COM812)
834-834: Trailing comma missing
Add trailing comma
(COM812)
842-842: Trailing comma missing
Add trailing comma
(COM812)
846-846: Trailing comma missing
Add trailing comma
(COM812)
865-865: Missing return type annotation for public function test_25_embedding_different_models
Add return type annotation: None
(ANN201)
865-865: Missing type annotation for function argument openai_client
(ANN001)
865-865: Missing type annotation for function argument test_config
(ANN001)
865-865: Unused method argument: test_config
(ARG002)
871-871: Trailing comma missing
Add trailing comma
(COM812)
878-878: Trailing comma missing
Add trailing comma
(COM812)
888-888: Trailing comma missing
Add trailing comma
(COM812)
891-891: Do not catch blind exception: Exception
(BLE001)
896-896: Missing return type annotation for public function test_26_embedding_long_text
Add return type annotation: None
(ANN201)
896-896: Missing type annotation for function argument openai_client
(ANN001)
896-896: Missing type annotation for function argument test_config
(ANN001)
896-896: Unused method argument: test_config
(ARG002)
899-899: Trailing comma missing
Add trailing comma
(COM812)
911-911: Missing return type annotation for public function test_27_embedding_error_handling
Add return type annotation: None
(ANN201)
911-911: Missing type annotation for function argument openai_client
(ANN001)
911-911: Missing type annotation for function argument test_config
(ANN001)
911-911: Unused method argument: test_config
(ARG002)
915-915: pytest.raises(Exception) is too broad, set the match parameter or use a more specific exception
(PT011)
917-917: Trailing comma missing
Add trailing comma
(COM812)
926-926: Trailing comma missing
Add trailing comma
(COM812)
932-932: Do not catch blind exception: Exception
(BLE001)
934-936: Found assertion on exception e in except block, use pytest.raises() instead
(PT017)
934-936: Found assertion on exception e in except block, use pytest.raises() instead
(PT017)
939-939: Missing return type annotation for public function test_28_embedding_dimensionality_reduction
Add return type annotation: None
(ANN201)
939-939: Missing type annotation for function argument openai_client
(ANN001)
939-939: Missing type annotation for function argument test_config
(ANN001)
939-939: Unused method argument: test_config
(ARG002)
951-951: Trailing comma missing
Add trailing comma
(COM812)
956-956: Trailing comma missing
Add trailing comma
(COM812)
967-967: Trailing comma missing
Add trailing comma
(COM812)
970-970: Do not catch blind exception: Exception
(BLE001)
975-975: Missing return type annotation for public function test_29_embedding_encoding_format
Add return type annotation: None
(ANN201)
975-975: Missing type annotation for function argument openai_client
(ANN001)
975-975: Missing type annotation for function argument test_config
(ANN001)
975-975: Unused method argument: test_config
(ARG002)
1004-1004: Do not catch blind exception: Exception
(BLE001)
1007-1007: Do not catch blind exception: Exception
(BLE001)
1012-1012: Missing return type annotation for public function test_30_embedding_usage_tracking
Add return type annotation: None
(ANN201)
1012-1012: Missing type annotation for function argument openai_client
(ANN001)
1012-1012: Missing type annotation for function argument test_config
(ANN001)
1012-1012: Unused method argument: test_config
(ARG002)
1016-1016: Trailing comma missing
Add trailing comma
(COM812)
1030-1030: Trailing comma missing
Add trailing comma
(COM812)
1034-1034: Trailing comma missing
Add trailing comma
(COM812)
tests/transports-integrations/tests/utils/common.py
1032-1032: Dynamically typed expressions (typing.Any) are disallowed in response
(ANN401)
1032-1032: Trailing comma missing
Add trailing comma
(COM812)
1045-1045: Trailing comma missing
Add trailing comma
(COM812)
1060-1060: Trailing comma missing
Add trailing comma
(COM812)
1079-1079: Avoid specifying long messages outside the exception class
(TRY003)
1083-1083: Dynamically typed expressions (typing.Any) are disallowed in response
(ANN401)
1083-1083: Trailing comma missing
Add trailing comma
(COM812)
1096-1096: Trailing comma missing
Add trailing comma
(COM812)
1101-1101: Trailing comma missing
Add trailing comma
(COM812)
1116-1116: Trailing comma missing
Add trailing comma
(COM812)
1130-1130: Trailing comma missing
Add trailing comma
(COM812)
1143-1145: Avoid specifying long messages outside the exception class
(TRY003)
1144-1144: Trailing comma missing
Add trailing comma
(COM812)
1149-1149: Use list instead of List for type annotation
Replace with list
(UP006)
1149-1149: Use list instead of List for type annotation
Replace with list
(UP006)
1149-1149: Trailing comma missing
Add trailing comma
(COM812)
1171-1171: Use list instead of List for type annotation
Replace with list
(UP006)
1172-1172: Use list instead of List for type annotation
Replace with list
(UP006)
1184-1184: Use list instead of List for type annotation
Replace with list
(UP006)
1184-1184: Use list instead of List for type annotation
Replace with list
(UP006)
1184-1184: Trailing comma missing
Add trailing comma
(COM812)
🪛 LanguageTool
tests/transports-integrations/README.md
[grammar] ~3-~3: Use correct spacing
Context: ...dings, speech synthesis, and multimodal workflows. ## 🌉 Architecture Overview The Bifrost in...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~37-~37: Use correct spacing
Context: ...similarity analysis testing ## 📋 Test Categories Our test suite covers 30 comprehensive s...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~39-~39: There might be a mistake here.
Context: ...ers 30 comprehensive scenarios for each integration: ### Core Chat & Conversation Tests 1. **Sim...
(QB_NEW_EN_OTHER)
[grammar] ~79-~79: There might be a problem here.
Context: ...tion & Error Tests 28. Complex End-to-End - Comprehensive multimodal workflows 29. **Integration-...
(QB_NEW_EN_MERGED_MATCH)
[grammar] ~80-~80: There might be a problem here.
Context: ...al workflows 29. Integration-Specific Features - Integration-unique capabilities 30. **Error Handlin...
(QB_NEW_EN_MERGED_MATCH)
[grammar] ~81-~81: There might be a problem here.
Context: ...gration-unique capabilities 30. Error Handling - Invalid request error processing and propagatio...
(QB_NEW_EN_MERGED_MATCH)
[grammar] ~81-~81: Use correct spacing
Context: ... - Invalid request error processing and propagation ## 📁 Directory Structure ```text transpor...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~668-~668: Use correct spacing
Context: ...scenarios supported (including speech & embeddings) #### Anthropic - ✅ **Full Bifrost Integratio...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
🪛 markdownlint-cli2 (0.17.2)
tests/transports-integrations/README.md
66-66: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
67-67: Ordered list item prefix
Expected: 1; Actual: 18; Style: 1/2/3
(MD029, ol-prefix)
67-67: Lists should be surrounded by blank lines
(MD032, blanks-around-lists)
68-68: Ordered list item prefix
Expected: 2; Actual: 19; Style: 1/2/3
(MD029, ol-prefix)
69-69: Ordered list item prefix
Expected: 3; Actual: 20; Style: 1/2/3
(MD029, ol-prefix)
70-70: Ordered list item prefix
Expected: 4; Actual: 21; Style: 1/2/3
(MD029, ol-prefix)
71-71: Ordered list item prefix
Expected: 5; Actual: 22; Style: 1/2/3
(MD029, ol-prefix)
72-72: Ordered list item prefix
Expected: 6; Actual: 23; Style: 1/2/3
(MD029, ol-prefix)
73-73: Ordered list item prefix
Expected: 7; Actual: 24; Style: 1/2/3
(MD029, ol-prefix)
74-74: Ordered list item prefix
Expected: 8; Actual: 25; Style: 1/2/3
(MD029, ol-prefix)
75-75: Ordered list item prefix
Expected: 9; Actual: 26; Style: 1/2/3
(MD029, ol-prefix)
76-76: Ordered list item prefix
Expected: 10; Actual: 27; Style: 1/2/3
(MD029, ol-prefix)
78-78: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
79-79: Ordered list item prefix
Expected: 1; Actual: 28; Style: 1/2/3
(MD029, ol-prefix)
79-79: Lists should be surrounded by blank lines
(MD032, blanks-around-lists)
80-80: Ordered list item prefix
Expected: 2; Actual: 29; Style: 1/2/3
(MD029, ol-prefix)
81-81: Ordered list item prefix
Expected: 3; Actual: 30; Style: 1/2/3
(MD029, ol-prefix)
🔇 Additional comments (28)
transports/bifrost-http/lib/store.go (1)
1980-1980: Minor log message improvement.The removal of "only" from the log message makes it more concise and consistent with the hybrid configuration loading approach. This improves message clarity.
transports/bifrost-http/integrations/openai/router.go (1)
58-83: Well-structured embeddings endpoint implementation.The embeddings endpoint follows the established patterns in this router with proper:
- Multiple path support (v1 and non-v1 endpoints)
- Type-safe request handling with
OpenAIEmbeddingRequest- Appropriate converters for request, response, and error handling
- No streaming configuration (correct for embeddings)
The implementation is consistent with other endpoints in the file.
transports/bifrost-http/integrations/utils.go (1)
303-304: Clean integration of embedding request handling.The addition properly extends the request routing logic to handle embedding requests, following the established pattern for other input types. The placement in the conditional chain is logical and maintains consistency with the existing codebase.
tests/transports-integrations/README.md (7)
3-3: LGTM!The feature list correctly includes embeddings alongside other supported capabilities.
30-30: LGTM!Test category count correctly updated to 14.
35-35: LGTM!Embeddings support properly documented in the features list.
39-39: LGTM!Total scenario count correctly updated to 30.
66-77: Well-structured embeddings test documentationThe 10 embedding test cases are clearly documented with appropriate descriptions covering all essential embedding functionality.
665-668: LGTM!OpenAI integration details correctly updated to include embedding models and features.
832-861: Helpful test execution examplesThe pytest command examples for embedding tests provide clear guidance for running specific embedding tests.
tests/transports-integrations/config.yml (5)
37-37: LGTM!Default embeddings model
text-embedding-3-smallis a good choice for general use.
47-49: LGTM!Alternative embedding models provide good coverage of OpenAI's embedding offerings.
151-151: Good clarificationExplicitly setting
embeddings: falsefor the whisper-1 model improves configuration clarity.
155-191: Accurate embedding model capabilitiesThe embedding model configurations correctly specify dimensions (1536 for small/ada-002, 3072 for large) and context windows.
252-252: LGTM!Correctly set
embeddings: nullin test settings as embeddings don't use token limits.tests/transports-integrations/tests/integrations/test_openai.py (8)
758-777: Well-structured single embedding testThe test thoroughly validates the embedding response structure including index, object type, and model name.
779-796: Comprehensive batch embedding testThe test properly validates batch responses and ensures correct indexing for each embedding.
798-825: Good similarity validationThe test appropriately uses cosine similarity with a reasonable threshold (0.7) to validate that similar texts produce similar embeddings.
827-863: Excellent dissimilarity testingThe test cleverly validates both dissimilarity between unrelated topics and higher similarity between related topics (programming vs ML).
896-909: Good token usage validationThe test appropriately validates that longer text consumes more tokens and that usage data is properly reported.
939-973: Well-designed dimension reduction testThe test properly validates custom dimension support and compares results with default dimensions.
975-1010: Comprehensive encoding format testThe test appropriately handles different encoding formats with proper fallback for unsupported features.
1012-1057: Excellent usage tracking validationThe test thoroughly validates token consumption for both single and batch embeddings, including proportional usage verification.
transports/bifrost-http/integrations/openai/types.go (4)
55-62: Well-structured embedding request typeThe
OpenAIEmbeddingRequeststruct correctly handles the flexible input type (string or []string) and includes all necessary OpenAI embedding parameters.
79-84: Correct streaming implementationAppropriately returns false as OpenAI embeddings don't support streaming.
97-112: Proper embedding response typesThe response structures correctly match OpenAI's embedding API format with appropriate field types.
438-461: Well-implemented response conversionThe function correctly converts Bifrost embedding responses to OpenAI format with proper nil checking and field mapping.
tests/transports-integrations/tests/utils/common.py (1)
83-117: LGTM! Well-designed embedding test data.The embedding test data constants comprehensively cover various scenarios including single/batch processing, similarity/dissimilarity analysis, empty text edge cases, and long-form content. The data selection is appropriate for validating embedding functionality across different use cases.
Pratham-Mishra04
force-pushed
the
08-05-feat_embeddings_openai_sdk_integration_added
branch
from
f1e9e4d to
80eb8fd
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (7)
tests/transports-integrations/tests/integrations/test_openai.py (3)
92-104: Remove unused importsThe following imports are not used in the code:
EMBEDDINGS_EMPTY_TEXTS(line 97)assert_embeddings_similarity(line 102)assert_embeddings_dissimilarity(line 103)Apply this diff to remove unused imports:
# Embeddings utilities EMBEDDINGS_SINGLE_TEXT, EMBEDDINGS_MULTIPLE_TEXTS, EMBEDDINGS_SIMILAR_TEXTS, EMBEDDINGS_DIFFERENT_TEXTS, - EMBEDDINGS_EMPTY_TEXTS, EMBEDDINGS_LONG_TEXT, assert_valid_embedding_response, assert_valid_embeddings_batch_response, calculate_cosine_similarity, - assert_embeddings_similarity, - assert_embeddings_dissimilarity, )
864-894: Consider more specific exception handlingThe test correctly validates different embedding models and dimensions. However, the broad
Exceptioncatch could mask unexpected errors.Consider catching more specific exceptions:
- except Exception as e: + except (openai.BadRequestError, openai.NotFoundError) as e: # If text-embedding-3-large is not available, just log it print(f"text-embedding-3-large not available: {e}")
910-937: Consider using pytest.raises with match parameterThe error handling tests are comprehensive, but could be improved by using pytest's match parameter for more specific error validation.
For the invalid model test:
- with pytest.raises(Exception) as exc_info: + with pytest.raises(Exception, match="invalid.*model|model.*not.*found") as exc_info: openai_client.embeddings.create( model="invalid-embedding-model", input=EMBEDDINGS_SINGLE_TEXT )tests/transports-integrations/tests/utils/common.py (4)
1031-1080: Address static analysis style issues.The validation logic is comprehensive and correctly handles different response formats. The style improvements needed (trailing commas, type annotations, exception handling) are identical to previous review feedback.
1082-1146: Address static analysis style issues.The batch validation logic is excellent with comprehensive validation of multiple embeddings and proper error messages for each embedding index. The style improvements needed are identical to previous review feedback.
1148-1168: LGTM! Mathematically correct cosine similarity implementation.The cosine similarity calculation is implemented correctly with proper dot product and magnitude calculations, including appropriate zero-division handling. The type annotation style improvements needed are identical to previous review feedback.
1170-1191: LGTM! Well-designed similarity assertion utilities.Both similarity and dissimilarity assertion functions provide valuable testing capabilities with reasonable default thresholds and clear error messages including actual similarity values. The type annotation style improvements needed are identical to previous review feedback.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (8)
tests/transports-integrations/README.md(6 hunks)tests/transports-integrations/config.yml(4 hunks)tests/transports-integrations/tests/integrations/test_openai.py(4 hunks)tests/transports-integrations/tests/utils/common.py(2 hunks)transports/bifrost-http/integrations/openai/router.go(1 hunks)transports/bifrost-http/integrations/openai/types.go(6 hunks)transports/bifrost-http/integrations/utils.go(1 hunks)transports/bifrost-http/lib/store.go(1 hunks)
🧰 Additional context used
🧠 Learnings (51)
📓 Common learnings
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#212
File: transports/bifrost-http/integrations/openai/types.go:250-288
Timestamp: 2025-08-04T22:19:12.965Z
Learning: In transports/bifrost-http/integrations/openai/types.go, the ConvertToBifrostRequest function for OpenAIEmbeddingRequest handles []interface{} input without validation for non-string elements because Pratham-Mishra04 confirmed this case will never happen in practice, consistent with their preference to avoid defensive programming for unreachable scenarios.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/types.go:89-119
Timestamp: 2025-06-10T11:00:02.875Z
Learning: In the Bifrost OpenAI integration (transports/bifrost-http/integrations/openai/types.go), the convertOpenAIContent function currently only handles the last image URL when multiple images are present in a content array. The user Pratham-Mishra04 has acknowledged this limitation and indicated it's part of a larger architectural issue that will be addressed comprehensively later, rather than with piecemeal fixes.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#169
File: docs/usage/http-transport/openapi.json:1352-1358
Timestamp: 2025-07-17T08:56:59.907Z
Learning: In the Bifrost project, the fallback format has been updated from object structure {"provider": "...", "model": "..."} to a simpler string format "provider/model" (e.g., "anthropic/claude-3-sonnet-20240229"). The current OpenAPI schema correctly reflects this new format.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: transports/bifrost-http/main.go:2-2
Timestamp: 2025-06-15T16:05:13.489Z
Learning: For the Bifrost project, HTTP transport integration routers for new providers (like Mistral and Ollama) are implemented in separate PRs from the core provider support, following a focused PR strategy.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/router.go:0-0
Timestamp: 2025-06-09T14:03:34.227Z
Learning: In the Bifrost HTTP transport layer (transports/bifrost-http/integrations/), request validation like checking for empty messages should be handled by the provider rather than at the transport layer. The transport layer should forward requests to Bifrost core/providers for validation.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#54
File: core/schemas/provider.go:148-148
Timestamp: 2025-06-04T03:57:50.981Z
Learning: Breaking changes in the Bifrost codebase are managed by first merging and tagging core schema changes, then updating dependent code references in subsequent steps after the core version is released.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/types.go:223-231
Timestamp: 2025-06-10T11:06:06.670Z
Learning: The OpenAI `name` field on messages cannot be preserved when converting to Bifrost format because the `schemas.BifrostMessage` struct in bifrost/core does not support a Name field. This is a known limitation of the Bifrost core schema design.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:358-388
Timestamp: 2025-06-04T05:37:59.699Z
Learning: User Pratham-Mishra04 prefers not to extract small code duplications (around 2 lines) into helper functions, considering the overhead not worth it for such minor repetition.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#102
File: README.md:62-66
Timestamp: 2025-06-19T17:03:03.639Z
Learning: Pratham-Mishra04 prefers using the implicit 'latest' tag for the maximhq/bifrost Docker image rather than pinning to specific versions.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#143
File: core/mcp.go:155-196
Timestamp: 2025-07-08T15:33:47.698Z
Learning: Pratham-Mishra04 prefers not to add explanatory comments for obvious code patterns, such as the unlock/lock strategy around network I/O operations, considering them self-explanatory to experienced developers.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#141
File: core/bifrost.go:198-272
Timestamp: 2025-07-08T18:30:08.258Z
Learning: Pratham-Mishra04 follows a pattern of implementing core functionality first and deferring non-critical improvements (like race condition fixes, optimizations) to later PRs. This is a reasonable development approach that prioritizes getting the main feature working before addressing edge cases.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#150
File: transports/bifrost-http/lib/store.go:370-466
Timestamp: 2025-07-09T04:58:08.229Z
Learning: Pratham-Mishra04 prefers not to add logging or error handling for unreachable code paths in the Bifrost project. When provider types or similar entities are predefined in the system, defensive programming like logging in default cases is considered unnecessary overhead.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#138
File: docs/usage/go-package/mcp.md:408-412
Timestamp: 2025-07-01T12:40:08.576Z
Learning: Pratham-Mishra04 is okay with keeping bullet list formatting that uses colons after dashes in markdown documentation, even if it triggers linter warnings, preferring functionality over strict formatting rules.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#162
File: tests/core-providers/scenarios/chat_completion_stream.go:103-105
Timestamp: 2025-07-16T04:26:09.288Z
Learning: Pratham-Mishra04 prefers to keep test code simple when it serves its basic functional purpose. For tests that are meant to validate core functionality (like verifying streaming works), they consider hard-coded reasonable limits acceptable rather than making them configurable.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#148
File: transports/bifrost-http/lib/store.go:880-910
Timestamp: 2025-07-08T17:14:21.544Z
Learning: Pratham-Mishra04 prefers resilient system design where missing environment variables for MCP connections should not cause complete system failure. The system should continue processing other MCP connections even when some fail, maintaining partial functionality rather than implementing fail-fast behavior.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#148
File: transports/bifrost-http/lib/store.go:1081-1098
Timestamp: 2025-07-08T17:16:50.811Z
Learning: Pratham-Mishra04 prefers practical redaction approaches over theoretical security improvements when the threat model is low-risk, such as admin-only interfaces in the Bifrost project. Fixed-length redaction is acceptable when only trusted administrators will see the redacted values.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#138
File: transports/README.md:26-28
Timestamp: 2025-07-01T12:45:06.906Z
Learning: Pratham-Mishra04 prefers keeping documentation examples simple and concise, trusting users to handle production-specific considerations like version pinning themselves rather than cluttering examples with additional notes.
📚 Learning: in transports/bifrost-http/integrations/openai/types.go, the converttobifrostrequest function for op...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#212
File: transports/bifrost-http/integrations/openai/types.go:250-288
Timestamp: 2025-08-04T22:19:12.965Z
Learning: In transports/bifrost-http/integrations/openai/types.go, the ConvertToBifrostRequest function for OpenAIEmbeddingRequest handles []interface{} input without validation for non-string elements because Pratham-Mishra04 confirmed this case will never happen in practice, consistent with their preference to avoid defensive programming for unreachable scenarios.
Applied to files:
transports/bifrost-http/integrations/utils.gotests/transports-integrations/README.mdtransports/bifrost-http/integrations/openai/router.gotests/transports-integrations/tests/utils/common.pytransports/bifrost-http/integrations/openai/types.go
📚 Learning: in core/schemas/bifrost.go, the requestinput struct uses chatcompletioninput *[]bifrostmessage (poin...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#54
File: core/schemas/bifrost.go:46-49
Timestamp: 2025-06-04T09:22:18.123Z
Learning: In core/schemas/bifrost.go, the RequestInput struct uses ChatCompletionInput *[]BifrostMessage (pointer-to-slice) rather than []BifrostMessage to properly represent union type semantics. For text completion requests, ChatCompletionInput should be nil to indicate "no chat payload at all", while for chat completion requests it should be non-nil (even if empty slice). This distinguishes between different request types rather than just empty vs non-empty chat messages.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: the generic router in transports/bifrost-http/integrations/utils.go already handles nil pointers fro...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#67
File: transports/bifrost-http/integrations/anthropic/router.go:26-34
Timestamp: 2025-06-10T11:19:29.604Z
Learning: The Generic router in transports/bifrost-http/integrations/utils.go already handles nil pointers from RequestConverter functions. When a RequestConverter returns a nil *schemas.BifrostRequest, the Generic router automatically returns an HTTP 400 error, making additional nil checks in individual router implementations redundant.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/router.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: the generic router in transports/bifrost-http/integrations/utils.go already handles nil pointers fro...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#67
File: transports/bifrost-http/integrations/anthropic/router.go:26-34
Timestamp: 2025-06-10T11:19:29.604Z
Learning: The Generic router in transports/bifrost-http/integrations/utils.go already handles nil pointers from RequestConverter functions. When a RequestConverter returns a nil *schemas.BifrostRequest, the Generic router automatically returns an HTTP 400 error with "Invalid request" message, making additional nil checks in individual router implementations redundant.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/router.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the genericrouter (transports/bifrost-http/integrations), responsefunc is not called if the bifro...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/anthropic/router.go:19-33
Timestamp: 2025-06-09T16:46:32.018Z
Learning: In the GenericRouter (transports/bifrost-http/integrations), ResponseFunc is not called if the BifrostResponse parameter is nil, providing built-in protection against nil response marshaling.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/router.go
📚 Learning: in the bifrost codebase, errors returned from client methods like chatcompletionrequest are of type ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/end_to_end_tool_calling.go:43-45
Timestamp: 2025-06-16T04:13:55.437Z
Learning: In the Bifrost codebase, errors returned from client methods like ChatCompletionRequest are of type BifrostError, not the standard error interface. For testing these errors, use require.Nilf instead of require.NoErrorf since BifrostError doesn't work with the standard error assertion methods.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in transports/bifrost-http/integrations/genai/types.go, when systeminstruction has an empty role, th...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#89
File: transports/bifrost-http/integrations/genai/types.go:151-155
Timestamp: 2025-06-16T14:45:48.563Z
Learning: In transports/bifrost-http/integrations/genai/types.go, when SystemInstruction has an empty role, the user prefers to let the downstream provider (Google GenAI) handle validation and return errors, rather than implementing validation in the bifrost layer. This represents a design preference for delegating validation to the appropriate service rather than duplicating validation logic in the proxy layer.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost http transport layer (transports/bifrost-http/integrations/), request validation like...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/router.go:0-0
Timestamp: 2025-06-09T14:03:34.227Z
Learning: In the Bifrost HTTP transport layer (transports/bifrost-http/integrations/), request validation like checking for empty messages should be handled by the provider rather than at the transport layer. The transport layer should forward requests to Bifrost core/providers for validation.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the google genai integration at transports/bifrost-http/integrations/genai/types.go, the manual u...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#89
File: transports/bifrost-http/integrations/genai/types.go:22-56
Timestamp: 2025-06-16T14:50:46.859Z
Learning: In the Google GenAI integration at transports/bifrost-http/integrations/genai/types.go, the manual URL-safe base64 decoding implementation (converting - to +, _ to /, and adding padding) is required because base64.RawURLEncoding.DecodeString fails for the specific url encoded bytes format being handled.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/router.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in transports/bifrost-http/handlers/completions.go, for speech completion requests, the user prefers...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#177
File: transports/bifrost-http/handlers/completions.go:248-264
Timestamp: 2025-07-22T12:14:08.826Z
Learning: In transports/bifrost-http/handlers/completions.go, for speech completion requests, the user prefers to let the provider handle ResponseFormat validation rather than validating supported audio formats ("mp3", "opus", "aac", "flac") at the HTTP transport layer. This follows the architectural pattern of delegating domain-specific validation to providers rather than duplicating validation logic in the transport layer.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/router.go
📚 Learning: the chatcompletionrequest method in the bifrost client follows a contract where the result parameter...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/utils.go:169-173
Timestamp: 2025-06-09T17:33:52.234Z
Learning: The ChatCompletionRequest method in the Bifrost client follows a contract where the result parameter will never be nil if the error parameter is nil. This means when error checking passes (err == nil), the result is guaranteed to be valid and can be safely used without additional nil checks.
Applied to files:
transports/bifrost-http/integrations/utils.go
📚 Learning: in the bifrost codebase, errors returned from methods like chatcompletionrequest are of type bifrost...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/simple_chat.go:39-41
Timestamp: 2025-06-16T04:13:42.755Z
Learning: In the Bifrost codebase, errors returned from methods like ChatCompletionRequest are of type BifrostError (a custom error type) rather than the standard Go error interface. Therefore, require.Nilf should be used for error assertions instead of require.NoErrorf.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: the `lib.converttobifrostcontext` function in the bifrost http transport never returns nil and handl...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/router.go:62-64
Timestamp: 2025-06-09T11:27:00.925Z
Learning: The `lib.ConvertToBifrostContext` function in the bifrost HTTP transport never returns nil and handles the conversion internally, so nil checks are not needed when calling this function.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost system, error returns are of type `bifrosterror` rather than the standard go `error` ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/complete_end_to_end.go:39-41
Timestamp: 2025-06-16T04:12:05.427Z
Learning: In the Bifrost system, error returns are of type `BifrostError` rather than the standard Go `error` interface. Therefore, use `require.Nilf(t, err, ...)` instead of `require.NoError(t, err)` when checking for errors in Bifrost function calls.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrosterror struct, the error field is a struct (not a pointer), so accessing bifrosterr.err...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#80
File: plugins/maxim/main.go:249-264
Timestamp: 2025-06-14T06:17:54.426Z
Learning: In the BifrostError struct, the Error field is a struct (not a pointer), so accessing bifrostErr.Error.Message, bifrostErr.Error.Code, and bifrostErr.Error.Type is safe without nil checks on the Error field itself. The Code and Type fields are of type *string.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: the `lib.converttobifrostcontext` function in the bifrost http transport never returns nil and alway...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/router.go:62-64
Timestamp: 2025-06-09T11:27:00.925Z
Learning: The `lib.ConvertToBifrostContext` function in the bifrost HTTP transport never returns nil and always returns a pointer to a valid context. It starts with context.Background() and only adds values to it, so nil checks are not needed when calling this function.
Applied to files:
transports/bifrost-http/integrations/utils.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in transports/bifrost-http/lib/store.go, the loadfromconfig method already handles missing config fi...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#149
File: transports/bifrost-http/main.go:259-261
Timestamp: 2025-07-08T18:20:24.086Z
Learning: In transports/bifrost-http/lib/store.go, the LoadFromConfig method already handles missing config files gracefully by initializing defaults and auto-detecting providers from environment variables. It only returns errors for actual parsing/processing failures, not missing files, making log.Fatalf appropriate in main.go since real errors should terminate the program.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost http transport logging plugin (transports/bifrost-http/plugins/logging/main.go), the ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#149
File: transports/bifrost-http/plugins/logging/main.go:0-0
Timestamp: 2025-07-08T18:09:32.147Z
Learning: In the Bifrost HTTP transport logging plugin (transports/bifrost-http/plugins/logging/main.go), the DatabasePath is explicitly set from main.go via a config struct pointing to appDir/logs, so the default value in the NewLoggerPlugin constructor is only used for standalone plugin usage and doesn't affect the main application flow.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost logging plugin (transports/bifrost-http/plugins/logging/utils.go), pratham-mishra04 p...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#152
File: transports/bifrost-http/plugins/logging/utils.go:378-399
Timestamp: 2025-07-10T13:44:14.518Z
Learning: In the Bifrost logging plugin (transports/bifrost-http/plugins/logging/utils.go), Pratham-Mishra04 prefers not to add error handling for JSON unmarshaling operations, considering logging not very critical and being confident that JSON marshalling won't fail in practice.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost logging plugin (transports/bifrost-http/plugins/logging/utils.go), pratham-mishra04 p...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#152
File: transports/bifrost-http/plugins/logging/utils.go:16-18
Timestamp: 2025-07-10T13:44:39.237Z
Learning: In the Bifrost logging plugin (transports/bifrost-http/plugins/logging/utils.go), Pratham-Mishra04 prefers to ignore JSON marshaling errors when storing log entries because logging is not critical for their use case and they are certain the marshaling operations won't fail.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: pratham-mishra04 prefers not to add error handling for json marshaling operations in the bifrost log...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#152
File: transports/bifrost-http/plugins/logging/utils.go:94-111
Timestamp: 2025-07-10T13:44:23.297Z
Learning: Pratham-Mishra04 prefers not to add error handling for JSON marshaling operations in the Bifrost logging plugin (transports/bifrost-http/plugins/logging/utils.go) because logging is not critical functionality and the structured schema data being marshaled is unlikely to fail. They accept the risk of not handling json.Marshal errors in logging contexts to keep the code simple.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost logging system (transports/bifrost-http/plugins/logging/utils.go), the content field ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#144
File: transports/bifrost-http/plugins/logging/utils.go:509-514
Timestamp: 2025-07-08T16:48:25.386Z
Learning: In the Bifrost logging system (transports/bifrost-http/plugins/logging/utils.go), the Content field in message structures is a struct value type, not a pointer, so it will never be nil and doesn't require nil checks. However, ContentStr within Content is a pointer and should be checked for nil.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost project's database models (transports/bifrost-http/lib/models.go), pratham-mishra04 c...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#174
File: transports/bifrost-http/lib/models.go:251-254
Timestamp: 2025-08-02T13:02:13.642Z
Learning: In the Bifrost project's database models (transports/bifrost-http/lib/models.go), Pratham-Mishra04 considers meta config types to be predefined and expects that unknown meta config types will never occur in practice. The current implementation that returns nil for unknown types in the AfterFind hook is intentional, as defensive programming for this scenario is considered unnecessary.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost repository's transport dependency update workflow, when updating the core dependency ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#103
File: .github/workflows/transport-dependency-update.yml:53-75
Timestamp: 2025-06-20T16:21:18.912Z
Learning: In the bifrost repository's transport dependency update workflow, when updating the core dependency to a new version using `go get`, the go.mod and go.sum files will always change in normal operation, making the safety check for changes more of a defensive programming practice rather than handling a common scenario.
Applied to files:
transports/bifrost-http/lib/store.go
📚 Learning: in the bifrost integration tests, the user prefers tests to fail hard when api keys are missing rath...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#82
File: tests/transports-integrations/tests/integrations/test_litellm.py:97-115
Timestamp: 2025-06-16T09:16:15.634Z
Learning: In the Bifrost integration tests, the user prefers tests to fail hard when API keys are missing rather than using @skip_if_no_api_key decorators for graceful skipping. This applies to LiteLLM tests that depend on OpenAI API keys.
Applied to files:
tests/transports-integrations/tests/integrations/test_openai.pytests/transports-integrations/README.md
📚 Learning: in the bifrost openai integration (transports/bifrost-http/integrations/openai/types.go), the conver...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/types.go:89-119
Timestamp: 2025-06-10T11:00:02.875Z
Learning: In the Bifrost OpenAI integration (transports/bifrost-http/integrations/openai/types.go), the convertOpenAIContent function currently only handles the last image URL when multiple images are present in a content array. The user Pratham-Mishra04 has acknowledged this limitation and indicated it's part of a larger architectural issue that will be addressed comprehensively later, rather than with piecemeal fixes.
Applied to files:
tests/transports-integrations/tests/integrations/test_openai.pytests/transports-integrations/README.mdtransports/bifrost-http/integrations/openai/router.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: anthropic's claude api supports the following tool_choice parameter values: "auto" (default), "any" ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:485-498
Timestamp: 2025-06-04T05:13:41.923Z
Learning: Anthropic's Claude API supports the following tool_choice parameter values: "auto" (default), "any" (force use of at least one tool), and {"type": "tool", "name": "tool_name"} (force use of specific tool). Anthropic does NOT support "none" as a tool_choice value - there's no way to disable tool usage once tools are provided in the request.
Applied to files:
tests/transports-integrations/config.yml
📚 Learning: anthropic's claude api supports the following tool_choice parameter values: "auto" (default - claude...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:485-498
Timestamp: 2025-06-04T05:13:41.923Z
Learning: Anthropic's Claude API supports the following tool_choice parameter values: "auto" (default - Claude decides whether to use tools), "none" (disables tool usage entirely), "any" (forces Claude to use at least one tool), and {"type": "tool", "name": "tool_name"} (forces use of a specific tool). All of these values are officially supported by Anthropic's API.
Applied to files:
tests/transports-integrations/config.yml
📚 Learning: in the bifrost project, the fallback format has been updated from object structure {"provider": "......
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#169
File: docs/usage/http-transport/openapi.json:1352-1358
Timestamp: 2025-07-17T08:56:59.907Z
Learning: In the Bifrost project, the fallback format has been updated from object structure {"provider": "...", "model": "..."} to a simpler string format "provider/model" (e.g., "anthropic/claude-3-sonnet-20240229"). The current OpenAPI schema correctly reflects this new format.
Applied to files:
tests/transports-integrations/config.ymltests/transports-integrations/README.mdtransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost project, the tests directory structure has `tests/core-providers/` and `tests/transpo...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#149
File: docs/contributing/README.md:22-27
Timestamp: 2025-07-08T18:12:13.590Z
Learning: In the Bifrost project, the tests directory structure has `tests/core-providers/` and `tests/transports-integrations/` as sibling directories. From `tests/core-providers/`, the correct relative path to reach `tests/transports-integrations/` is `../transports-integrations/`, not `../../tests/transports-integrations/`.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: breaking changes in the bifrost codebase are managed by first merging and tagging core schema change...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#54
File: core/schemas/provider.go:148-148
Timestamp: 2025-06-04T03:57:50.981Z
Learning: Breaking changes in the Bifrost codebase are managed by first merging and tagging core schema changes, then updating dependent code references in subsequent steps after the core version is released.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: in the bifrost test suite (tests/core-providers), parallel tests using t.parallel() are not being im...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/automatic_function_calling.go:22-22
Timestamp: 2025-06-16T04:55:11.886Z
Learning: In the Bifrost test suite (tests/core-providers), parallel tests using t.Parallel() are not being implemented currently. The team plans to add parallel test execution in future enhancements.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: in the bifrost openai integration, tool calls should be allowed on any message role (not just assist...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/types.go:264-285
Timestamp: 2025-06-10T12:58:45.501Z
Learning: In the Bifrost OpenAI integration, tool calls should be allowed on any message role (not just assistant messages) and the downstream provider should handle validation. Users passing tool calls to non-assistant messages is considered deliberate behavior that should be preserved.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: in the bifrost project, environment variables should be used only for secrets (like api keys), not f...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/tests/e2e_tool_test.go:29-30
Timestamp: 2025-06-04T04:58:12.239Z
Learning: In the Bifrost project, environment variables should be used only for secrets (like API keys), not for general configuration. Test parameters like provider and model can be hardcoded at the start of test files for predictability and consistency.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: in anthropic api integration for bifrost, messages won't contain both image and tool_result content ...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/anthropic/types.go:131-180
Timestamp: 2025-06-10T13:11:37.867Z
Learning: In Anthropic API integration for Bifrost, messages won't contain both image and tool_result content blocks in the same message, so defensive guards against multiple embedded message structs are unnecessary in the content processing loop.
Applied to files:
tests/transports-integrations/README.mdtransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost project, the user prefers to use `package main` for test files in the tests/core-prov...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/openai_test.go:1-2
Timestamp: 2025-06-16T04:29:53.409Z
Learning: In the Bifrost project, the user prefers to use `package main` for test files in the tests/core-providers directory rather than more descriptive package names like `coreproviders_test`.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: for the bifrost project, http transport integration routers for new providers (like mistral and olla...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: transports/bifrost-http/main.go:2-2
Timestamp: 2025-06-15T16:05:13.489Z
Learning: For the Bifrost project, HTTP transport integration routers for new providers (like Mistral and Ollama) are implemented in separate PRs from the core provider support, following a focused PR strategy.
Applied to files:
tests/transports-integrations/README.mdtransports/bifrost-http/integrations/openai/router.gotransports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost project, the team prioritizes maintaining consistent error handling patterns over exp...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#94
File: core/bifrost.go:823-845
Timestamp: 2025-06-18T15:15:51.323Z
Learning: In the Bifrost project, the team prioritizes maintaining consistent error handling patterns over exposing detailed error context. All errors should be wrapped in the standard `BifrostError` structure rather than creating specific error types or exposing richer error details like exit codes or stderr output.
Applied to files:
tests/transports-integrations/README.md
📚 Learning: the openai `name` field on messages cannot be preserved when converting to bifrost format because th...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#63
File: transports/bifrost-http/integrations/openai/types.go:223-231
Timestamp: 2025-06-10T11:06:06.670Z
Learning: The OpenAI `name` field on messages cannot be preserved when converting to Bifrost format because the `schemas.BifrostMessage` struct in bifrost/core does not support a Name field. This is a known limitation of the Bifrost core schema design.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: the bifrost codebase uses string alias types (like `type contextkey string`) for context keys consis...
Learnt from: connyay
PR: maximhq/bifrost#92
File: core/providers/utils.go:60-64
Timestamp: 2025-06-17T14:04:21.104Z
Learning: The bifrost codebase uses string alias types (like `type ContextKey string`) for context keys consistently across multiple packages including plugins/maxim/main.go and transports/bifrost-http/tracking/plugin.go. This pattern should be maintained for consistency rather than switching to the canonical struct{} approach.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in core/providers/anthropic.go, the content field in formattedmessages is always of type []interface...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:526-550
Timestamp: 2025-06-04T09:29:46.287Z
Learning: In core/providers/anthropic.go, the content field in formattedMessages is always of type []interface{} because it's explicitly constructed that way upstream in the prepareAnthropicChatRequest function. Defensive type casting for multiple types is not needed since the type is guaranteed by the construction logic.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: for cohere v1 api in core/providers/cohere.go, the tool_choice parameter formatting uses uppercase s...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#83
File: core/providers/cohere.go:327-335
Timestamp: 2025-06-15T13:46:24.869Z
Learning: For Cohere v1 API in core/providers/cohere.go, the tool_choice parameter formatting uses uppercase strings for the "type" field (e.g., "AUTO", "TOOL") and follows a different structure than initially assumed. The current implementation with strings.ToUpper() for the type field is correct for the v1 API.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in core/schemas/bifrost.go, the toolchoice unmarshaljson validation intentionally only checks for em...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#83
File: core/schemas/bifrost.go:186-190
Timestamp: 2025-06-15T14:18:32.703Z
Learning: In core/schemas/bifrost.go, the ToolChoice UnmarshalJSON validation intentionally only checks for empty Type fields and lets providers handle validation of specific tool choice values. This architectural decision keeps schema validation focused on structure while allowing provider-specific semantic validation.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in convertgenerationconfigtoparams method in transports/bifrost-http/integrations/genai/types.go, pr...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:273-313
Timestamp: 2025-06-09T16:35:26.914Z
Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost governance plugin (transports/bifrost-http/plugins/governance/resolver.go), pratham-m...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#202
File: transports/bifrost-http/plugins/governance/resolver.go:68-74
Timestamp: 2025-08-03T20:17:11.876Z
Learning: In the Bifrost governance plugin (transports/bifrost-http/plugins/governance/resolver.go), Pratham-Mishra04 considers nil validation in constructors like NewBudgetResolver unnecessary because the parameters will never be nil in practice. This aligns with their general preference to avoid defensive programming for scenarios they consider unreachable.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost http transport handlers using fasthttp router, path parameters extracted via ctx.user...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#202
File: transports/bifrost-http/handlers/governance.go:220-220
Timestamp: 2025-08-03T20:11:26.945Z
Learning: In the Bifrost HTTP transport handlers using fasthttp router, path parameters extracted via ctx.UserValue() can be safely type-asserted to string without additional checks because the router guarantees their presence and type when routes match. Pratham-Mishra04 confirmed that defensive type assertion checks are unnecessary overhead in this context.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in transports/bifrost-http/handlers/completions.go, pratham-mishra04 intentionally delegates file si...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#208
File: transports/bifrost-http/handlers/completions.go:221-225
Timestamp: 2025-08-04T11:27:45.020Z
Learning: In transports/bifrost-http/handlers/completions.go, Pratham-Mishra04 intentionally delegates file size validation for audio transcription uploads to the downstream API/provider rather than enforcing MaxFileSize limits at the HTTP transport layer. This follows their architectural preference of keeping the transport layer simple and letting providers handle domain-specific validation concerns, even for potential memory exhaustion scenarios.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: pratham-mishra04 prefers to avoid redundant error handling across architectural layers in the bifros...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#162
File: transports/bifrost-http/integrations/genai/types.go:0-0
Timestamp: 2025-07-16T07:13:29.496Z
Learning: Pratham-Mishra04 prefers to avoid redundant error handling across architectural layers in the Bifrost streaming implementation. When error handling (such as timeouts, context cancellation, and JSON marshaling failures) is already handled at the provider level, they prefer not to duplicate this logic at the transport integration layer to keep the code simple and avoid unnecessary complexity.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost governance handler (transports/bifrost-http/handlers/governance.go), pratham-mishra04...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#202
File: transports/bifrost-http/handlers/governance.go:1038-1049
Timestamp: 2025-08-03T20:28:00.857Z
Learning: In the Bifrost governance handler (transports/bifrost-http/handlers/governance.go), Pratham-Mishra04 is comfortable with exposing full error details including err.Error() in production responses for governance API endpoints, prioritizing simplicity over potential information disclosure concerns.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
📚 Learning: in the bifrost governance plugin (transports/bifrost-http/plugins/governance/resolver.go), pratham-m...
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#202
File: transports/bifrost-http/plugins/governance/resolver.go:183-191
Timestamp: 2025-08-03T20:36:21.906Z
Learning: In the Bifrost governance plugin (transports/bifrost-http/plugins/governance/resolver.go), Pratham-Mishra04 considers the current string matching approach for determining rate limit violation decision types (DecisionTokenLimited vs DecisionRequestLimited) adequate and not important enough to refactor with more robust explicit checks, preferring functional simplicity over theoretical robustness improvements.
Applied to files:
transports/bifrost-http/integrations/openai/types.go
🪛 Ruff (0.12.2)
tests/transports-integrations/tests/integrations/test_openai.py
97-97: ..utils.common.EMBEDDINGS_EMPTY_TEXTS imported but unused
Remove unused import
(F401)
102-102: ..utils.common.assert_embeddings_similarity imported but unused
Remove unused import
(F401)
103-103: ..utils.common.assert_embeddings_dissimilarity imported but unused
Remove unused import
(F401)
759-759: Missing return type annotation for public function test_21_single_text_embedding
Add return type annotation: None
(ANN201)
759-759: Missing type annotation for function argument openai_client
(ANN001)
759-759: Missing type annotation for function argument test_config
(ANN001)
759-759: Unused method argument: test_config
(ARG002)
762-762: Trailing comma missing
Add trailing comma
(COM812)
779-779: Missing return type annotation for public function test_22_batch_text_embeddings
Add return type annotation: None
(ANN201)
779-779: Missing type annotation for function argument openai_client
(ANN001)
779-779: Missing type annotation for function argument test_config
(ANN001)
779-779: Unused method argument: test_config
(ARG002)
782-782: Trailing comma missing
Add trailing comma
(COM812)
787-787: Trailing comma missing
Add trailing comma
(COM812)
798-798: Missing return type annotation for public function test_23_embedding_similarity_analysis
Add return type annotation: None
(ANN201)
798-798: Missing type annotation for function argument openai_client
(ANN001)
798-798: Missing type annotation for function argument test_config
(ANN001)
798-798: Unused method argument: test_config
(ARG002)
801-801: Trailing comma missing
Add trailing comma
(COM812)
805-805: Trailing comma missing
Add trailing comma
(COM812)
827-827: Missing return type annotation for public function test_24_embedding_dissimilarity_analysis
Add return type annotation: None
(ANN201)
827-827: Missing type annotation for function argument openai_client
(ANN001)
827-827: Missing type annotation for function argument test_config
(ANN001)
827-827: Unused method argument: test_config
(ARG002)
830-830: Trailing comma missing
Add trailing comma
(COM812)
834-834: Trailing comma missing
Add trailing comma
(COM812)
842-842: Trailing comma missing
Add trailing comma
(COM812)
846-846: Trailing comma missing
Add trailing comma
(COM812)
865-865: Missing return type annotation for public function test_25_embedding_different_models
Add return type annotation: None
(ANN201)
865-865: Missing type annotation for function argument openai_client
(ANN001)
865-865: Missing type annotation for function argument test_config
(ANN001)
865-865: Unused method argument: test_config
(ARG002)
871-871: Trailing comma missing
Add trailing comma
(COM812)
878-878: Trailing comma missing
Add trailing comma
(COM812)
888-888: Trailing comma missing
Add trailing comma
(COM812)
891-891: Do not catch blind exception: Exception
(BLE001)
896-896: Missing return type annotation for public function test_26_embedding_long_text
Add return type annotation: None
(ANN201)
896-896: Missing type annotation for function argument openai_client
(ANN001)
896-896: Missing type annotation for function argument test_config
(ANN001)
896-896: Unused method argument: test_config
(ARG002)
899-899: Trailing comma missing
Add trailing comma
(COM812)
911-911: Missing return type annotation for public function test_27_embedding_error_handling
Add return type annotation: None
(ANN201)
911-911: Missing type annotation for function argument openai_client
(ANN001)
911-911: Missing type annotation for function argument test_config
(ANN001)
911-911: Unused method argument: test_config
(ARG002)
915-915: pytest.raises(Exception) is too broad, set the match parameter or use a more specific exception
(PT011)
917-917: Trailing comma missing
Add trailing comma
(COM812)
926-926: Trailing comma missing
Add trailing comma
(COM812)
932-932: Do not catch blind exception: Exception
(BLE001)
934-936: Found assertion on exception e in except block, use pytest.raises() instead
(PT017)
934-936: Found assertion on exception e in except block, use pytest.raises() instead
(PT017)
939-939: Missing return type annotation for public function test_28_embedding_dimensionality_reduction
Add return type annotation: None
(ANN201)
939-939: Missing type annotation for function argument openai_client
(ANN001)
939-939: Missing type annotation for function argument test_config
(ANN001)
939-939: Unused method argument: test_config
(ARG002)
951-951: Trailing comma missing
Add trailing comma
(COM812)
956-956: Trailing comma missing
Add trailing comma
(COM812)
967-967: Trailing comma missing
Add trailing comma
(COM812)
970-970: Do not catch blind exception: Exception
(BLE001)
975-975: Missing return type annotation for public function test_29_embedding_encoding_format
Add return type annotation: None
(ANN201)
975-975: Missing type annotation for function argument openai_client
(ANN001)
975-975: Missing type annotation for function argument test_config
(ANN001)
975-975: Unused method argument: test_config
(ARG002)
1004-1004: Do not catch blind exception: Exception
(BLE001)
1007-1007: Do not catch blind exception: Exception
(BLE001)
1012-1012: Missing return type annotation for public function test_30_embedding_usage_tracking
Add return type annotation: None
(ANN201)
1012-1012: Missing type annotation for function argument openai_client
(ANN001)
1012-1012: Missing type annotation for function argument test_config
(ANN001)
1012-1012: Unused method argument: test_config
(ARG002)
1016-1016: Trailing comma missing
Add trailing comma
(COM812)
1030-1030: Trailing comma missing
Add trailing comma
(COM812)
1034-1034: Trailing comma missing
Add trailing comma
(COM812)
tests/transports-integrations/tests/utils/common.py
1032-1032: Dynamically typed expressions (typing.Any) are disallowed in response
(ANN401)
1032-1032: Trailing comma missing
Add trailing comma
(COM812)
1045-1045: Trailing comma missing
Add trailing comma
(COM812)
1060-1060: Trailing comma missing
Add trailing comma
(COM812)
1079-1079: Avoid specifying long messages outside the exception class
(TRY003)
1083-1083: Dynamically typed expressions (typing.Any) are disallowed in response
(ANN401)
1083-1083: Trailing comma missing
Add trailing comma
(COM812)
1096-1096: Trailing comma missing
Add trailing comma
(COM812)
1101-1101: Trailing comma missing
Add trailing comma
(COM812)
1116-1116: Trailing comma missing
Add trailing comma
(COM812)
1130-1130: Trailing comma missing
Add trailing comma
(COM812)
1143-1145: Avoid specifying long messages outside the exception class
(TRY003)
1144-1144: Trailing comma missing
Add trailing comma
(COM812)
1149-1149: Use list instead of List for type annotation
Replace with list
(UP006)
1149-1149: Use list instead of List for type annotation
Replace with list
(UP006)
1149-1149: Trailing comma missing
Add trailing comma
(COM812)
1171-1171: Use list instead of List for type annotation
Replace with list
(UP006)
1172-1172: Use list instead of List for type annotation
Replace with list
(UP006)
1184-1184: Use list instead of List for type annotation
Replace with list
(UP006)
1184-1184: Use list instead of List for type annotation
Replace with list
(UP006)
1184-1184: Trailing comma missing
Add trailing comma
(COM812)
🪛 LanguageTool
tests/transports-integrations/README.md
[grammar] ~3-~3: Use correct spacing
Context: ...dings, speech synthesis, and multimodal workflows. ## 🌉 Architecture Overview The Bifrost in...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~37-~37: Use correct spacing
Context: ...similarity analysis testing ## 📋 Test Categories Our test suite covers 30 comprehensive s...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~39-~39: There might be a mistake here.
Context: ...ers 30 comprehensive scenarios for each integration: ### Core Chat & Conversation Tests 1. **Sim...
(QB_NEW_EN_OTHER)
[grammar] ~79-~79: There might be a problem here.
Context: ...tion & Error Tests 28. Complex End-to-End - Comprehensive multimodal workflows 29. **Integration-...
(QB_NEW_EN_MERGED_MATCH)
[grammar] ~80-~80: There might be a problem here.
Context: ...al workflows 29. Integration-Specific Features - Integration-unique capabilities 30. **Error Handlin...
(QB_NEW_EN_MERGED_MATCH)
[grammar] ~81-~81: There might be a problem here.
Context: ...gration-unique capabilities 30. Error Handling - Invalid request error processing and propagatio...
(QB_NEW_EN_MERGED_MATCH)
[grammar] ~81-~81: Use correct spacing
Context: ... - Invalid request error processing and propagation ## 📁 Directory Structure ```text transpor...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~668-~668: Use correct spacing
Context: ...scenarios supported (including speech & embeddings) #### Anthropic - ✅ **Full Bifrost Integratio...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
🪛 markdownlint-cli2 (0.17.2)
tests/transports-integrations/README.md
66-66: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
67-67: Ordered list item prefix
Expected: 1; Actual: 18; Style: 1/2/3
(MD029, ol-prefix)
67-67: Lists should be surrounded by blank lines
(MD032, blanks-around-lists)
68-68: Ordered list item prefix
Expected: 2; Actual: 19; Style: 1/2/3
(MD029, ol-prefix)
69-69: Ordered list item prefix
Expected: 3; Actual: 20; Style: 1/2/3
(MD029, ol-prefix)
70-70: Ordered list item prefix
Expected: 4; Actual: 21; Style: 1/2/3
(MD029, ol-prefix)
71-71: Ordered list item prefix
Expected: 5; Actual: 22; Style: 1/2/3
(MD029, ol-prefix)
72-72: Ordered list item prefix
Expected: 6; Actual: 23; Style: 1/2/3
(MD029, ol-prefix)
73-73: Ordered list item prefix
Expected: 7; Actual: 24; Style: 1/2/3
(MD029, ol-prefix)
74-74: Ordered list item prefix
Expected: 8; Actual: 25; Style: 1/2/3
(MD029, ol-prefix)
75-75: Ordered list item prefix
Expected: 9; Actual: 26; Style: 1/2/3
(MD029, ol-prefix)
76-76: Ordered list item prefix
Expected: 10; Actual: 27; Style: 1/2/3
(MD029, ol-prefix)
78-78: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
79-79: Ordered list item prefix
Expected: 1; Actual: 28; Style: 1/2/3
(MD029, ol-prefix)
79-79: Lists should be surrounded by blank lines
(MD032, blanks-around-lists)
80-80: Ordered list item prefix
Expected: 2; Actual: 29; Style: 1/2/3
(MD029, ol-prefix)
81-81: Ordered list item prefix
Expected: 3; Actual: 30; Style: 1/2/3
(MD029, ol-prefix)
🔇 Additional comments (18)
transports/bifrost-http/lib/store.go (1)
1980-1980: LGTM! Minor log message refinement.The removal of "only" from the log message makes it more concise while maintaining clarity.
transports/bifrost-http/integrations/openai/router.go (1)
58-82: LGTM! Embeddings endpoint follows established patterns.The implementation correctly:
- Supports multiple path variants for compatibility
- Uses proper type assertion and error handling
- Reuses existing error conversion functions
- Omits StreamConfig appropriately (embeddings are not streamed)
- Follows the same structure as other endpoints in the router
tests/transports-integrations/config.yml (2)
37-50: Consistent model key naming – LGTMThe new
embeddingsandembeddings_alternativeskeys mirror the existing speech/transcription pattern and resolve cleanly in code that already looks those up. No issues spotted.
155-190: Nice capability rows – addstreamingfor completenessThe three embedding models set
streaming: falseat lines 160, 172, 184 but omit it for the “small” row (155-167). Add it to keep the matrix homogenous and avoid nil-lookup edge-cases in tests that simply readcapabilities["streaming"]."text-embedding-3-small": @@ embeddings: true + streaming: falseLikely an incorrect or invalid review comment.
tests/transports-integrations/tests/integrations/test_openai.py (8)
758-777: LGTM!The test correctly validates single text embedding generation, including response structure verification and model name checks.
778-796: LGTM!The test properly validates batch text embeddings, ensuring correct count and index assignment for each embedding.
797-825: LGTM!The test correctly validates embedding similarity for semantically related texts using cosine similarity with appropriate thresholds.
826-863: LGTM!Excellent test design that validates both dissimilarity between unrelated topics and relative similarity between related topics (programming and ML).
895-909: LGTM!The test properly validates embedding generation for longer text and verifies that token usage is correctly reported.
938-973: LGTM!The test correctly validates custom embedding dimensions when supported and gracefully handles cases where the feature is not available.
974-1010: LGTM!The test properly validates different encoding formats with appropriate error handling for potentially unsupported features.
1011-1057: LGTM!Excellent test that validates token usage tracking and proportional scaling between single and batch embeddings with reasonable variance bounds.
transports/bifrost-http/integrations/openai/types.go (5)
55-83: LGTM!The
OpenAIEmbeddingRequeststruct is well-designed with appropriate fields, and theIsStreamingRequestedmethod correctly returns false since embeddings don't support streaming in the OpenAI API.
97-112: LGTM!The response structures correctly match OpenAI's embedding API format with appropriate field types.
250-288: LGTM!The conversion function correctly handles multiple input types and properly constructs the Bifrost embedding request.
378-396: LGTM!The
convertEmbeddingParametersmethod correctly maps embedding-specific parameters without duplication.
435-458: LGTM!The
DeriveOpenAIEmbeddingFromBifrostResponsefunction correctly converts Bifrost embedding responses to OpenAI format with proper null checks and data transformation.tests/transports-integrations/tests/utils/common.py (1)
83-117: Well-structured embedding test data.The embedding test constants provide excellent coverage for various testing scenarios - single text, batch processing, similarity/dissimilarity analysis, edge cases with empty inputs, and long text handling. The similar texts are semantically related (weather topics) while different texts span diverse domains, making them ideal for embedding validation.
akshaydeo
deleted the
08-05-feat_embeddings_openai_sdk_integration_added
branch
# Add Embeddings Support to Bifrost Integration Tests This PR adds comprehensive embeddings support to the Bifrost integration test suite, expanding the test coverage to include text-to-vector conversion and similarity analysis. The implementation includes: - 10 new embedding test cases for OpenAI, covering single text embedding, batch processing, similarity analysis, dimensionality reduction, and error handling - Updated configuration to support embedding models like `text-embedding-3-small` and `text-embedding-3-large` - New utility functions for embedding validation and similarity calculations - Backend support for embedding requests in the Bifrost HTTP transport The test suite now covers 30 comprehensive scenarios (up from 21), with embeddings joining chat, tool calling, vision, and speech as fully supported capabilities. The README has been updated to reflect these new capabilities and test categories. This enhancement allows for thorough testing of vector embedding functionality, which is critical for semantic search, clustering, and recommendation systems built on the Bifrost proxy.
2 participants
Add Embeddings Support to Bifrost Integration Tests
This PR adds comprehensive embeddings support to the Bifrost integration test suite, expanding the test coverage to include text-to-vector conversion and similarity analysis. The implementation includes:
text-embedding-3-smallandtext-embedding-3-largeThe test suite now covers 30 comprehensive scenarios (up from 21), with embeddings joining chat, tool calling, vision, and speech as fully supported capabilities. The README has been updated to reflect these new capabilities and test categories.
This enhancement allows for thorough testing of vector embedding functionality, which is critical for semantic search, clustering, and recommendation systems built on the Bifrost proxy.