Skip to content

test: add realtime WebSocket and client secret integration tests#2343

Merged
akshaydeo merged 7 commits intov1.5.0from
feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests
Apr 7, 2026
Merged

test: add realtime WebSocket and client secret integration tests#2343
akshaydeo merged 7 commits intov1.5.0from
feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests

Conversation

@danpiths
Copy link
Copy Markdown
Collaborator

Summary

Adds Python integration tests for the Realtime API, covering WebSocket
connections, client secret HTTP endpoints, OpenAI SDK compatibility, and error
handling for unsupported providers.

Changes

  • Added run_ws_realtime_test utility: establishes WebSocket connection, runs
    full conversation flow (session.created → session.update →
    conversation.item.create → response.create → response.text.delta →
    response.done), returns structured result
  • Added run_realtime_client_secret_request utility: HTTP POST to client secret
    endpoints with OpenAI-Beta: realtime=v1 header
  • Added run_openai_base_url_client_secret_request utility: tests SDK
    constructor base_url override compatibility
  • Added get_realtime_test_model helper: reads provider-specific model from env
    vars
  • Test: Realtime WebSocket via base path /v1/realtime
  • Test: Realtime WebSocket via integration paths (/openai/v1/realtime,
    /openai/realtime)
  • Test: Client secret creation across 4 route variations
    (/v1/realtime/client_secrets, /v1/realtime/sessions, OpenAI integration
    paths)
  • Test: OpenAI SDK base_url compatibility for client secret creation
  • Test: Unsupported provider (Anthropic) returns HTTP 400 error

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (Next.js)
  • Docs

How to test

# Requires running Bifrost instance with OpenAI API key configured
cd tests/integrations/python
OPENAI_REALTIME_MODEL=gpt-4o-realtime-preview pytest tests/test_openai.py -k "realtime" -v

Screenshots/Recordings

N/A

Breaking changes

  • Yes
  • No

Related issues

N/A

Security considerations

  • Tests use environment-variable API keys, no hardcoded secrets
  • Virtual key tests respect existing VK auth patterns

Checklist

  • I read docs/contributing/README.md and followed the guidelines
  • I added/updated tests where appropriate
  • I updated documentation where needed
  • I verified builds succeed (Go and UI)
  • I verified the CI pipeline passes locally if applicable

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 27, 2026

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0b8593fe-94ce-4225-b332-258aa44e4b10

📥 Commits

Reviewing files that changed from the base of the PR and between 0a7f3ae and 7b591db.

⛔ Files ignored due to path filters (1)
  • ui/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (17)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py
  • ui/app/workspace/dashboard/page.tsx
  • ui/app/workspace/logs/page.tsx
  • ui/app/workspace/logs/sheets/logDetailView.tsx
  • ui/app/workspace/logs/sheets/logDetailsSheet.tsx
  • ui/app/workspace/logs/sheets/sessionDetailsSheet.tsx
  • ui/app/workspace/logs/views/columns.test.ts
  • ui/app/workspace/logs/views/columns.tsx
  • ui/app/workspace/logs/views/filters.tsx
  • ui/app/workspace/logs/views/logsTable.tsx
  • ui/app/workspace/providers/fragments/openaiConfigFormFragment.tsx
  • ui/components/filters/filterPopover.tsx
  • ui/lib/constants/logs.test.ts
  • ui/lib/constants/logs.ts
  • ui/lib/store/apis/logsApi.ts
  • ui/lib/types/logs.ts
 _________________________________
< Pvt. Rabbit reporting for duty! >
 ---------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
📝 Walkthrough

Walkthrough

Adds Realtime integration tests and test helpers, URL-encodes Realtime WS parameters, and introduces session-oriented logging UI (session sheet, log detail view), log API endpoints/types, message/column rendering for realtime turns, and filter/popover changes to support session-based filtering.

Changes

Cohort / File(s) Summary
Realtime integration tests
tests/integrations/python/tests/test_openai.py
Adds WebSocket and HTTP client-secret integration tests for Realtime routes (multiple path variants), unsupported-provider negative test, and ensures x-bf-vk header handling and encoded model/deployment query values.
Realtime test helpers & socket timeouts
tests/integrations/python/tests/utils/common.py
Adds get_realtime_test_model, run_ws_realtime_test, run_realtime_client_secret_request, run_openai_base_url_client_secret_request; changes socket loop to deadline-based per-receive conn.settimeout(remaining).
Logs UI — session & detail surfaces
ui/app/workspace/logs/sheets/sessionDetailsSheet.tsx, ui/app/workspace/logs/sheets/logDetailsSheet.tsx, ui/app/workspace/logs/sheets/logDetailView.tsx
Adds SessionDetailsSheet (session summary + paginated session logs, live websocket subscription), refactors LogDetailSheet to delegate rendering to new LogDetailView, and adds session view/filter callbacks.
Logs table/columns & message rendering
ui/app/workspace/logs/views/columns.tsx, ui/app/workspace/logs/views/columns.test.ts, ui/app/workspace/logs/views/logsTable.tsx, ui/app/workspace/logs/views/filters.tsx
Introduces LogMessageCell, realtime.turn message extraction/summaries and tests; removes dynamic metadata columns and metadataKeys prop; expands filter handler to accept a single string.
Logs page integration
ui/app/workspace/logs/page.tsx, ui/app/workspace/logs/sheets/logDetailView.tsx
Adds session view support and session-based filtering state/handlers; wires new callbacks into log detail sheet/view.
Filter popover & dashboard
ui/components/filters/filterPopover.tsx, ui/app/workspace/dashboard/page.tsx
Expands onFilterChange to accept `string[]
Logs API, types & constants
ui/lib/store/apis/logsApi.ts, ui/lib/types/logs.ts, ui/lib/constants/logs.ts, ui/lib/constants/logs.test.ts
Adds parent_request_id filter support in params; new endpoints getLogSessionById and getLogSessionSummaryById with hooks; extends types for session responses and LogEntry.parent_request_id; adds realtime.turn request type and tests.
Provider fragment minor changes
ui/app/workspace/providers/fragments/openaiConfigFormFragment.tsx
Small import adjustments (type-only import, relative utils) and formatting of updateProvider invocation.
Misc tests & formatting
tests/... (helpers & added tests)
Adds tests for new message behavior and logs constants; fixes trailing newline in helper file.

Sequence Diagram(s)

sequenceDiagram
    participant User as User (UI)
    participant Sheet as SessionDetailsSheet
    participant API as logsApi (REST)
    participant WS as WebSocket (log events)

    User->>Sheet: Open session sheet (sessionId, liveEnabled)
    Sheet->>API: GET /logs/sessions/:id/summary
    API-->>Sheet: session summary (count, cost, tokens)
    Sheet->>API: GET /logs/sessions/:id (limit, offset, order)
    API-->>Sheet: page of logs
    alt liveEnabled
        Sheet->>WS: subscribe to "log" events
        WS-->>Sheet: log event (parent_request_id matches)
        Sheet->>Sheet: merge/update sessionLogs, update UI counts
    end
    User->>Sheet: Click highlighted log
    Sheet-->>User: open LogDetailView for that log
Loading

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Possibly related PRs

Suggested reviewers

  • akshaydeo

Poem

🐇 I hopped into tests at break of day,

Encoded queries led the way,
Sessions hummed and deltas sprung,
Logs and sheets sang, webhooks sung,
The burrow's CI passed hooray!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely describes the main change: adding realtime WebSocket and client secret integration tests.
Description check ✅ Passed The description is comprehensive and follows the template structure with all major sections completed, including summary, changes, type, affected areas, testing instructions, and security considerations.
Docstring Coverage ✅ Passed Docstring coverage is 91.67% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 27, 2026

Confidence Score: 4/5

Safe to merge; one P2 session.update field mismatch does not block the endpoints under test from functioning correctly

All previously flagged P1 issues are resolved. The remaining finding is P2 — the wrong field name in session.update means text-only modality is not enforced, but tests likely still pass because OpenAI defaults to text+audio and emits response.output_text.delta regardless

tests/integrations/python/tests/utils/common.py — session.update field name correction

Important Files Changed

Filename Overview
tests/integrations/python/tests/test_openai.py Adds tests 62–66 for Realtime WebSocket and client-secret HTTP endpoints; previously flagged skip-guard and response-shape issues are resolved
tests/integrations/python/tests/utils/common.py Adds run_ws_realtime_test, run_realtime_client_secret_request, run_openai_base_url_client_secret_request utilities; session.update sends unrecognised field output_modalities instead of modalities

Reviews (17): Last reviewed commit: "test: add realtime WebSocket and client ..." | Re-trigger Greptile

Comment thread tests/integrations/python/tests/test_openai.py
Comment thread tests/integrations/python/tests/utils/common.py Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
tests/integrations/python/tests/utils/common.py (1)

3540-3551: Use a context manager to properly close the OpenAI SDK client.

The current code instantiates the client without closing it. Use a context manager or explicitly call .close() to properly release underlying HTTP resources.

Suggested refactor
-    client = OpenAI(
-        api_key=api_key,
-        base_url=base_url,
-        timeout=timeout,
-        default_headers=merged_headers,
-    )
-
-    response = client.post(
-        "v1/realtime/client_secrets",
-        cast_to=httpx.Response,
-        body=request_body,
-    )
+    with OpenAI(
+        api_key=api_key,
+        base_url=base_url,
+        timeout=timeout,
+        default_headers=merged_headers,
+    ) as client:
+        response = client.post(
+            "v1/realtime/client_secrets",
+            cast_to=httpx.Response,
+            body=request_body,
+        )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integrations/python/tests/utils/common.py` around lines 3540 - 3551,
The OpenAI client (instantiated as client = OpenAI(...)) is not closed after
use; modify the test to either instantiate the client with a context manager
(e.g., using "with OpenAI(...) as client:") or call client.close() after the
post call so underlying HTTP resources are released; ensure the post call to
"v1/realtime/client_secrets" (client.post) remains inside the context or before
the explicit client.close() to avoid using a closed client.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integrations/python/tests/test_openai.py`:
- Around line 4375-4378: Add the missing third compatibility route to the
integration_urls list so the tests exercise the double-prefixed path; update the
list in tests/integrations/python/tests/test_openai.py (where integration_urls,
ws_base and realtime_model are defined) to include a third entry like
f"{ws_base}/openai/openai/realtime?deployment={quote(realtime_model, safe='')}"
(or use ?model=... if your other tests expect model instead of deployment) so
the /openai/openai/realtime route is covered alongside /openai/v1/realtime and
/openai/realtime.
- Around line 4521-4528: The test
test_66_realtime_client_secret_unsupported_provider directly calls
get_api_key("openai") causing failures when OpenAI creds are absent; modify the
test to early-skip when no OpenAI key is available by using the same guard used
in the standalone OpenAI tests (check get_api_key("openai") and call pytest.skip
or return if it is None/empty) so the negative-path assertion for unsupported
providers still runs only in environments with credentials.

In `@tests/integrations/python/tests/utils/common.py`:
- Around line 3407-3416: The loop currently stops after a fixed 10-event cap
which can cause flaky failures; replace the for _ in range(10) loop with a
time-based wait (e.g., compute deadline = time.time() + timeout_seconds, default
~30s) and loop while time.time() < deadline, calling recv_event() until you see
event_type == "session.created" (set got_session_created) or an "error" event
(set error) and break; after the loop, if neither was received, raise/return a
timeout error indicating the handshake timed out. Update references to
recv_event(), got_session_created, and error in
tests/integrations/python/tests/utils/common.py accordingly.

---

Nitpick comments:
In `@tests/integrations/python/tests/utils/common.py`:
- Around line 3540-3551: The OpenAI client (instantiated as client =
OpenAI(...)) is not closed after use; modify the test to either instantiate the
client with a context manager (e.g., using "with OpenAI(...) as client:") or
call client.close() after the post call so underlying HTTP resources are
released; ensure the post call to "v1/realtime/client_secrets" (client.post)
remains inside the context or before the explicit client.close() to avoid using
a closed client.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 66c81d5a-4180-4cdf-aa42-7d0252ae653e

📥 Commits

Reviewing files that changed from the base of the PR and between cc0cadd and 32475da.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py

Comment thread tests/integrations/python/tests/test_openai.py
Comment thread tests/integrations/python/tests/test_openai.py
Comment thread tests/integrations/python/tests/utils/common.py Outdated
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from cc0cadd to ecd583b Compare March 30, 2026 10:24
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 32475da to 8c028c2 Compare March 30, 2026 10:24
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/integrations/python/tests/test_openai.py (1)

4301-4306: Avoid piggybacking the realtime suite on the simple_chat scenario matrix.

These tests already hard-gate on OPENAI_REALTIME_MODEL, but the current parametrization adds an unrelated dependency on the standard chat scenario. In a realtime-only setup they collapse to _no_providers_ and silently skip even though the realtime model is configured. A small OpenAI/VK-specific parametrization helper would make this suite self-contained.

Also applies to: 4347-4352, 4395-4400, 4465-4469

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integrations/python/tests/test_openai.py` around lines 4301 - 4306, The
test parametrization is incorrectly reusing
get_cross_provider_params_with_vk_for_scenario("simple_chat",
include_providers=["openai"]) which ties the realtime tests to the standard chat
scenario and causes silent skips; create and use a small OpenAI/VK-specific
parametrization helper (e.g., get_openai_vk_params_for_realtime or similar) that
returns params independent of the "simple_chat" matrix, update the
`@pytest.mark.parametrize` calls that currently reference
get_cross_provider_params_with_vk_for_scenario to use this new helper (affecting
the parametrize usages around the current test and the other occurrences you
noted), and keep the existing OPENAI_REALTIME_MODEL gating logic intact so the
suite is self-contained for realtime-only setups.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integrations/python/tests/test_openai.py`:
- Around line 4456-4463: The two assertions for the client_secrets response in
tests/integrations/python/tests/test_openai.py disagree on the JSON shape (one
expects top-level value/expires_at, the other expects nested
client_secret.value/client_secret.expires_at); pick one canonical shape for the
/v1/realtime/client_secrets endpoints and make both assertion sites use the same
structure (update the block that checks result["body"] and the block gated by
response_shape == "sessions" so they assert the identical keys and nesting for
result["body"] and client_secret), ensuring you refer to result["body"],
response_shape, and url when aligning the checks.

---

Nitpick comments:
In `@tests/integrations/python/tests/test_openai.py`:
- Around line 4301-4306: The test parametrization is incorrectly reusing
get_cross_provider_params_with_vk_for_scenario("simple_chat",
include_providers=["openai"]) which ties the realtime tests to the standard chat
scenario and causes silent skips; create and use a small OpenAI/VK-specific
parametrization helper (e.g., get_openai_vk_params_for_realtime or similar) that
returns params independent of the "simple_chat" matrix, update the
`@pytest.mark.parametrize` calls that currently reference
get_cross_provider_params_with_vk_for_scenario to use this new helper (affecting
the parametrize usages around the current test and the other occurrences you
noted), and keep the existing OPENAI_REALTIME_MODEL gating logic intact so the
suite is self-contained for realtime-only setups.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 256b0c8f-5220-4422-97b5-be2d3ea94644

📥 Commits

Reviewing files that changed from the base of the PR and between 32475da and 8c028c2.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/integrations/python/tests/utils/common.py

Comment thread tests/integrations/python/tests/test_openai.py
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 8c028c2 to 88db2f3 Compare March 30, 2026 12:49
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from ecd583b to 839db86 Compare March 30, 2026 12:49
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from 839db86 to c28e16f Compare March 30, 2026 14:50
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 88db2f3 to 6d5d8d1 Compare March 30, 2026 14:50
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tests/integrations/python/tests/utils/common.py (1)

3377-3384: Send the Realtime beta header on WebSocket connects.

The HTTP helpers below always send OpenAI-Beta: realtime=v1, but this helper omits it. That leaves the new WS path under-testing the same compatibility contract this PR is adding for the HTTP realtime helpers.

💡 Proposed change
     headers = {}
     if api_key:
         headers["Authorization"] = f"Bearer {api_key}"
     if extra_headers:
         headers.update(extra_headers)
+    headers.setdefault("OpenAI-Beta", "realtime=v1")
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integrations/python/tests/utils/common.py` around lines 3377 - 3384,
The WebSocket connect helper currently builds headers (headers/header_list) but
doesn't include the Realtime beta header; update the header construction before
calling ws_client.create_connection (the code that builds headers, header_list
and calls create_connection) to ensure "OpenAI-Beta": "realtime=v1" is set when
not already present (merge it into headers or header_list so the ws connect
sends the same realtime=v1 beta header as the HTTP helpers).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integrations/python/tests/utils/common.py`:
- Around line 3538-3559: The OpenAI client created in this helper is never
closed, leaking httpx connection pools; wrap the use of OpenAI(...) and
client.post(...) in a try/finally and call the client's close method in the
finally block (e.g., client.close() or await client.aclose() depending on
sync/async usage) so the connection pool is released; update the block around
OpenAI, client.post, response.json handling to ensure client is always closed
even on JSON parsing errors.

---

Nitpick comments:
In `@tests/integrations/python/tests/utils/common.py`:
- Around line 3377-3384: The WebSocket connect helper currently builds headers
(headers/header_list) but doesn't include the Realtime beta header; update the
header construction before calling ws_client.create_connection (the code that
builds headers, header_list and calls create_connection) to ensure
"OpenAI-Beta": "realtime=v1" is set when not already present (merge it into
headers or header_list so the ws connect sends the same realtime=v1 beta header
as the HTTP helpers).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 22f05548-0d2e-40a7-b93c-76cfd48758b1

📥 Commits

Reviewing files that changed from the base of the PR and between 88db2f3 and 6d5d8d1.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/integrations/python/tests/test_openai.py

Comment thread tests/integrations/python/tests/utils/common.py Outdated
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from c28e16f to bd6ac46 Compare March 30, 2026 17:00
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 6d5d8d1 to 99b22d0 Compare March 30, 2026 17:00
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integrations/python/tests/utils/common.py`:
- Around line 3500-3503: The default OpenAI-Beta header is set incorrectly;
update the headers dict (variable name "headers") in
tests/integrations/python/tests/utils/common.py so that "OpenAI-Beta" defaults
to None (not "realtime=v1"), and make the code that builds request headers skip
adding the header when its value is None so GA requests do not send the
deprecated header; locate any helper that assembles headers from the "headers"
dict and ensure it omits keys with None values.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 02e21628-0236-45ff-81f1-313f6df46986

📥 Commits

Reviewing files that changed from the base of the PR and between 6d5d8d1 and 99b22d0.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py

Comment thread tests/integrations/python/tests/utils/common.py
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from bd6ac46 to 9f9abc4 Compare March 31, 2026 06:16
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 99b22d0 to adc8638 Compare March 31, 2026 06:16
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integrations/python/tests/utils/common.py`:
- Around line 3395-3405: recv_event and run_ws_responses_test check the clock
before calling conn.recv() but don't bound each recv call, so a slow/blocked
recv can repeatedly block up to the socket timeout; fix by computing a real
deadline (deadline = start_time + timeout), then before each conn.recv() compute
remaining = deadline - time.monotonic(), raise TimeoutError if remaining <= 0,
and pass that remaining time into the recv call (or set the socket recv timeout
to remaining) so each recv is limited by the overall deadline rather than the
full socket timeout.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f98a30d2-b44c-4310-af63-92b4e9bdda8b

📥 Commits

Reviewing files that changed from the base of the PR and between 99b22d0 and adc8638.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py

Comment thread tests/integrations/python/tests/utils/common.py Outdated
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from 9f9abc4 to e0198ef Compare March 31, 2026 08:00
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from adc8638 to 6861108 Compare March 31, 2026 08:00
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 7d8cd01 to d2b591c Compare April 7, 2026 07:45
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from 37d588c to d30e179 Compare April 7, 2026 07:45
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/integrations/python/tests/test_openai.py (1)

4514-4520: Consider asserting expires_at for completeness.

The test validates client_secret.value but omits expires_at, which test_64 checks for the same endpoint type. Adding this assertion would ensure the full response contract is validated.

➕ Proposed enhancement
         assert result["body"]["client_secret"]["value"], (
             f"Missing client_secret.value for base_url={base_url}"
         )
+        assert result["body"]["client_secret"]["expires_at"], (
+            f"Missing client_secret.expires_at for base_url={base_url}"
+        )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integrations/python/tests/test_openai.py` around lines 4514 - 4520, The
test in tests/integrations/python/tests/test_openai.py currently asserts
result["body"]["client_secret"]["value"] but omits checking the expiry; update
the assertion to also verify result["body"]["client_secret"]["expires_at"] is
present (non-empty/null) for the same base_url case so the response contract is
fully validated—locate the assertion block using the variables result and
base_url and add an analogous assert for client_secret["expires_at"] (or a type
check if preferred).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/integrations/python/tests/test_openai.py`:
- Around line 4514-4520: The test in
tests/integrations/python/tests/test_openai.py currently asserts
result["body"]["client_secret"]["value"] but omits checking the expiry; update
the assertion to also verify result["body"]["client_secret"]["expires_at"] is
present (non-empty/null) for the same base_url case so the response contract is
fully validated—locate the assertion block using the variables result and
base_url and add an analogous assert for client_secret["expires_at"] (or a type
check if preferred).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bb1368dd-f15a-43dd-8884-f870e4757c4e

📥 Commits

Reviewing files that changed from the base of the PR and between 7d8cd01 and d2b591c.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py

@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from d30e179 to 8448b2e Compare April 7, 2026 09:09
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from d2b591c to 589c406 Compare April 7, 2026 09:09
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
tests/integrations/python/tests/test_openai.py (1)

4505-4520: ⚠️ Potential issue | 🟠 Major

test_65 is asserting the /client_secrets body as if it were /sessions.

run_openai_base_url_client_secret_request() posts to v1/realtime/client_secrets, but this test still expects a nested body["client_secret"]["value"]. In the same file, test_64 already treats .../client_secrets as the top-level value / expires_at shape and reserves the nested object for .../sessions, so the SDK compatibility check is validating the wrong contract for the endpoint it actually calls. (raw.githubusercontent.com)

🩹 Minimal fix
-            assert result["body"]["client_secret"]["value"], (
-                f"Missing client_secret.value for base_url={base_url}"
-            )
+            assert result["body"]["value"], (
+                f"Missing value for base_url={base_url}"
+            )
+            assert result["body"]["expires_at"], (
+                f"Missing expires_at for base_url={base_url}"
+            )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/integrations/python/tests/test_openai.py` around lines 4505 - 4520,
test_65 is validating the wrong response shape for the
v1/realtime/client_secrets endpoint: run_openai_base_url_client_secret_request()
returns a top-level object (value, expires_at) not a nested client_secret object
as used by sessions; update the assertions in test_65 to mirror test_64 by
checking result["body"]["value"] and result["body"]["expires_at"] (and adjust
the error message text accordingly), rather than asserting
result["body"]["client_secret"]["value"].
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integrations/python/tests/test_openai.py`:
- Around line 4313-4345: The realtime tests still use the deprecated OpenAI
model name and old event type; update the shared realtime helper so OpenAI's
default realtime model is "gpt-realtime-1.5" (replace "gpt-4o-realtime-preview")
and change any checks for the streaming event "response.text.delta" to
"response.output_text.delta" (this affects the helper used by
get_realtime_test_model / run_ws_realtime_test and any assertions that inspect
event types in the test result). Ensure the extra_headers/vk logic and ws_url
construction remain unchanged and only the default model string and event-type
string are updated wherever they appear.

---

Duplicate comments:
In `@tests/integrations/python/tests/test_openai.py`:
- Around line 4505-4520: test_65 is validating the wrong response shape for the
v1/realtime/client_secrets endpoint: run_openai_base_url_client_secret_request()
returns a top-level object (value, expires_at) not a nested client_secret object
as used by sessions; update the assertions in test_65 to mirror test_64 by
checking result["body"]["value"] and result["body"]["expires_at"] (and adjust
the error message text accordingly), rather than asserting
result["body"]["client_secret"]["value"].
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9f7c5212-3bbf-4246-bc16-37731f759db4

📥 Commits

Reviewing files that changed from the base of the PR and between d2b591c and 589c406.

📒 Files selected for processing (2)
  • tests/integrations/python/tests/test_openai.py
  • tests/integrations/python/tests/utils/common.py

Comment thread tests/integrations/python/tests/test_openai.py
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 589c406 to bab6a32 Compare April 7, 2026 10:59
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from 8448b2e to 67cde0e Compare April 7, 2026 10:59
Comment thread tests/integrations/python/tests/test_openai.py Outdated
coderabbitai[bot]
coderabbitai Bot previously approved these changes Apr 7, 2026
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from bab6a32 to 0a7f3ae Compare April 7, 2026 12:33
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from 67cde0e to 3cb15fa Compare April 7, 2026 12:33
Comment thread tests/integrations/python/tests/utils/common.py
@danpiths danpiths force-pushed the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch from 0a7f3ae to 7b591db Compare April 7, 2026 14:55
@danpiths danpiths force-pushed the feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui branch from 3cb15fa to 7799ba5 Compare April 7, 2026 14:55
Copy link
Copy Markdown
Contributor

akshaydeo commented Apr 7, 2026

Merge activity

  • Apr 7, 2:56 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Apr 7, 3:03 PM UTC: @akshaydeo merged this pull request with Graphite.

@akshaydeo akshaydeo changed the base branch from feature/03-27-feat_add_realtime_session_browsing_and_log_filtering_ui to graphite-base/2343 April 7, 2026 15:02
@akshaydeo akshaydeo changed the base branch from graphite-base/2343 to v1.5.0 April 7, 2026 15:03
@akshaydeo akshaydeo dismissed coderabbitai[bot]’s stale review April 7, 2026 15:03

The base branch was changed.

@akshaydeo akshaydeo merged commit 2f6ec55 into v1.5.0 Apr 7, 2026
7 of 17 checks passed
@akshaydeo akshaydeo deleted the feature/03-27-test_add_realtime_websocket_and_client_secret_integration_tests branch April 7, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants