Repository navigation
fix: run the monitor load test through the real traces client - #4364
Closed
kcoopermiller wants to merge 1 commit into
Closed
kcoopermiller wants to merge 1 commit into
kcoopermiller wants to merge 1 commit into
Conversation
The load test's fake replaced TracesAPIClient and gzipped at level 9 itself, so it could not see the client's gzip level or the rest of its upload path. Serve the fake endpoint behind the real client through an httpx mock transport. Pin prime-traces to the commit with PrimeIntellect-ai/prime#1129 (gzip level 1). prime-runs stays where it is: deps/verifiers pins the same commit, and the gzip change is in prime-traces alone. prime-1000 stays an expected failure. Every episode now arrives, but log() still blocks for about 3 s while the upload queue is full. The TODO described an uploader that sends one episode at a time; it takes everything queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
Closing in favor of #4363, which moves the SDK to its PyPI releases and makes this load test respond to the gzip level. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TracesClient, with a fake endpoint behind anhttpx.MockTransport. The old fake replaced the API client and gzipped at level 9 itself, so the test couldn't see the client's gzip level.prime-tracesmoves to the commit with fix: gzip trace uploads at level 1 prime#1129 (gzip level 1).prime-runsstays on its commit:deps/verifierspins the same one, and Change default micro batch size to 1 #1129 only changesprime-traces.prime-1000stays a strict expected failure, and its TODO now describes the actual cause.Why prime-1000 still fails
Level 1 fixes delivery in this test, but
log()still blocks for about 3 s, over the test's 250 ms limit. The uploader takes everything queued into one long write whilelog()fills the queue again. The queue, the 5 s blocking put and the drops are unchanged in the SDK.Verification
prime-1000with the assertion enforced (--runxfail), on a Mac:log()took 2954–2964 msWhole file at level 1: 8 passed, 1 xfailed, in 37 s.
ruff check,ruff format --checkanduv lock --checkpass. The environment skippedlitellmandharbor, which don't build on macOS.The fake endpoint costs no time, so these runs can't show drops caused by a slow service. I also ran the real
UploadWorker,TracesSinkandTracesClientagainst a real dev Prime Traces deployment, from a laptop on about 320 Mbit/s of uplink: 1000 episodes of about 2.6 MiB, submitted as fast as possible.submit()So #1129 speeds up uploads by about a third on a real service, but it doesn't stop the drops. The stored counts were read back from the service.
🤖 Generated with Claude Code