Skip to content

perf: encode json with orjson - #4266

Closed
mikasenghaas wants to merge 1 commit into
mainfrom
perf/orjson
Closed

mikasenghaas wants to merge 1 commit into
mainfrom
perf/orjson

Conversation

@mikasenghaas

@mikasenghaas mikasenghaas commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Encode and decode JSON with orjson across src and the unit tests:
    • resolved configs (rl, sft, eval, inference, per-role PD configs) and the LoRA adapter config, indented as before
    • safetensors index files: indented, with sorted keys, byte-identical to before for ASCII content
    • metric rows, JSON log lines (loguru sink and the vLLM formatter), and the vLLM logging config
    • the dashboard registry, WANDB_ARGS, eval --source, the vLLM override fragment, tool-call and tool-schema parsing in SFT data
  • Keep the stdlib in three places:
    • scripts/docker_dependencies.py and tests/regression/scripts/eval.py, which run without the project venv
    • tests/unit/orchestrator/test_episode_records.py, which checks the prime SDK's stdlib encoder
  • Leave packages/prime-rl-configs unchanged. It does not depend on orjson.

Output changes: non-ASCII text is UTF-8 instead of \uXXXX escapes. WANDB_ARGS and the vLLM override fragment are compact. Every consumer parses the JSON.

Related: PrimeIntellect-ai/prime#1109 moves the prime SDK to orjson, and PrimeIntellect-ai/verifiers#2981 does the same for verifiers.

Verification

  • tests/unit (CPU box): 718 passed.
    • 60 failed and 33 errored, all for environmental reasons: missing reverse_text/gsm8k env packages, no NVIDIA driver, or no prime_kernels.
    • The two DID NOT WARN config tests fail on main as well.
  • test_weights.py needs prime_kernels and could not run. The index encoding matches json.dumps(indent=2, sort_keys=True) byte for byte on a sample index.

🤖 Generated with Claude Code


Note

Medium Risk
Wide mechanical swap across config persistence, logging, and W&B argv round-trips; format differences are documented but any missed consumer assuming stdlib JSON escaping could break edge cases.

Overview
Replaces the stdlib json module with orjson across launchers, utilities, monitors, inference logging, trainer code, and unit tests.

Serialization paths now use orjson.loads/dumps with read_bytes/write_bytes where appropriate: resolved run configs (RL, SFT, eval, inference, env-server JSON), dashboard registry files, safetensors index files (still indented + sorted keys), metrics JSONL rows, loguru and vLLM JSON log lines, and the temp vLLM dictConfig file.

CLI and env wiring changes include eval --source shorthands and WANDB_ARGS (compact JSON, .decode() for strings), plus vLLM override fragments embedded in SLURM/bash strings. SFT/tool-call parsing and JSON-encoded tool schemas use orjson.loads as well.

Intentional exceptions (per description): stdlib json remains in a few scripts/tests outside the main venv path; prime-rl-configs is untouched.

Output differences: non-ASCII text is written as UTF-8 instead of \u escapes; some embedded JSON is compact rather than pretty-printed. Call sites that need human-readable files keep orjson.OPT_INDENT_2.

Reviewed by Cursor Bugbot for commit 5499284. Bugbot is set up for automated code reviews on this repo. Configure here.

Resolved configs, metric rows, log lines, weight indices and test
fixtures now go through orjson. The indented files keep the stdlib
layout; non-ASCII text is written as UTF-8 instead of \u escapes.

Two scripts keep the stdlib: scripts/docker_dependencies.py and
tests/regression/scripts/eval.py run without the project venv. The
episode-record test keeps it because it checks the SDK's stdlib
encoder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5499284. Configure here.

if self.producer is not None:
row["producer"] = self.producer
self.file.write(json.dumps(row) + "\n")
self.file.write(orjson.dumps(row).decode() + "\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

orjson rejects numpy metric scalars

Medium Severity

log_metrics now encodes each row with orjson.dumps and no OPT_SERIALIZE_NUMPY. sanitize keeps finite numpy.float64 / numpy.int64 values (they pass the float/int checks), and orjson then raises TypeError on those types. json.dumps accepted them because those numpy scalars subclass the builtins. The same monitor already uses OPT_SERIALIZE_NUMPY for traces, so a numpy scalar in a metric row can fail the write and abort the logging call.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5499284. Configure here.

@mikasenghaas

Copy link
Copy Markdown
Member Author

Closing: orjson only pays off on the episode serialization path, where records are large. Entrypoints, logging, weights and the other small payloads stay on the stdlib json. The file monitor's episode path already uses orjson.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant