perf(runtime): add request-plane msgpack payload codec

[jthomson04]

Summary

• add an opt-in DYN_REQUEST_PLANE_CODEC=json|msgpack runtime setting for request-plane payloads
• carry the selected codec in RequestControlMessage while preserving JSON as the default/wire-compatible behavior
• encode/decode unary, bidirectional request frames, and response stream wrappers with the selected codec

Benchmarked Perf

Measured on the agentx trace at concurrency 512, using fast mockers:

Config	Throughput	Avg latency	Avg TTFT	Errors
Response-path PR only, JSON request plane	334.36 req/s	1638.98 ms	1387.49 ms	0
Response-path PR + DYN_REQUEST_PLANE_CODEC=msgpack	361.45 req/s	1510.35 ms	1253.13 ms	0

Observed uplift from the transport switch in that setup:

• throughput: +8.1%
• avg latency: -7.8%
• avg TTFT: -9.7%
• addressed/egress network-path CPU: 13.92% -> 9.57% of on-CPU samples
• dominant request-plane encode leaves: 11.07% -> 6.91% of on-CPU samples

Testing

• rustfmt --edition 2024 lib/runtime/src/config/environment_names.rs lib/runtime/src/pipeline/network.rs lib/runtime/src/pipeline/network/egress/addressed_router.rs lib/runtime/src/pipeline/network/ingress/push_handler.rs
• git diff --check -- lib/runtime/src/config/environment_names.rs lib/runtime/src/pipeline/network.rs lib/runtime/src/pipeline/network/egress/addressed_router.rs lib/runtime/src/pipeline/network/ingress/push_handler.rs
• cargo test -p dynamo-runtime request_plane_payload_codec
• cargo test -p dynamo-runtime request_control_message

---

Summary by CodeRabbit

• New Features

  • Added configurable request payload codec selection via environment variable, supporting JSON (default) and Msgpack encoding formats.
  • Request control messages now carry codec information for consistent payload interpretation across system components.

• Tests

  • Expanded test suite to validate codec defaults and round-trip encoding/decoding for both supported formats.

[coderabbitai]

Walkthrough

This PR introduces configurable request-plane payload encoding: an environment variable selects between JSON and Msgpack codecs, the choice is embedded in request control messages and cached globally, and both egress (sending requests, receiving responses) and ingress (receiving requests, sending responses) paths use the codec to serialize and deserialize typed payloads.

Changes

Request-plane codec selection and integration

Layer / File(s)	Summary
Configuration and codec infrastructure lib/runtime/src/config/environment_names.rs, lib/runtime/src/pipeline/network.rs	Introduces request_plane::DYN_REQUEST_PLANE_CODEC environment variable; adds RequestPlanePayloadCodec enum with JSON/Msgpack variants, env-driven configuration with OnceLock caching, and encode/decode methods; extends RequestControlMessage with payload_codec field (serde-driven, omitted when JSON); adds PartialEq/Eq derives to NetworkStreamWrapper and validates codec functionality with round-trip tests.
Egress: request building and response decoding lib/runtime/src/pipeline/network/egress/addressed_router.rs	build_request_envelope creates a configured payload codec, embeds it into RequestControlMessage, and encodes request data via the codec; spawn_request_stream_forwarder receives the codec and encodes each forwarded item; decode_response_stream accepts the codec and decodes NetworkStreamWrapper<U> payloads; dispatch_and_finalize constructs the codec once and wires it through both request and response paths; error logging includes codec metadata.
Ingress: request parsing and response encoding lib/runtime/src/pipeline/network/ingress/push_handler.rs	Request parsing for both unary and bidirectional paths extracts payload_codec from RequestControlMessage, decodes typed requests via codec-driven deserialization, and stores the codec in ParsedRequest; pump_response_stream accepts the codec and encodes both regular and terminal response frames via the codec; shared handler retrieves the codec and passes it to response pumping; error reporting includes codec naming.

---

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	PR description includes summary, benchmarked performance data, testing procedures, and implementation details; however, it lacks structured adherence to the template's required 'Related Issues' section.	Add the required 'Related Issues' section with either 'Closes #XXXX' or confirmation that no related issue exists, as specified in the repository template.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly and concisely describes the main change: adding msgpack codec support at the request-plane level as a performance optimization.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[grahamking]

If you run the test 5 times, how much variance are you getting? Those numbers look like they could be within measurement error.

Parsing JSON is expensive (you have to read it byte-by-byte) so I support us changing it. We could do our own binary format, it's easy. The forward direction (frontend -> backend) does that. msgpack is fine too. FlatBuffers https://crates.io/crates/flatbuffers might be worth a try.

[jthomson04]

@grahamking Results from 5 runs:

Run	Before / JSON req/s	After / msgpack req/s	Uplift
1	408.08	444.66	8.96%
2	409.82	448.53	9.45%
3	410.60	444.53	8.26%
4	409.07	442.93	8.28%
5	405.26	444.39	9.66%
Avg	408.56	445.01	8.92%

[grahamking]

It's better. Let's make if the default.

[grahamking]

It's better. Let's make if the default.

I don't think we can do that. We need Dynamo 1.3 to be backwards compatible with 1.2, so we can't switch the codec until both frontend and backend already have it.

[GuanLuo]

I don't think we can do that. We need Dynamo 1.3 to be backwards compatible with 1.2, so we can't switch the codec until both frontend and backend already have it.

Can you elaborate on "both frontend and backend already have it"? This codec change covers both ingress and egress so both sides should have it. Are you worried about the mix of use between Dynamo 1.3 and 1.2?

[jthomson04]

Will merge this in for now, and make this the default after the 1.3.0 code freeze.

[michaelfeil]

@jthomson04 @GuanLuo do you think this pr can be cherry-picked on earlier versions of dynamo?

[jthomson04]

@michaelfeil Can you clarify the ask here? Are you looking for just a branch off of a release version with that fix, or also published/republished containers/wheels? We generally don't support re-publishing existing releases, but I can put together a branch off a release version with this PR that you can build from.