Cap TrackedSession activity grid at 500 rows to prevent OOM on timeout

[jeremydmiller]

A chaos-monkey'd integration test (originally hit on JasperFx/CritterWatch#304) can have BuildActivityMessage produce a grid so large that Grid.Write → StringBuilder.ToString() throws System.OutOfMemoryException before the TimeoutException is ever surfaced to the test.

The OOM masks the actual timeout for several diagnostic cycles — you don't see "tracked session timed out at 60s waiting for X", you see a stack trace bottoming out in StringBuilder.ToString and nothing about which message type was actually flooding the activity buffer.

Concrete repro from the CritterWatch follow-up: 7,078,703 envelope records accumulated in 60s because of a relay-handler recursion on the consumer side, and the OOM had me chasing the wrong root cause for a couple iterations before I patched the grid cap locally to see the real story.

The fix

Cap BuildActivityMessage at the most recent 500 envelope records when the activity list would otherwise be larger. The cap is exposed as internal const int ActivityGridRowLimit = 500 so tests can read it back if needed. When truncation kicks in, the output gets a one-line preamble so the operator knows what was omitted:

(showing last 500 of 7,078,703 envelope records — earlier 7,078,203 omitted for readability)

500 is chosen as a value that:

• comfortably formats within StringBuilder bounds (~50KB output);
• shows enough trailing activity to identify where the session hung or what message type dominated;
• doesn't truncate normal-shape integration tests (most run with well under 500 envelope events).

Why "last N" and not "first N"

Once you're already at avalanche scale, the leading edge is uninteresting — you already know there was an avalanche. The trailing edge is the actionable signal: which message type was the session still chasing when the budget ran out. In the CritterWatch repro this immediately surfaced "every recent envelope is alert_changes_batch" and named the recursion loop within seconds of reading the output.

Testing

Verified against the same chaos-monkey'd integration scenario that surfaced the bug (~7M envelope records in 60s):

• Before: BuildActivityMessage throws OutOfMemoryException, the timeout cause is masked.
• After: BuildActivityMessage formats cleanly with the truncation preamble, the timeout cause is immediately visible.

No existing TrackedSession unit/integration tests in the repo are affected — the cap only changes output when there are >500 records, and Wolverine's own test suites stay well under that.