Skip to content

fix(sccm): bound opaque ids and harden finding deserialization - #404

Merged
adamgell merged 15 commits into
codex/parser-family-skeletonfrom
codex/sccm-spine-findings-hardening
Aug 2, 2026
Merged

fix(sccm): bound opaque ids and harden finding deserialization#404
adamgell merged 15 commits into
codex/parser-family-skeletonfrom
codex/sccm-spine-findings-hardening

Conversation

@adamgell

@adamgell adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner

What this changes

Two foundation-level hardening items in crates/cmtraceopen-parser/src/sccm/findings.rs, both surfaced by an exact-range CodeRabbit review of the spine as it sits on codex/parser-family-skeleton. Both are fail-closed tightenings; no public serialized field names change.

1. One shared opaque-id bound (RED 10616a0e / GREEN ba95b4d8).
Before: validate_coverage_gaps bounded gap.artifact_id to 256 chars, but validate_evidence_reference applied no length bound at all, and is_canonical_opaque_id checked only non-empty and trimmed. A deserialized finding could therefore carry arbitrarily long artifactId and entryId values through evidence, terminalEvidence, and correlation-key evidence, use them as BTreeMap keys, and re-serialize them. findingId, title, summary, and the correlation-key raw and normalized fields were likewise unbounded.

After: the bound lives inside is_canonical_opaque_id, which structurally gates finding ids, evidence artifact and entry ids, coverage-gap artifact ids, request logical ids, and extraction profile ids. The correlation-key bound sits inside has_canonical_value, which every key must pass, so low-confidence keys have no bypass. Title and summary bounds follow the existing is_bounded_request_reason precedent exactly: trim first, count chars() rather than bytes.

2. Public Deserialize no longer bypasses the wire contract (RED ae622752 / GREEN f4d30bc3).
Before: SccmFindingCoverageGap and SccmArtifactRequest are public and re-exported by pub use findings::*, and both derived Deserialize without deny_unknown_fields and without validation, while the parallel wire structs set deny_unknown_fields and SccmFinding::deserialize routes through them. Any caller deserializing those two types directly got neither strictness nor validation.

After: both keep implementing Deserialize, now via manual impls over the existing deny_unknown_fields wire structs plus validate_coverage_gaps / validate_artifact_requests. This was chosen over removing the derive because removal is a breaking API change; grep evidence that routing is safe is in the review notes below.

Why routing rather than removing Deserialize

Nothing outside the finding wire path deserializes these types: no from_value::<>, from_str::<>, or from_slice::<> of either type anywhere in the tree; the only Deserialize-deriving structs naming them are the wire structs in findings.rs; SccmServerIntakeAssessment holds Vec<SccmArtifactRequest> but derives Serialize only; src-tauri/src/ has zero sccm references. The change is source-compatible, and the only payloads that stop deserializing are ones the spine already considered invalid.

Verified as already fixed, no change made

The overlapping-identity-range class (a Users\DOMAIN\alice value leaving \alice in public output) is already closed at the spine head by the merged #332 work: the sort, dedup, and merge logic is at evidence.rs:262-271 from 59af7b5f, and export_merges_overlapping_identity_ranges_without_mutating_raw_snapshot covers exactly the standalone-matcher plus user-path-matcher overlap, asserting neither ADMIN nor secret survives into exported JSON.

Verification

  • cargo test --locked -p cmtraceopen-parser --test sccm_spine_contract: 142 passed, 0 failed (137 baseline plus 5 new)
  • cargo test --locked -p cmtraceopen-parser: 918 passed, 0 failed
  • cargo clippy --locked -p cmtraceopen-parser --all-targets -- -D warnings: clean
  • cargo check --locked -p cmtraceopen-parser --target wasm32-unknown-unknown: clean
  • git diff --check 7655826f HEAD: clean; rustfmt --check clean on both changed files
  • No new dependencies; Cargo.toml untouched; camelCase wire names preserved

Assumptions and notes

  • Bounds chosen: 256 for opaque ids, 512 for title, 2048 for summary, following the existing request-reason precedent. If the program wants different display-string limits, they are single constants.
  • Nine lane branches stack on this spine, so every change here is deliberately a tightening rather than a semantic change.
  • Repo note for whoever works in this crate next: a bare cargo fmt -p cmtraceopen-parser reformats four unrelated ESP files (esp/redaction.rs, esp/reducer.rs, esp/timeline.rs, tests/esp_diagnostics.rs) because this machine's rustfmt disagrees with their committed formatting. Format only the files you changed.

Refs #317. Not claiming native Windows acceptance; this is pure parser-layer work.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes

    • Added consistent validation for evidence references, coverage gaps, terminal evidence, artifact requests, and correlation keys.
    • Oversized, malformed, incomplete, or invalid data is now rejected instead of being accepted as valid.
    • Out-of-bounds key values are reported as malformed gaps.
    • Added safeguards for invalid line ranges, missing evidence, forged confidence values, and malformed nested data.
  • Tests

    • Expanded coverage for boundary conditions, validation failures, malformed inputs, and successful data round trips.

adamgell added 4 commits July 31, 2026 17:19
RED: deserialized findings currently accept arbitrarily long
finding ids, evidence artifact and entry ids, titles, summaries,
correlation-key raw and normalized values, and extraction profile
ids. Only the coverage-gap artifact id is bounded today.

Refs #317
is_canonical_opaque_id checked only non-empty and trimmed, so
finding ids, evidence artifact and entry ids, artifact request
logical ids, and extraction profile ids were unbounded on the
wire. Only the coverage-gap artifact id carried a length bound.

Rename the constant to MAX_SCCM_OPAQUE_ID_CHARS and enforce it
inside is_canonical_opaque_id so no identifier path can skip it,
then drop the now-redundant explicit check in
validate_coverage_gaps. Bound title, summary, and correlation-key
raw and normalized values, following the existing request-reason
precedent of trimming first and counting chars, not bytes.

Fail-closed tightening: previously accepted values stay accepted.

Refs #317
RED: SccmFindingCoverageGap and SccmArtifactRequest are public and
derive Deserialize directly, so they accept unknown fields and
skip every validation SccmFinding applies to the same payloads.
Captured coverage, empty, untrimmed, overlong, and undeclared
ids, and unbounded reasons all deserialize today.

Refs #317
SccmFindingCoverageGap and SccmArtifactRequest derived Deserialize
directly, so standalone payloads bypassed deny_unknown_fields and
every check SccmFinding applies to the same values.

Replace the derive with manual impls over the existing wire structs
and validators, matching how SccmFinding::deserialize already works.
Both types still implement Deserialize, so this is source compatible
for consumers; only payloads the spine already considered invalid
stop deserializing.

Nothing outside the finding wire path deserializes either type:
SccmServerIntakeAssessment is Serialize only and no caller names
them in a Deserialize position.

Refs #317
@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3f994c4b-dea4-475f-97ea-98da33e77639

📥 Commits

Reviewing files that changed from the base of the PR and between d5beb5b and a78c028.

📒 Files selected for processing (4)
  • crates/cmtraceopen-parser/src/sccm/findings.rs
  • crates/cmtraceopen-parser/src/sccm/keys.rs
  • crates/cmtraceopen-parser/src/sccm/models.rs
  • crates/cmtraceopen-parser/tests/sccm_spine_contract.rs

📝 Walkthrough

Walkthrough

The PR adds shared SCCM length limits, validates nested finding types during serialization and deserialization, validates evidence references before citation checks, and records oversized correlation-key candidates as malformed extraction gaps. Contract tests cover boundary, wire-format, nested-validation, and extraction behavior.

Changes

SCCM validation hardening

Layer / File(s) Summary
Shared bounds and finding validation
crates/cmtraceopen-parser/src/sccm/findings.rs, crates/cmtraceopen-parser/tests/sccm_spine_contract.rs
Adds shared limits for opaque IDs, finding text, correlation-key values, coverage gaps, and artifact-request reasons. Finding and evidence validation applies these limits before normalization and citation checks.
Validated nested deserialization
crates/cmtraceopen-parser/src/sccm/findings.rs, crates/cmtraceopen-parser/src/sccm/models.rs, crates/cmtraceopen-parser/tests/sccm_spine_contract.rs
Replaces derived serialization and deserialization with validated implementations for evidence references, terminal evidence, coverage gaps, correlation keys, and artifact requests. Tests cover malformed nested payloads, unknown fields, invalid spans, forged confidence, and missing evidence.
Correlation-key extraction enforcement
crates/cmtraceopen-parser/src/sccm/keys.rs, crates/cmtraceopen-parser/tests/sccm_spine_contract.rs
Rejects candidates whose raw or normalized values exceed the limit and records them as malformed gaps. Extraction tests verify this behavior for CI and KB candidates.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant WirePayload
  participant SccmDeserializer
  participant SccmValidator
  WirePayload->>SccmDeserializer: deserialize nested wire type
  SccmDeserializer->>SccmValidator: validate references, bounds, roles, states, and confidence
  SccmValidator-->>SccmDeserializer: validated SCCM model or error
Loading

Possibly related PRs

Suggested labels: test

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main changes: bounding opaque IDs and hardening SCCM finding deserialization.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added bug Something isn't working enhancement New feature or request parser Log parser related sccm SCCM/ConfigMgr related labels Aug 1, 2026
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 54 minutes.

@adamgell adamgell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head review of f4d30bc. Worktree detached at f4d30bc, clean before and
after; all probes removed.

VERDICT: BLOCK on one item. The change is correct, non-regressive, and every
claim in the PR body is verified true. The block is scope completion on the
shared spine, not a correctness defect.

BATTERY

  • cargo test --locked -p cmtraceopen-parser --test sccm_spine_contract:
    142 passed, 0 failed.
  • cargo test --locked -p cmtraceopen-parser: 918 passed, 0 failed,
    14 test binaries plus doctests.
  • cargo test --locked --workspace: 1672 passed, 0 failed, 30 test binaries.
  • cargo clippy --locked -p cmtraceopen-parser --all-targets -- -D warnings: clean.
  • cargo check --locked -p cmtraceopen-parser --target wasm32-unknown-unknown: clean.
  • rustfmt --check on the two changed files only: clean.
  • git diff --check 7655826 HEAD: clean.

All 12 SCCM contract binaries green, 344 tests: client deployment 8,
client health 3, client intake 3, client inventory/compliance/metering 47,
server distribution point 48, server hierarchy and replication 28,
server intake 11, server intake fixture 1, server provider and admin service 30,
server software update point 18, site core 5, spine 142.

RED DISCIPLINE, BOTH PAIRS

  • RED 10616a0: 137 passed, 3 failed. Reported mismatches included
    "builder accepted an overlong finding ID", "builder accepted an overlong
    evidence artifact ID", "serializer accepted an overlong finding ID", and
    "deserializer accepted an overlong evidence artifactId".
  • GREEN ba95b4d: 140 passed, 0 failed.
  • RED ae62275: 140 passed, 2 failed. Reported mismatches included
    "coverage gap deserializer accepted an unknown field", "coverage gap
    deserializer accepted captured coverage", and "artifact request deserializer
    accepted an undeclared logical ID".
  • GREEN f4d30bc: 142 passed, 0 failed.

BACKWARD COMPATIBILITY
No merged lane regresses. Longest values measured across all 141 shipped
expected.json plus every fixture JSON in the corpus:

findingId 39 / 256 (15%)
evidence artifactId 67 / 256 (26%)
evidence entryId 46 / 256 (18%)
coverage gap artifactId 67 / 256 (26%)
request logicalId 34 / 256 (13%)
extractionProfileId 35 / 256 (14%)
correlation key value 36 / 256 (14%)
title 53 / 512 (10%)
summary 53 / 2048 (3%)
request reason 122 / 240 (51%, bound unchanged by this PR)

Nothing is within 20% of a new limit. Two structural notes worth recording:
SccmFindingBuilder::new defaults title and summary to finding_id, so the default
path is bounded by the 256 id limit and cannot reach 512 or 2048; and
normalize_server_host self-caps at 253 characters, so the theoretical worst-case
ServerHost correlation key is 254 against a 256 limit, about 99% of the bound.
That is safe today but it is the tightest constraint in the change, and
MAX_SCCM_CORRELATION_KEY_VALUE_CHARS should not be lowered.

BOUND PROBES
Exactly-at-limit accepted and limit+1 rejected on every gated field: finding id,
evidence artifact id, evidence entry id, coverage gap artifact id, extraction
profile id, and correlation key value at 256/257; title at 512/513; summary at
2048/2049. Characters rather than bytes confirmed: 256 four-byte scalars
(1024 bytes) accepted, 257 rejected, and the value round-trips unchanged.
Trim-then-count confirmed: 512 characters wrapped in six spaces accepted and
normalized to 512, 513 wrapped in the same padding rejected, whitespace-only
rejected. Opaque ids treat padding as rejection rather than normalization, which
matches is_canonical_opaque_id. No builder versus deserialize asymmetry was found
on any gated path.

DESERIALIZE ROUTING
Both manual impls behave as claimed. Unknown fields are rejected with the wire
struct's own serde error listing the permitted names. Invalid values are rejected
by the corresponding validator. Valid payloads round-trip for both types
individually and inside a full SccmFinding carrying both. Serialized field names
are unchanged (artifactId, role, coverage; logicalId, role, reason) and
optionality is unchanged: removing a required field still produces
"missing field". Independent greps confirm no from_value, from_str, from_slice,
or from_reader of either type outside the test files, that
SccmServerIntakeAssessment derives Serialize only, and that src-tauri/src has no
SCCM type usage at all (its single "SCCM" hit is a comment string).

EVIDENCE.RS OVERLAP
Confirmed already fixed at the base, not by this PR. git blame attributes the
sort, dedup, and merge loop to 59af7b5, which is outside this PR's four commits,
and the regression test is inside the green 918.

BLOCKING GAP

  1. Three more public, re-exported SccmFinding member types still bypass the wire
    contract, and their deny_unknown_fields wire structs already exist.

    Observed at f4d30bc via serde_json::from_value:

    • SccmEvidenceRef accepted a 5000-character artifactId, an untrimmed entryId,
      and an unknown field.
    • SccmTerminalEvidence accepted a 5000-character nested artifactId and an
      unknown field.
    • SccmCorrelationKey accepted a 5000-character raw, an incoherent span
      (start 99999, end 1), an unknown field, and confidence "exact".

    The confidence case is why this is blocking rather than follow-up.
    REGISTERED_STABLE_CORRELATION_PROFILE_IDS is deliberately empty, with a comment
    stating that adding an entry requires contract review, and
    validate_correlation_key_evidence therefore requires every key to be Low.
    Standalone deserialization forges past that gate. That is a trust-signal
    bypass, materially more consequential than the length bypasses this PR closed,
    and it is exactly the class of hole the PR set out to close. Nine lanes stack
    on this spine, so closing it after they land is considerably more expensive
    than closing it now, and the remedy is the same ten-line pattern already
    applied twice in this PR: route through SccmEvidenceRefWire,
    SccmTerminalEvidenceWire, and SccmCorrelationKeyWire.

    To resolve: route those three types, or state on the PR why the spine
    deliberately stops at two.

NON-BLOCKING OBSERVATIONS

  1. Collection cardinality is unbounded. A finding with 50,000 evidence refs and
    50,000 coverage gaps built, validated, serialized to roughly 8 MB, and
    deserialized cleanly. Only next_artifacts has a cardinality bound, at 16.
    The per-element strings are now bounded, but the collections are not. This is
    pre-existing and outside the stated thesis; worth a tracking issue.

  2. The ServerHost correlation key headroom noted above (254 of 256) is the
    tightest ratio introduced by this change. Recommend a comment on the constant
    so it is not lowered later.

Everything else in this PR is clean. The RED/GREEN discipline is exemplary, the
bounds are placed structurally rather than at call sites, the display-string
handling follows the is_bounded_request_reason precedent exactly, and no
serialized field name or optionality changed.

adamgell added 2 commits July 31, 2026 21:36
RED: SccmEvidenceRef, SccmTerminalEvidence, and SccmCorrelationKey
are public, re-exported SccmFinding members that still derive
Deserialize directly, so standalone payloads bypass
deny_unknown_fields and every check the finding applies.

Proven accepted today: 5000-char artifact ids, untrimmed and empty
entry ids, incoherent line ranges, unknown fields at both the outer
and nested level, non-failure terminal kinds, 5000-char key values,
an incoherent 99999..1 span, and overlong profile ids.

The serious case is confidence forging. Because
REGISTERED_STABLE_CORRELATION_PROFILE_IDS is deliberately empty,
validate_correlation_key_evidence holds every key at Low, yet a
standalone payload deserializes confidence exact or strong and
forges a trust signal no registered profile can authorize.

Refs #317
SccmEvidenceRef, SccmTerminalEvidence, and SccmCorrelationKey were
the remaining public SccmFinding members whose derived Deserialize
skipped deny_unknown_fields and every finding-level check.

Route all three through their existing wire structs and validators,
matching the coverage-gap and artifact-request pattern. Standalone
payloads have no surrounding finding, so a terminal evidence and a
correlation key stand as their own citation set; that satisfies the
citation rule while leaving every other gate in force.

The gate that matters is confidence. While
REGISTERED_STABLE_CORRELATION_PROFILE_IDS stays empty, no payload
can now deserialize a Strong or Exact key and forge corroboration
strength that no registered profile authorizes.

SccmEvidenceRef and SccmCorrelationKey are declared in models.rs,
so their impls live in findings.rs beside the validators they must
satisfy. This also tightens SccmEvidence, SccmExtractionGap, and
SccmKeyExtractionResult, which embed them; extraction output still
round trips because extract_keys already downgrades every emitted
key to Low.

Also record that MAX_SCCM_CORRELATION_KEY_VALUE_CHARS must not be
lowered: a worst-case ServerHost key sits at about 99% of it.

Refs #317
@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 29 minutes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the SCCM findings wire contract in cmtraceopen-parser by bounding previously-unbounded identifier/text fields and ensuring public Deserialize entry points route through the same deny_unknown_fields wire structs and validators as SccmFinding.

Changes:

  • Centralizes opaque-id length bounds in is_canonical_opaque_id and adds bounded validation for finding title/summary and correlation-key values.
  • Replaces derived Deserialize with manual Deserialize impls for several public SCCM types to enforce the same wire strictness + validation when deserialized standalone.
  • Adds contract tests to assert reject/accept behavior across builder, direct validation, serialization, and deserialization boundaries.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
crates/cmtraceopen-parser/src/sccm/findings.rs Adds shared bounds + manual Deserialize routing through strict wire structs and validators for standalone deserialization paths.
crates/cmtraceopen-parser/src/sccm/models.rs Removes derived Deserialize for SccmEvidenceRef and SccmCorrelationKey, documenting that deserialization is enforced via findings.rs.
crates/cmtraceopen-parser/tests/sccm_spine_contract.rs Expands contract tests to cover overlong fields, unknown-field rejection, and standalone deserialization strictness.
Suppressed comments (1)

crates/cmtraceopen-parser/src/sccm/findings.rs:2262

  • The new length bound checks use chars().count(), which iterates the entire string. For adversarially large JSON strings this turns the validation path into an avoidable O(n) scan per field; switching to an early-exit check keeps the bound fail-fast.
fn is_canonical_opaque_id(value: &str) -> bool {
    !value.is_empty() && value.trim() == value && value.chars().count() <= MAX_SCCM_OPAQUE_ID_CHARS
}

Comment thread crates/cmtraceopen-parser/src/sccm/findings.rs Outdated
Comment thread crates/cmtraceopen-parser/src/sccm/findings.rs Outdated
@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

Code review

Five independent reviewers examined this change from different angles (CLAUDE.md compliance, focused bug scan, git-blame history, prior-PR feedback, and code-comment guidance). Each candidate finding was then independently verified and confidence-scored; anything below 80 is reported separately as unconfirmed.
3 confirmed finding(s) (score >= 80), 3 unconfirmed, 3 rejected as false positives.

1. SccmCorrelationKey::deserialize leaves its nested evidence ref unvalidated — the exact door the PR set out to close

cmtraceopen-parser/src/sccm/findings.rs lines 404-448 (confidence 85)
Confirmed by execution, not inspection. I built a probe crate with a path dep on the branch's parser and ran the exact payloads: empty artifactId, untrimmed entryId, reversed 9->2 range, half-set range, and a 5000-char artifactId ALL deserialize successfully as a standalone SccmCorrelationKey, while the identical reference is REJECTED as a bare SccmEvidenceRef and inside SccmTerminalEvidence. Grep confirms validate_correlation_key_evidence (findings.rs:2154-2200) never calls validate_evidence_reference; its only reference check is evidence.contains(reference) against key.evidence.as_slice(), which is trivially self-satisfying. The sibling SccmTerminalEvidence::deserialize added in the same commit does call validate_evidence_reference at line 354, so this is an internal inconsistency in one commit, not a design choice. It directly falsifies the doc comments the PR itself adds at findings.rs:310-313 and models.rs:150-153, and bypasses the MAX_SCCM_OPAQUE_ID_CHARS bound this same PR introduces. Reachability through SccmKeyExtractionResult (models.rs:265-271, derives Deserialize) is confirmed. Test-coverage claim verified: the new correlation_key test (tests/sccm_spine_contract.rs:1863-1939) never mutates canonical["evidence"], while the terminal-evidence test at 1808-1851 does test a 5000-char nested artifactId. Held back from 100 only because nothing in-tree deserializes SccmKeyExtractionResult today outside one test, so today's exposure is latent rather than live — but on a spine nine lanes stack on, this is exactly the guarantee those lanes will assume.

Fix is one line mirroring findings.rs:354: run validate_evidence_reference over key.evidence before validate_correlation_key_evidence, or retype SccmCorrelationKeyWire.evidence as Option.

2. SccmCorrelationKey::deserialize never validates its nested evidence reference — the one door the PR left open

cmtraceopen-parser/src/sccm/findings.rs lines 436-448 (confidence 85)
Same defect, verified. The historical framing also checks out: commit 5fd80ee 'fix(sccm): reject nested finding wire fields' is present in this branch's history, and the wire structs it introduced carry only deny_unknown_fields, with the semantic reference checks living in SccmFinding::validate -> validate_all_evidence_references. PR #404 re-supplies that glue for the evidence-ref and terminal-evidence doors but not the correlation-key one. My probe reproduced the quoted payload verbatim (extractionProfileId "sccm-keys-experimental-v1", artifactId "", entryId untrimmed, lineStart 9 / lineEnd 2) as Ok, with both named controls rejecting the identical reference with the exact error strings quoted. SccmKeyExtractionResult reachability confirmed. Same latency caveat as the other copies keeps it below 100.

Also verified: correlation_key_deserializes_through_the_same_wire_contract_as_a_finding (tests/sccm_spine_contract.rs:1863-1939) has no nested-reference case, unlike its terminal-evidence sibling at 1808-1851.

3. New standalone SccmCorrelationKey deserializer never validates its nested evidence reference

cmtraceopen-parser/src/sccm/findings.rs lines 406-448 (confidence 85)
Same defect, and this write-up's control matrix is the one I reproduced most exactly. My probe ran all five listed cases (5000-char artifactId, empty artifactId, untrimmed entryId, inverted 9..1 range, half-set lineStart=7/lineEnd=null) and every one was ACCEPTED as a standalone SccmCorrelationKey and through SccmKeyExtractionResult, while every one was REJECTED by both the bare SccmEvidenceRef door and the SccmTerminalEvidence door. The half-set-range case in particular is a good catch: it is the one shape a reader might assume the (None,None)/(Some,Some) span logic already covers, and it does not. All line references check out (wire field 412, impl 436-448, validator 2154-2200, terminal sibling 348-368, in-finding coverage 763-794). Below 100 only because no lane deserializes an extraction result today.

Fix: retype the wire field or add key.evidence.as_ref().map_or(Ok(()), validate_evidence_reference) before validate_correlation_key_evidence.


Unconfirmed findings (scored 50-79, verify before acting)
  • [50] New correlation-key value bound is enforced at the validator but not at the producer, so extract_keys output no longer round-trips
  • [50] MAX_SCCM_CORRELATION_KEY_VALUE_CHARS is enforced only at the boundary; extract_keys can now emit a key that fails the crate's own contract, taking the whole finding with it
  • [50] New 256-char correlation-key value bound is not reconciled with the extractor that produces the values

SccmCorrelationKeyWire carries evidence as Option<SccmEvidenceRefWire>,
so a nested reference never reaches the SccmEvidenceRef deserializer.
validate_correlation_key_evidence only checks that the citation set
contains the reference, and the key's own reference is that set, so the
check is self-satisfying and the reference itself is never validated.

Share one noncanonical reference payload list across the standalone
evidence-ref door, the terminal-evidence door, and the correlation-key
door so no door can be tested against a weaker list than its siblings,
and cover the SccmKeyExtractionResult path that nests keys.

Also cover the nested role each of the coverage-gap and artifact-request
wire doors carries.

Refs #317
validate_correlation_key_evidence now validates the reference itself
before the containment check, so every door reaches the same bar: the
standalone key deserializer, the SccmKeyExtractionResult path that nests
keys, SccmFinding::validate, the builder, and the serializer.

Placed inside the validator rather than at the deserializer call site
because the defect was a call site that omitted the check. A caller
cannot omit a check the validator performs.

Refs #317
MAX_SCCM_CORRELATION_KEY_VALUE_CHARS is enforced at the validator but
not at the producer, so extract_keys can emit a key that fails the
crate's own contract and no longer round trips.

Covers the raw value crossing the bound, and a KB id whose raw stays
inside the bound and only crosses it once normalization prepends "KB",
so the producer has to weigh the normalized value too. Also pins the
out-of-bound candidate as a recorded gap rather than a silent drop.

Refs #317
extract_keys now weighs an out-of-bound value the same way it weighs a
value that fails to normalize: the candidate becomes a MalformedCandidate
gap instead of a key, so it stays visible and the producer stops emitting
keys the crate's own validator rejects.

Weighs the normalized value as well as the raw one, since normalize_kb_id
prepends "KB" and can push a bounded raw past the bound.

MAX_SCCM_CORRELATION_KEY_VALUE_CHARS becomes crate-visible so the
producer and the validator share one number rather than two that can
drift. It stays out of the public surface: it is an internal agreement,
not wire surface.

Refs #317
@adamgell
adamgell requested a review from Copilot August 1, 2026 14:11
@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 6 minutes.

@coderabbitai coderabbitai Bot added the test Testing related label Aug 1, 2026
@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

Please review exact head 2fc676b. The three prior actionable threads have been fixed, replied to, and resolved; local CodeRabbit returned zero findings.

@adamgell
adamgell requested a review from Copilot August 1, 2026 22:14
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

@adamgell I will perform a full review of exact head 2fc676b7bb0e42457c93f0d64525295aa4046656.

✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/cmtraceopen-parser/src/sccm/findings.rs`:
- Around line 215-235: Update SccmArtifactRequest::serialize to invoke the same
validate_artifact_requests validator used by Deserialize before constructing
SccmArtifactRequestSerializeWire. Propagate validation failures through
S::Error, while preserving the existing trimmed reason serialization and wire
representation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ccedb243-2ee1-4304-b1ae-49d1655a56ab

📥 Commits

Reviewing files that changed from the base of the PR and between e56ad38 and 2fc676b.

📒 Files selected for processing (4)
  • crates/cmtraceopen-parser/src/sccm/findings.rs
  • crates/cmtraceopen-parser/src/sccm/keys.rs
  • crates/cmtraceopen-parser/src/sccm/models.rs
  • crates/cmtraceopen-parser/tests/sccm_spine_contract.rs

Comment thread crates/cmtraceopen-parser/src/sccm/findings.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

crates/cmtraceopen-parser/src/sccm/keys.rs:215

  • is_bounded_key_value uses chars().count() to enforce the 256-char bound. For oversized candidate values this walks the entire string, which is avoidable and inconsistent with the fail-fast has_at_most_chars(..) approach used in findings.rs. Switching to an nth(MAX) check prevents scanning very large inputs during key extraction.
fn is_bounded_key_value(key: &SccmCorrelationKey) -> bool {
    key.raw.chars().count() <= MAX_SCCM_CORRELATION_KEY_VALUE_CHARS
        && key.normalized.chars().count() <= MAX_SCCM_CORRELATION_KEY_VALUE_CHARS
}

@adamgell

adamgell commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

Exact-head review checkpoint

PR #404 remains draft with CHANGES_REQUESTED by design. The repair is locally green, but it is not merge-ready until #335 resolves the contract and the exact-head review cycle is rerun.

@adamgell

adamgell commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

Exact-head repair checkpoint for 298530cdf517b8de1217e5df5627fc603132289a:

  • Resynced non-force onto current integration d5beb5bbb1da9c8e6b8fab22af14dee792da74c1.
  • d7ddd228: standalone artifact-request serialization now uses the full catalog/role/bounded-reason validator.
  • 298530cd: standalone evidence references, terminal evidence, coverage gaps, and correlation keys now validate on Serialize through the same contracts as Deserialize/containing findings.
  • Added focused RED coverage for undeclared IDs, role mismatch, rooted/cross-artifact reasons, invalid/overlong evidence identities and ranges, non-terminal kinds, captured gaps, forged key confidence, missing/invalid key evidence, incoherent UTF-16 spans, and independent raw/normalized key bounds.
  • Preserved camelCase wire fields and existing null/option shape.

Verification on the exact committed head:

  • artifact-request focused tests: 12/12 green
  • sccm_spine_contract: 159/159 green
  • sccm_server_intake: 20/20 green
  • sccm_server_management_point: 26/26 green
  • full cmtraceopen-parser: 354 unit tests plus all integration targets and doctests green
  • cargo check --locked -p cmtraceopen-parser --target wasm32-unknown-unknown: green
  • cargo clippy --locked -p cmtraceopen-parser --all-targets --all-features -- -D warnings: green
  • Rust 1.88 rustfmt check on all three touched files: green
  • git diff --check: green
  • repo-wide cargo +1.88.0 fmt --all -- --check: still red only on pre-existing ESP/Tauri paths outside this PR delta
  • independent agent review: clean
  • local CodeRabbit committed review of the full four-file PR delta against current integration: 0 findings

Fresh hosted CodeRabbit and Copilot reviews were requested on this exact head. The existing artifact-request thread remains unresolved until the hosted review and required CI are green.

@adamgell
adamgell marked this pull request as ready for review August 2, 2026 00:49
@adamgell

adamgell commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

crates/cmtraceopen-parser/src/sccm/keys.rs:215

  • is_bounded_key_value uses chars().count() for both raw and normalized values. Since the regex patterns can capture arbitrarily long value substrings (e.g., [0-9]+[A-Za-z0-9_.-]*), this will fully scan oversized inputs; using the same fail-fast chars().nth(MAX).is_none() pattern as the validator avoids unnecessary work on malicious/degenerate evidence messages.
fn is_bounded_key_value(key: &SccmCorrelationKey) -> bool {
    key.raw.chars().count() <= MAX_SCCM_CORRELATION_KEY_VALUE_CHARS
        && key.normalized.chars().count() <= MAX_SCCM_CORRELATION_KEY_VALUE_CHARS
}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/cmtraceopen-parser/src/sccm/keys.rs (1)

209-215: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Reuse the shared character-bound helper.

findings.rs added has_at_most_chars, which stops after maximum + 1 characters instead of counting the whole string. This producer-side check counts every character of both values. The two sides now express the same bound through two different mechanisms, and only one of them fails fast.

Make has_at_most_chars pub(crate) and call it here, so the producer and the validator share one implementation of the bound.

♻️ Proposed refactor

In crates/cmtraceopen-parser/src/sccm/findings.rs:

-fn has_at_most_chars(value: &str, maximum: usize) -> bool {
+pub(crate) fn has_at_most_chars(value: &str, maximum: usize) -> bool {
     value.chars().nth(maximum).is_none()
 }

In crates/cmtraceopen-parser/src/sccm/keys.rs:

-use super::findings::MAX_SCCM_CORRELATION_KEY_VALUE_CHARS;
+use super::findings::{MAX_SCCM_CORRELATION_KEY_VALUE_CHARS, has_at_most_chars};
 fn is_bounded_key_value(key: &SccmCorrelationKey) -> bool {
-    key.raw.chars().count() <= MAX_SCCM_CORRELATION_KEY_VALUE_CHARS
-        && key.normalized.chars().count() <= MAX_SCCM_CORRELATION_KEY_VALUE_CHARS
+    has_at_most_chars(&key.raw, MAX_SCCM_CORRELATION_KEY_VALUE_CHARS)
+        && has_at_most_chars(&key.normalized, MAX_SCCM_CORRELATION_KEY_VALUE_CHARS)
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/cmtraceopen-parser/src/sccm/keys.rs` around lines 209 - 215, Expose
the existing has_at_most_chars helper in findings.rs as pub(crate), then update
is_bounded_key_value in keys.rs to validate both key.raw and key.normalized
through that shared helper with MAX_SCCM_CORRELATION_KEY_VALUE_CHARS. Remove the
direct chars().count() checks while preserving the requirement that both values
satisfy the bound.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/cmtraceopen-parser/src/sccm/keys.rs`:
- Around line 209-215: Expose the existing has_at_most_chars helper in
findings.rs as pub(crate), then update is_bounded_key_value in keys.rs to
validate both key.raw and key.normalized through that shared helper with
MAX_SCCM_CORRELATION_KEY_VALUE_CHARS. Remove the direct chars().count() checks
while preserving the requirement that both values satisfy the bound.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d3723537-ee63-4844-8480-34aca48c97fa

📥 Commits

Reviewing files that changed from the base of the PR and between d5beb5b and 298530c.

📒 Files selected for processing (4)
  • crates/cmtraceopen-parser/src/sccm/findings.rs
  • crates/cmtraceopen-parser/src/sccm/keys.rs
  • crates/cmtraceopen-parser/src/sccm/models.rs
  • crates/cmtraceopen-parser/tests/sccm_spine_contract.rs

@adamgell

adamgell commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

Review follow-up pushed at exact head a78c0288adef20595e6182f8bee02aceaa136d55.

The CodeRabbit nitpick and Copilot suppressed observation were valid: producer-side extracted-key bounds used full chars().count() scans while the public validator already had a fail-fast maximum-plus-one helper. The producer now reuses that shared helper for both raw and normalized values; wire behavior and the 256-character contract are unchanged.

Verification:

  • key extraction filter: 10/10 green
  • SCCM spine: 159/159 green
  • full parser aggregate: green
  • wasm32 check: green
  • strict all-target/all-feature Clippy: green
  • Rust 1.88 formatting on touched files: green
  • git diff --check: green
  • local CodeRabbit on the exact two-file follow-up: 0 findings

Requesting final hosted CodeRabbit and Copilot reviews on this exact head.

@adamgell

adamgell commented Aug 2, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@adamgell
adamgell requested a review from Copilot August 2, 2026 00:56
@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

@adamgell
adamgell merged commit 34c3aba into codex/parser-family-skeleton Aug 2, 2026
11 checks passed
adamgell added a commit that referenced this pull request Aug 4, 2026
* test(sccm): prove finding text and id fields are unbounded

RED: deserialized findings currently accept arbitrarily long
finding ids, evidence artifact and entry ids, titles, summaries,
correlation-key raw and normalized values, and extraction profile
ids. Only the coverage-gap artifact id is bounded today.

Refs #317

* fix(sccm): bound every finding id and display string

is_canonical_opaque_id checked only non-empty and trimmed, so
finding ids, evidence artifact and entry ids, artifact request
logical ids, and extraction profile ids were unbounded on the
wire. Only the coverage-gap artifact id carried a length bound.

Rename the constant to MAX_SCCM_OPAQUE_ID_CHARS and enforce it
inside is_canonical_opaque_id so no identifier path can skip it,
then drop the now-redundant explicit check in
validate_coverage_gaps. Bound title, summary, and correlation-key
raw and normalized values, following the existing request-reason
precedent of trimming first and counting chars, not bytes.

Fail-closed tightening: previously accepted values stay accepted.

Refs #317

* test(sccm): prove gap and request deser skip the wire contract

RED: SccmFindingCoverageGap and SccmArtifactRequest are public and
derive Deserialize directly, so they accept unknown fields and
skip every validation SccmFinding applies to the same payloads.
Captured coverage, empty, untrimmed, overlong, and undeclared
ids, and unbounded reasons all deserialize today.

Refs #317

* fix(sccm): route gap and request deser through the wire contract

SccmFindingCoverageGap and SccmArtifactRequest derived Deserialize
directly, so standalone payloads bypassed deny_unknown_fields and
every check SccmFinding applies to the same values.

Replace the derive with manual impls over the existing wire structs
and validators, matching how SccmFinding::deserialize already works.
Both types still implement Deserialize, so this is source compatible
for consumers; only payloads the spine already considered invalid
stop deserializing.

Nothing outside the finding wire path deserializes either type:
SccmServerIntakeAssessment is Serialize only and no caller names
them in a Deserialize position.

Refs #317

* test(sccm): prove three more public types skip the wire contract

RED: SccmEvidenceRef, SccmTerminalEvidence, and SccmCorrelationKey
are public, re-exported SccmFinding members that still derive
Deserialize directly, so standalone payloads bypass
deny_unknown_fields and every check the finding applies.

Proven accepted today: 5000-char artifact ids, untrimmed and empty
entry ids, incoherent line ranges, unknown fields at both the outer
and nested level, non-failure terminal kinds, 5000-char key values,
an incoherent 99999..1 span, and overlong profile ids.

The serious case is confidence forging. Because
REGISTERED_STABLE_CORRELATION_PROFILE_IDS is deliberately empty,
validate_correlation_key_evidence holds every key at Low, yet a
standalone payload deserializes confidence exact or strong and
forges a trust signal no registered profile can authorize.

Refs #317

* fix(sccm): close the last three unvalidated deser doors

SccmEvidenceRef, SccmTerminalEvidence, and SccmCorrelationKey were
the remaining public SccmFinding members whose derived Deserialize
skipped deny_unknown_fields and every finding-level check.

Route all three through their existing wire structs and validators,
matching the coverage-gap and artifact-request pattern. Standalone
payloads have no surrounding finding, so a terminal evidence and a
correlation key stand as their own citation set; that satisfies the
citation rule while leaving every other gate in force.

The gate that matters is confidence. While
REGISTERED_STABLE_CORRELATION_PROFILE_IDS stays empty, no payload
can now deserialize a Strong or Exact key and forge corroboration
strength that no registered profile authorizes.

SccmEvidenceRef and SccmCorrelationKey are declared in models.rs,
so their impls live in findings.rs beside the validators they must
satisfy. This also tightens SccmEvidence, SccmExtractionGap, and
SccmKeyExtractionResult, which embed them; extraction output still
round trips because extract_keys already downgrades every emitted
key to Low.

Also record that MAX_SCCM_CORRELATION_KEY_VALUE_CHARS must not be
lowered: a worst-case ServerHost key sits at about 99% of it.

Refs #317

* test(sccm): prove nested key evidence skips the contract

SccmCorrelationKeyWire carries evidence as Option<SccmEvidenceRefWire>,
so a nested reference never reaches the SccmEvidenceRef deserializer.
validate_correlation_key_evidence only checks that the citation set
contains the reference, and the key's own reference is that set, so the
check is self-satisfying and the reference itself is never validated.

Share one noncanonical reference payload list across the standalone
evidence-ref door, the terminal-evidence door, and the correlation-key
door so no door can be tested against a weaker list than its siblings,
and cover the SccmKeyExtractionResult path that nests keys.

Also cover the nested role each of the coverage-gap and artifact-request
wire doors carries.

Refs #317

* fix(sccm): validate nested correlation key evidence

validate_correlation_key_evidence now validates the reference itself
before the containment check, so every door reaches the same bar: the
standalone key deserializer, the SccmKeyExtractionResult path that nests
keys, SccmFinding::validate, the builder, and the serializer.

Placed inside the validator rather than at the deserializer call site
because the defect was a call site that omitted the check. A caller
cannot omit a check the validator performs.

Refs #317

* test(sccm): prove extract_keys emits rejected keys

MAX_SCCM_CORRELATION_KEY_VALUE_CHARS is enforced at the validator but
not at the producer, so extract_keys can emit a key that fails the
crate's own contract and no longer round trips.

Covers the raw value crossing the bound, and a KB id whose raw stays
inside the bound and only crosses it once normalization prepends "KB",
so the producer has to weigh the normalized value too. Also pins the
out-of-bound candidate as a recorded gap rather than a silent drop.

Refs #317

* fix(sccm): bound correlation key values at the producer

extract_keys now weighs an out-of-bound value the same way it weighs a
value that fails to normalize: the candidate becomes a MalformedCandidate
gap instead of a key, so it stays visible and the producer stops emitting
keys the crate's own validator rejects.

Weighs the normalized value as well as the raw one, since normalize_kb_id
prepends "KB" and can push a bounded raw past the bound.

MAX_SCCM_CORRELATION_KEY_VALUE_CHARS becomes crate-visible so the
producer and the validator share one number rather than two that can
drift. It stays out of the public surface: it is an internal agreement,
not wire surface.

Refs #317

* fix(sccm): bound stored finding request text

* fix(sccm): validate standalone artifact requests

* fix(sccm): validate standalone citation serialization

* perf(sccm): fail fast on oversized extracted keys
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request parser Log parser related sccm SCCM/ConfigMgr related test Testing related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants