Repository navigation
feat(intune): model remediation detection and remediation evidence #400
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
98497d1
refactor(intune): give the Windows analyzers one masking implementation
adamgell eb033db
feat(intune): add the remediation detection/remediation analyzer
adamgell 271ede9
fix(intune): close four defects found by review of the remediation an…
adamgell 2850451
fix(intune): close nine review findings on the remediation analyzer
adamgell 5956cbc
Merge origin/main into the remediation analyzer branch
adamgell File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
10 changes: 10 additions & 0 deletions
10
crates/cmtraceopen-parser/src/intune/apps/windows/common/mod.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| //! Primitives shared by the Intune Windows workload analyzers. | ||
| //! | ||
| //! Each analyzer owns its own state machine, because a platform script, a | ||
| //! remediation pair, and a Win32 installer reach terminal states for different | ||
| //! reasons. What they legitimately share is lower-level machinery -- privacy | ||
| //! masking, for now -- and that lives here so there is a single owner. | ||
|
|
||
| mod redaction; | ||
|
|
||
| pub use redaction::redact_text; |
137 changes: 137 additions & 0 deletions
137
crates/cmtraceopen-parser/src/intune/apps/windows/common/redaction.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| //! Deterministic text masking shared by the Intune Windows analyzers. | ||
| //! | ||
| //! Platform scripts and remediations quote the same kinds of sensitive value -- | ||
| //! UPNs, user profile paths, inline command lines and credential switches -- so | ||
| //! they mask them with one implementation rather than two that can drift apart. | ||
| //! Each analyzer still owns its own *projection*: which fields are classified | ||
| //! sensitive is a property of that analyzer's contract, not of this module. | ||
| //! | ||
| //! Masking is a pure function of the masked text, so the same input always | ||
| //! produces the same token and two records that mentioned the same user still | ||
| //! visibly mention the same user. The projection is idempotent: replacement | ||
| //! tokens cannot themselves match a rule. | ||
|
|
||
| use std::sync::OnceLock; | ||
|
|
||
| use regex::Regex; | ||
|
|
||
| /// FNV-1a. Stable across runs and platforms, which `DefaultHasher` is not. | ||
| fn stable_token(kind: &str, value: &str) -> String { | ||
| let mut hash: u64 = 0xcbf2_9ce4_8422_2325; | ||
| for byte in value.as_bytes() { | ||
| hash ^= u64::from(*byte); | ||
| hash = hash.wrapping_mul(0x0000_0100_0000_01b3); | ||
| } | ||
| format!("[{kind}:{:016x}]", hash) | ||
| } | ||
|
|
||
| fn upn_re() -> &'static Regex { | ||
| static CELL: OnceLock<Regex> = OnceLock::new(); | ||
| CELL.get_or_init(|| { | ||
| Regex::new(r"[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}") | ||
| .expect("UPN regex must compile") | ||
| }) | ||
| } | ||
|
|
||
| /// The profile segment of a user path, in either slash direction. | ||
| /// | ||
| /// The separator may be doubled. These logs embed JSON payloads, and a Windows | ||
| /// path inside one arrives JSON-escaped as `C:\\Users\\Someone`; requiring a | ||
| /// single separator let every such path through unmasked. | ||
| fn user_path_re() -> &'static Regex { | ||
| static CELL: OnceLock<Regex> = OnceLock::new(); | ||
| CELL.get_or_init(|| { | ||
| // The segment is bounded by the path separator, a quote, or the end of | ||
| // the line -- not by whitespace. Windows permits spaces in profile | ||
| // directory names, and bounding on `\s` masked only the first word: | ||
| // `C:\Users\John Doe\...` leaked "Doe" into the export. | ||
| // | ||
| // The leading `[` exclusion is what makes the projection idempotent: | ||
| // an already-masked `[user:...]` segment must not be masked again. | ||
| Regex::new( | ||
| r"(?i)(?P<prefix>[\\/]{1,2}Users[\\/]{1,2})(?P<user>[^\\/\r\n\x22\[][^\\/\r\n\x22]*)", | ||
| ) | ||
| .expect("user path regex must compile") | ||
| }) | ||
| } | ||
|
|
||
| /// A command line or credential value supplied inline after a flag. | ||
| /// | ||
| /// Two properties matter here and both were bugs before: | ||
| /// | ||
| /// * The value must stop at a line break rather than at the end of the string. | ||
| /// A CCM record is one *logical* record and routinely contains newlines (see | ||
| /// the `multiline-ccm-record` fixture). Anchoring on `$` meant a `-Command` | ||
| /// inside a multi-line record matched nothing at all and leaked in full. | ||
| /// * The flag vocabulary covers credential-bearing switches, not just | ||
| /// `-Command`. A launch record reading `-Password hunter2` is exactly as | ||
| /// sensitive as an inline command and was previously exported verbatim. | ||
| fn command_line_re() -> &'static Regex { | ||
| static CELL: OnceLock<Regex> = OnceLock::new(); | ||
| CELL.get_or_init(|| { | ||
| Regex::new( | ||
| r"(?i)(?P<flag>-(?:Command|EncodedCommand|Password|Secret|ClientSecret|Token|AccessToken|ApiKey|Api[-_]?Key|Credential|Authorization)\s+)(?P<value>[^\r\n]+)", | ||
| ) | ||
| .expect("command line regex must compile") | ||
| }) | ||
| } | ||
|
|
||
| /// Mask the sensitive spans inside a free-text value. | ||
| pub fn redact_text(value: &str) -> String { | ||
| let masked = upn_re().replace_all(value, |caps: ®ex::Captures<'_>| { | ||
| stable_token("upn", &caps[0]) | ||
| }); | ||
|
|
||
| let masked = user_path_re().replace_all(&masked, |caps: ®ex::Captures<'_>| { | ||
| // Trailing whitespace is not part of the profile name; keeping it out | ||
| // of the hashed value means `C:\Users\John Doe ` and `C:\Users\John Doe` | ||
| // still resolve to the same user. | ||
| let user = caps["user"].trim_end(); | ||
| let trailing = &caps["user"][user.len()..]; | ||
| format!( | ||
| "{}{}{}", | ||
| &caps["prefix"], | ||
| stable_token("user", user), | ||
| trailing | ||
| ) | ||
| }); | ||
|
|
||
| command_line_re() | ||
| .replace_all(&masked, |caps: ®ex::Captures<'_>| { | ||
| let value = caps["value"].trim_end(); | ||
| // Already masked: re-masking would hash the token and break | ||
| // idempotence. | ||
| if value.starts_with("[command:") && value.ends_with(']') { | ||
| return format!("{}{}", &caps["flag"], value); | ||
| } | ||
| format!("{}{}", &caps["flag"], stable_token("command", value)) | ||
| }) | ||
| .into_owned() | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::redact_text; | ||
|
|
||
| #[test] | ||
| fn a_json_escaped_windows_path_is_masked() { | ||
| // These logs embed JSON payloads; a path inside one is escaped. | ||
| let redacted = redact_text(r#"{"Path":"C:\\Users\\John Doe\\AppData\\Local"}"#); | ||
| assert!(!redacted.contains("John"), "got {redacted:?}"); | ||
| assert!(!redacted.contains("Doe"), "got {redacted:?}"); | ||
| assert!(redacted.contains(r"AppData"), "got {redacted:?}"); | ||
| } | ||
|
|
||
| #[test] | ||
| fn a_json_escaped_path_mask_is_idempotent() { | ||
| let once = redact_text(r#"{"Path":"C:\\Users\\John Doe\\AppData"}"#); | ||
| assert_eq!(once, redact_text(&once)); | ||
| } | ||
|
|
||
| #[test] | ||
| fn a_single_separator_path_is_still_masked() { | ||
| let redacted = redact_text(r"C:\Users\adele.vance\a.ps1"); | ||
| assert!(!redacted.contains("adele.vance")); | ||
| assert!(redacted.ends_with(r"\a.ps1")); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
37 changes: 37 additions & 0 deletions
37
crates/cmtraceopen-parser/src/intune/apps/windows/remediations/mod.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| //! Intune Windows remediation (detection + remediation pair) evidence. | ||
| //! | ||
| //! This is a **semantic analyzer over supplied IME evidence**, not a log | ||
| //! format. Raw records are framed by the shared CCM parser first | ||
| //! ([`crate::intune::ime_parser`]); only then are signals classified and | ||
| //! reduced. The module performs no I/O of any kind. | ||
| //! | ||
| //! Remediations have a deliberately separate public lifecycle from platform | ||
| //! scripts (issue #359) because they are a *pair*: detection decides whether | ||
| //! anything is wrong, and remediation runs only if it was. | ||
| //! | ||
| //! The one rule everything else rests on: **an exit code with no stage | ||
| //! terminates nothing.** `0` means "compliant" from a detection script and | ||
| //! "succeeded" from a remediation script. Attributing a code to the wrong half | ||
| //! inverts the diagnosis, so a record that does not name its stage yields no | ||
| //! exit token at all. | ||
| //! | ||
| //! What the analyzer will not do: | ||
| //! | ||
| //! - pool detection and remediation outcomes, or read one stage's exit code | ||
| //! with the other stage's semantics; | ||
| //! - join a detection to a remediation because they occurred near each other; | ||
| //! - treat a missing output artifact as proof that a script produced no output; | ||
| //! - repair a malformed embedded JSON payload, or concatenate payload fragments | ||
| //! across records to close a brace. | ||
|
|
||
| mod models; | ||
| mod redaction; | ||
| mod reducer; | ||
| mod rules; | ||
| mod sources; | ||
|
|
||
| pub use models::*; | ||
| pub use redaction::redacted_export_projection; | ||
| pub use reducer::analyze_remediation_bundle; | ||
| pub use rules::{classify_record, RecordClassification}; | ||
| pub use sources::{candidate_source_kind, classify_artifact, RemediationSourceInput}; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.