Implement Compaction of Desktop Shared Directory Read/Write Events#57815
Merged
rhammonds-teleport merged 1 commit intomasterfrom Aug 22, 2025
Merged
Implement Compaction of Desktop Shared Directory Read/Write Events#57815rhammonds-teleport merged 1 commit intomasterfrom
rhammonds-teleport merged 1 commit intomasterfrom
Conversation
rhammonds-teleport
force-pushed
the
rhammonds/desktop-audit-combiner
branch
from
f5b97f6 to
9cd3c2a
Compare
rhammonds-teleport
changed the title
zmb3
reviewed
Aug 14, 2025
| } | ||
| } | ||
|
|
||
| func (s *stream) compactEvents() []streamEvent { |
Collaborator
There was a problem hiding this comment.
Would it make sense to return an iter.Seq here instead?
zmb3
approved these changes
Aug 14, 2025
| } | ||
| case tdp.SharedDirectoryReadResponse: | ||
| s.emit(ctx, audit.makeSharedDirectoryReadResponse(msg)) | ||
| audit.compactor.handleRead(ctx, audit.makeSharedDirectoryReadResponse(msg)) |
Collaborator
There was a problem hiding this comment.
I might just add a comment to these lines that says something like:
// shared directory audit events can be noisy, so we use a compactor
// in order to bucket them instead of writing straight to the audit log
Collaborator
There was a problem hiding this comment.
Same goes for handleWrite below.
Comment on lines
+65
to
+72
| refreshInterval time.Duration | ||
| maxDelayInterval time.Duration |
Collaborator
There was a problem hiding this comment.
What's the difference between these two?
Contributor
Author
There was a problem hiding this comment.
I added some comments to clarify. refreshInterval defines how long to wait for the next event to arrive before flushing a given bucket. maxDelayInterval defines the maximum time a bucket should exist before flushing. Basically the latter prevents some pathological read/write scenario from delaying audit events indefinitely.
Collaborator
There was a problem hiding this comment.
I see, so event A and event B will be combined assuming they overlap and occur within refreshInterval, so long as the first event in this bucket occurred less than maxDelayInterval ago.
Contributor
Author
There was a problem hiding this comment.
That's right!
zmb3
approved these changes
Aug 15, 2025
Collaborator
|
Don't forget to link this PR to the issue it closes. |
probakowski
approved these changes
Aug 22, 2025
public-teleport-github-review-bot
bot
removed the request for review
from tcsc
rhammonds-teleport
force-pushed
the
rhammonds/desktop-audit-combiner
branch
from
2750f20 to
ba118c6
Compare
danielashare
approved these changes
Aug 22, 2025
…eads and writes Attempt #2. Read requests for a given copy operation can arrive out of order. Adjusted this approach to record all read/write events within a given time period, then find the longest contiguous set of reads/writes that can be joined into a single audit event before emitting audit event(s). bit of cleanup A bit of code cleanup plus extra test case. Removed timestamps from testing since the compaction algorithm can't really guarantee which segments will get grouped together. Fix racy tests by bringing assertions into synctest bubble and using 'flush' for syncronization. Also add more thorough test cases for timer expiration and flush behavior. lint fixes Add license header Replace unnecessary sort with 'slices.Min' Apply suggestions from code review Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Rename 'evnt' -> 'event' Return iterator instead of slice Abandon 'stream' terminology in favor of 'fileOperationsBucket'. This might be a better name since the code is basically sorting audit events into a set of 'buckets' and attempting to compact each bucket of events later on. Add a few comments
rhammonds-teleport
force-pushed
the
rhammonds/desktop-audit-combiner
branch
from
ba118c6 to
e470056
Compare
Contributor
|
@rhammonds-teleport See the table below for backport results.
|
This was referenced Aug 28, 2025
mmcallister
pushed a commit
that referenced
this pull request
Sep 22, 2025
…eads and writes (#57815) Attempt #2. Read requests for a given copy operation can arrive out of order. Adjusted this approach to record all read/write events within a given time period, then find the longest contiguous set of reads/writes that can be joined into a single audit event before emitting audit event(s). bit of cleanup A bit of code cleanup plus extra test case. Removed timestamps from testing since the compaction algorithm can't really guarantee which segments will get grouped together. Fix racy tests by bringing assertions into synctest bubble and using 'flush' for syncronization. Also add more thorough test cases for timer expiration and flush behavior. lint fixes Add license header Replace unnecessary sort with 'slices.Min' Apply suggestions from code review Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Rename 'evnt' -> 'event' Return iterator instead of slice Abandon 'stream' terminology in favor of 'fileOperationsBucket'. This might be a better name since the code is basically sorting audit events into a set of 'buckets' and attempting to compact each bucket of events later on. Add a few comments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copying files into or out of the shared directory often creates many small read/write operations that clutter the audit log. This PR aims to implement an algorithm that compacts audit events resulting from reads/writes to files within a given shared directory.
The audit compactor caches subsequent reads/writes to a given file for some configurable period of time. Once this timeout expires, audit entries are examined to determine which operations constitute sequential reads/writes that could be compacted into a single audit event. The reduced set of audit events are then emitted.
Closes #40341
Changelog: Reduce audit log clutter by compacting contiguous shared directory read/write events into a single audit log event.