support session metadata generation for encrypted sessions by tigrato · Pull Request #60838 · gravitational/teleport

tigrato · 2025-10-30T17:17:21Z

Encrypted sessions do not use the normal uploader flow as the agents encrypt the data themselves and push the encrypted blobs directly to auth server.

This behavior caused the session metadata and summaries generation to be skipped. This PR hooks the session summarization and metadata generation into the encrypted sessions completer - it can be via gRPC call or the local AuditLog.

Since the encrypted session recordings receives the payloads encrypted and parts might be sent to different auth servers in case one disconnects in the middle, we always stream the full parts to retrieve the session.end event. This can be further improved in the future.

This PR also adds support for encrypted session metadata and thumbnails files so all data stored in the recordings backend will be encrypted and users can't download the file and inspect the svg data.

Contributes to #60689

Changelog: Enabled summarization and metadata generation for encrypted session recordings, storing metadata and summaries in encrypted form.

Encrypted sessions do not use the normal uploader flow as the agents encrypt the data themselves and push the encrypted blobs directly to auth server. This behavior caused the session metadata and summaries generation to be skipped. This PR hooks the session summarization and metadata generation into the encrypted sessions completer - it can be via gRPC call or the local AuditLog. Since the encrypted session recordings receives the payloads encrypted and parts might be sent to different auth servers in case one disconnects in the middle, we always stream the full parts to retrieve the `session.end` event. This can be further improved in the future. This PR also adds support for encrypted session metadata and thumbnails files so all data stored in the recordings backend will be encrypted and users can't download the file and inspect the svg data. Contributes to #60689 Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>

rosstimothy

Can you include a manual test plan either in the PR description of in a comment on this PR detailing which configurations you tested in and that all tests pass as expected?

rosstimothy · 2025-10-31T17:51:33Z

+		if err := summarizer.SummarizeSSH(ctx, o); err != nil {
+			s.logger.WarnContext(ctx, "failed to summarize upload", "error", err)
+		}
+		metadataSvc := s.recordingMetadataProvider.Service()


What happens if recordingMetadataProvider is nil?

nothing. Nothing will fail because it will return a no-op service

rosstimothy · 2025-10-31T18:00:53Z

+	err = l.UploadHandler.CompleteUpload(ctx, *upload, streamParts)
+	if err != nil {
+		return trace.Wrap(err, "completing upload")
+	}
+
+	sessionEnd, err := l.findSessionEndEvent(ctx, upload.SessionID)
+	if err != nil || sessionEnd == nil {
+		return nil
+	}
+
+	summarizer := l.AuditLogConfig.SessionSummarizerProvider.SessionSummarizer()
+	switch o := sessionEnd.(type) {
+	case *apievents.SessionEnd:
+		if err := summarizer.SummarizeSSH(ctx, o); err != nil {
+			l.log.WarnContext(ctx, "failed to summarize upload", "error", err)
+		}
+		metadataSvc := l.AuditLogConfig.RecordingMetadataProvider.Service()
+		if !o.EndTime.IsZero() && !o.StartTime.IsZero() {
+			duration := o.EndTime.Sub(o.StartTime)
+			if err := metadataSvc.ProcessSessionRecording(ctx, upload.SessionID, duration); err != nil {
+				l.log.WarnContext(ctx, "failed to process session recording metadata", "error", err)
+			}
+		}
+	case *apievents.DatabaseSessionEnd:
+		if err := summarizer.SummarizeDatabase(ctx, o); err != nil {
+			l.log.WarnContext(ctx, "failed to summarize upload", "error", err)
+		}
+	}

-	return trace.Wrap(l.UploadHandler.CompleteUpload(ctx, *upload, streamParts), "completing upload")
+	return nil
+}
+
+// findSessionEndEvent streams session events to find the session end event for the given session ID.
+// It returns either a SessionEnd or DatabaseSessionEnd event, or nil if none is found.
+func (l *AuditLog) findSessionEndEvent(ctx context.Context, sessionID session.ID) (apievents.AuditEvent, error) {


This is the second time in this PR that this same pattern has been implemented. Is there anything that we can do to avoid the repetition?

I plan to change this code soon to avoid downloading all the sessions to find the session.end event. instead I plan to download only the last parts of the session

rosstimothy · 2025-10-31T20:23:25Z

+package postprocessing
+
+import (
+	"context"
+
+	"github.com/gravitational/trace"
+
+	apievents "github.com/gravitational/teleport/api/types/events"
+	"github.com/gravitational/teleport/lib/auth/recordingmetadata"
+	"github.com/gravitational/teleport/lib/auth/summarizer"
+	"github.com/gravitational/teleport/lib/session"
+)
+
+// SessionPostProcessorConfig is the configuration for the session post-processor.
+type SessionPostProcessorConfig struct {


Thanks for unifying this in a single location! I'm struggling to come up with a good suggestion, naming is hard, but postprocessing.SessionPostProcessorConfig is slightly redundant and a lot to type.

My first instinct was session.PostProcessingConfig but session is overloaded.

rosstimothy

Thanks Tiago!

eriktate · 2025-10-31T16:25:38Z

+		ServerID:  "testcluster",
+		PrintData: []string{"net", "stat"},
+	})
+	returnChan <- events[len(events)-1]


Should we stream at least the last two events to make sure earlier events are discarded?

This is already tested in sessionend_test.go.

backport-bot-workflows · 2025-11-03T12:34:37Z

@tigrato See the table below for backport results.

Branch	Result
branch/v18	Failed

* support session metadata generation for encrypted sessions Encrypted sessions do not use the normal uploader flow as the agents encrypt the data themselves and push the encrypted blobs directly to auth server. This behavior caused the session metadata and summaries generation to be skipped. This PR hooks the session summarization and metadata generation into the encrypted sessions completer - it can be via gRPC call or the local AuditLog. Since the encrypted session recordings receives the payloads encrypted and parts might be sent to different auth servers in case one disconnects in the middle, we always stream the full parts to retrieve the `session.end` event. This can be further improved in the future. This PR also adds support for encrypted session metadata and thumbnails files so all data stored in the recordings backend will be encrypted and users can't download the file and inspect the svg data. Contributes to #60689 Signed-off-by: Tiago Silva <tiago.silva@goteleport.com> * add unit tests for session completer * add unit tests for audit log session completer * move file decrypter to file * handle code review comments * add todo list * remove duplicate code * fix typo * rename --------- Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>

…60945) * support session metadata generation for encrypted sessions Encrypted sessions do not use the normal uploader flow as the agents encrypt the data themselves and push the encrypted blobs directly to auth server. This behavior caused the session metadata and summaries generation to be skipped. This PR hooks the session summarization and metadata generation into the encrypted sessions completer - it can be via gRPC call or the local AuditLog. Since the encrypted session recordings receives the payloads encrypted and parts might be sent to different auth servers in case one disconnects in the middle, we always stream the full parts to retrieve the `session.end` event. This can be further improved in the future. This PR also adds support for encrypted session metadata and thumbnails files so all data stored in the recordings backend will be encrypted and users can't download the file and inspect the svg data. Contributes to #60689 * add unit tests for session completer * add unit tests for audit log session completer * move file decrypter to file * handle code review comments * add todo list * remove duplicate code * fix typo * rename --------- Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>

* support session metadata generation for encrypted sessions Encrypted sessions do not use the normal uploader flow as the agents encrypt the data themselves and push the encrypted blobs directly to auth server. This behavior caused the session metadata and summaries generation to be skipped. This PR hooks the session summarization and metadata generation into the encrypted sessions completer - it can be via gRPC call or the local AuditLog. Since the encrypted session recordings receives the payloads encrypted and parts might be sent to different auth servers in case one disconnects in the middle, we always stream the full parts to retrieve the `session.end` event. This can be further improved in the future. This PR also adds support for encrypted session metadata and thumbnails files so all data stored in the recordings backend will be encrypted and users can't download the file and inspect the svg data. Contributes to #60689 Signed-off-by: Tiago Silva <tiago.silva@goteleport.com> * add unit tests for session completer * add unit tests for audit log session completer * move file decrypter to file * handle code review comments * add todo list * remove duplicate code * fix typo * rename --------- Signed-off-by: Tiago Silva <tiago.silva@goteleport.com>

tigrato added the backport/branch/v18 label Oct 30, 2025

tigrato requested review from eriktate and ryanclark October 30, 2025 17:17

github-actions bot added audit-log Issues related to Teleports Audit Log size/md labels Oct 30, 2025

github-actions bot requested review from marcoandredinis and zmb3 October 30, 2025 17:17

ryanclark approved these changes Oct 30, 2025

View reviewed changes

tigrato force-pushed the tigrato/supportsummariesmetadataencrypted branch from d669043 to 56e1fbf Compare October 30, 2025 18:05

Joerger reviewed Oct 30, 2025

View reviewed changes

Comment thread lib/events/auditlog.go

Comment thread lib/events/complete.go

tigrato added 3 commits October 30, 2025 19:31

add unit tests for session completer

ec3706f

add unit tests for audit log session completer

ba6cfb3

move file decrypter to file

e6987c9

rosstimothy reviewed Oct 31, 2025

View reviewed changes

tigrato added 4 commits October 31, 2025 18:54

handle code review comments

8b65de2

add todo list

e7b0824

remove duplicate code

4f889cd

fix typo

7d85e74

rosstimothy reviewed Oct 31, 2025

View reviewed changes

rename

dc59b3f

rosstimothy approved these changes Oct 31, 2025

View reviewed changes

public-teleport-github-review-bot bot removed request for eriktate, marcoandredinis and zmb3 October 31, 2025 21:14

Joerger approved these changes Oct 31, 2025

View reviewed changes

eriktate approved these changes Oct 31, 2025

View reviewed changes

Merge branch 'master' into tigrato/supportsummariesmetadataencrypted

c6c03e6

tigrato enabled auto-merge November 3, 2025 11:53

tigrato added this pull request to the merge queue Nov 3, 2025

Merged via the queue into master with commit ebc3543 Nov 3, 2025
41 checks passed

tigrato deleted the tigrato/supportsummariesmetadataencrypted branch November 3, 2025 12:33

tigrato mentioned this pull request Nov 3, 2025

[v18] support session metadata generation for encrypted sessions #60945

Merged

Conversation

tigrato commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

rosstimothy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rosstimothy Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

tigrato Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rosstimothy Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

tigrato Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

rosstimothy Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

rosstimothy left a comment

Choose a reason for hiding this comment

Uh oh!

eriktate Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

tigrato Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

backport-bot-workflows bot commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants