Skip to content

athena audit logs - single consumer on auth#25639

Merged
tobiaszheller merged 1 commit intomasterfrom
tobiaszheller/auditevents-athena-single-auth-consumer-v3
May 15, 2023
Merged

athena audit logs - single consumer on auth#25639
tobiaszheller merged 1 commit intomasterfrom
tobiaszheller/auditevents-athena-single-auth-consumer-v3

Conversation

@tobiaszheller
Copy link
Copy Markdown
Contributor

@tobiaszheller tobiaszheller commented May 4, 2023

Part of https://github.com/gravitational/teleport.e/issues/894
RFD: https://github.com/gravitational/teleport/blob/master/rfd/0118-scalable-audit-logs.md

This PR enable running consumer on single auth instance using object locking.

in 7270f8b
there are changes to runWhileLocked: passing config and releasing lock with ctx.background.

Fixes #15210

@github-actions github-actions Bot requested review from camscale and timothyb89 May 4, 2023 16:18
@tobiaszheller tobiaszheller requested a review from rosstimothy May 4, 2023 16:19
@tobiaszheller
Copy link
Copy Markdown
Contributor Author

@mdwn you may be interested in review just this single commit: 7270f8b

@github-actions github-actions Bot added audit-log Issues related to Teleports Audit Log size/md labels May 4, 2023
@tobiaszheller tobiaszheller changed the title athena audit logs - single auth athena audit logs - single consumer on auth May 5, 2023
Comment thread lib/backend/helpers.go Outdated
Comment thread lib/backend/helpers.go
Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
@tobiaszheller tobiaszheller requested a review from camscale May 9, 2023 09:36
Comment thread lib/auth/auth.go Outdated
Comment thread lib/backend/helpers.go Outdated
Comment thread lib/backend/helpers.go Outdated
Comment thread lib/backend/helpers.go Outdated
Comment thread lib/backend/helpers.go Outdated
Comment thread lib/events/athena/athena.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
@tobiaszheller tobiaszheller requested a review from rosstimothy May 11, 2023 10:26
@tobiaszheller
Copy link
Copy Markdown
Contributor Author

@rosstimothy @camscale @timothyb89 friendly ping for review

Comment thread lib/events/athena/consumer.go Outdated
Comment thread lib/events/athena/consumer.go Outdated
@public-teleport-github-review-bot public-teleport-github-review-bot Bot removed the request for review from timothyb89 May 15, 2023 08:13
@tobiaszheller tobiaszheller force-pushed the tobiaszheller/auditevents-athena-single-auth-consumer-v3 branch from bba942b to 05424c4 Compare May 15, 2023 09:25
@tobiaszheller tobiaszheller enabled auto-merge May 15, 2023 09:26
@tobiaszheller tobiaszheller added this pull request to the merge queue May 15, 2023
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 15, 2023
@tobiaszheller tobiaszheller added this pull request to the merge queue May 15, 2023
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 15, 2023
@tobiaszheller tobiaszheller added this pull request to the merge queue May 15, 2023
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 15, 2023
@tobiaszheller tobiaszheller added this pull request to the merge queue May 15, 2023
Merged via the queue into master with commit e9d397e May 15, 2023
@tobiaszheller tobiaszheller deleted the tobiaszheller/auditevents-athena-single-auth-consumer-v3 branch May 15, 2023 10:21
@public-teleport-github-review-bot
Copy link
Copy Markdown

@tobiaszheller See the table below for backport results.

Branch Result
branch/v13 Failed

@public-teleport-github-review-bot
Copy link
Copy Markdown

@tobiaszheller See the table below for backport results.

Branch Result
branch/v13 Failed

@public-teleport-github-review-bot
Copy link
Copy Markdown

@tobiaszheller See the table below for backport results.

Branch Result
branch/v13 Create PR

rosstimothy added a commit that referenced this pull request Jul 27, 2023
rosstimothy added a commit that referenced this pull request Jul 27, 2023
github-merge-queue Bot pushed a commit that referenced this pull request Jul 28, 2023
* Add configuration with optional timeout to AcquireLock (#24559)

* Add configuration with optional timeout to AcquireLock

* rename to RetryInterval

* backport RunWhileLocked changes from #25639

* Hold Auth init lock for the duration of initialization (#29593)

Auth now uses `RunWhileLocked` instead of `AcquireLock` to ensure
that the initialization lock is held until the bootstrapping
process is completed. Prior, Auth only held the lock for 30s which
could allow multiple Auths to attempt bootstrapping simultaneously.
Initialization should complete prior to 30s in most cases, but it
is not guarateed, especially on first boot when CAs are being
generated and a large data migration may be needed.

---------

Co-authored-by: Tobiasz Heller <14020794+tobiaszheller@users.noreply.github.com>
github-merge-queue Bot pushed a commit that referenced this pull request Jul 28, 2023
* Add configuration with optional timeout to AcquireLock (#24559)

* Add configuration with optional timeout to AcquireLock

* rename to RetryInterval

* backport RunWhileLocked changes from #25639

* Hold Auth init lock for the duration of initialization (#29593)

Auth now uses `RunWhileLocked` instead of `AcquireLock` to ensure
that the initialization lock is held until the bootstrapping
process is completed. Prior, Auth only held the lock for 30s which
could allow multiple Auths to attempt bootstrapping simultaneously.
Initialization should complete prior to 30s in most cases, but it
is not guarateed, especially on first boot when CAs are being
generated and a large data migration may be needed.

---------

Co-authored-by: Tobiasz Heller <14020794+tobiaszheller@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

audit-log Issues related to Teleports Audit Log size/md

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TeleportProcess.auditLog is never closed

4 participants