Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .changeset/recursive-key-redaction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
"evlog": minor
---

Add recursive key-based redaction to `RedactConfig`. Use `keys` to redact object property names at any nesting depth (e.g. `password` covers `user.password` and `data.a.b.password`), and `keyPatterns` for regex on key names. `auditRedactPreset` now uses `keys` instead of explicit dot-notation paths.

```ts
initLogger({
redact: {
keys: ['password', 'apiKey', 'authorization'],
keyPatterns: [/.*_token$/i],
},
})
```
43 changes: 32 additions & 11 deletions apps/docs/content/2.learn/6.redaction.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,19 +76,37 @@ Built-in patterns use **partial masking** instead of flat `[REDACTED]` — prese

## Configuration

### Key-Based Redaction

Redact fields by **key name at any nesting depth** — no need to know the full dot-notation path. A single `password` entry covers `user.password`, `data.a.b.password`, and every other occurrence:

```typescript
evlog: {
redact: {
keys: ['password', 'apiKey', 'authorization', 'cookie'],
keyPatterns: [/.*_token$/i],
}
}
```

Key-based redaction replaces the **entire value** (including nested objects) with `replacement`. Use `keyPatterns` for regex on property names; use `patterns` when you need regex on **string values**.

This matches the semantics of `auditDiff({ redactPaths: ['password'] })` — the same key-name rules, but applied globally at emit time.

### Custom Paths

Add dot-notation paths to redact specific fields with `[REDACTED]`, on top of the built-in patterns:
Add exact dot-notation paths when you need to target one location only (e.g. hyphenated header keys):

```typescript
evlog: {
redact: {
paths: ['user.password', 'headers.authorization'],
paths: ['headers.x-forwarded-for'],
keys: ['authorization'],
}
}
```

Path-based redaction replaces the **entire value** with the `replacement` string (default `[REDACTED]`), regardless of content.
Path-based redaction replaces the **entire leaf value** with the `replacement` string (default `[REDACTED]`), regardless of content. Unlike `keys`, `paths: ['password']` only redacts a top-level `password` field.

### Selective Built-ins

Expand Down Expand Up @@ -134,22 +152,25 @@ evlog: {
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `redact` | `boolean \| RedactConfig` | `true` in production | Enabled by default in production. `false` to disable. Object for fine-grained control |
| `paths` | `string[]` | `undefined` | Dot-notation paths to redact entirely (e.g. `user.password`) |
| `patterns` | `RegExp[]` | `undefined` | Custom regex patterns. Uses flat `replacement` string |
| `keys` | `string[]` | `undefined` | Object key names to redact at any depth (e.g. `password` → all `password` fields) |
| `keyPatterns` | `RegExp[]` | `undefined` | Regex on object key names at any depth (e.g. `/.*_token$/`) |
| `paths` | `string[]` | `undefined` | Exact dot-notation paths only (e.g. `headers.x-forwarded-for`) |
| `patterns` | `RegExp[]` | `undefined` | Custom regex on string values. Uses flat `replacement` string |
| `builtins` | `false \| string[]` | All enabled | `false` disables built-ins. Array selects specific ones |
| `replacement` | `string` | `'[REDACTED]'` | Replacement string for paths and custom patterns. Built-in patterns use smart masking instead |
| `replacement` | `string` | `'[REDACTED]'` | Replacement for keys, paths, and custom patterns. Built-ins use smart masking instead |

Available built-in names: `creditCard`, `email`, `ipv4`, `phone`, `jwt`, `bearer`, `iban`.

## How It Works

Redaction runs inside the emit pipeline, after the wide event is fully built but before any output:

1. **Path redaction** — targeted fields replaced with `[REDACTED]`
2. **Smart masking** — built-in patterns scan all string values recursively with partial masking
3. **Pattern redaction** — custom regex patterns scan all string values with flat replacement
4. **Console output** — masked event printed to stdout
5. **Drain** — masked event sent to external services
1. **Key redaction** — matching key names at any depth replaced with `[REDACTED]`
2. **Path redaction** — exact dot-notation leaves replaced with `[REDACTED]`
3. **Smart masking** — built-in patterns scan all string values recursively with partial masking
4. **Pattern redaction** — custom regex patterns scan all string values with flat replacement
5. **Console output** — masked event printed to stdout
6. **Drain** — masked event sent to external services

::callout{icon="i-lucide-zap" color="info"}
Redaction runs **after** the HTTP response is sent, so it adds zero latency to your API responses.
Expand Down
7 changes: 3 additions & 4 deletions apps/docs/content/4.use-cases/4.audit/05.compliance.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,14 @@ import { auditRedactPreset } from 'evlog'

initLogger({
redact: {
paths: [
...(auditRedactPreset.paths ?? []),
'user.password',
keys: [
...(auditRedactPreset.keys ?? []),
],
},
})
```

The preset drops `Authorization` / `Cookie` headers and common credential field names (`password`, `token`, `apiKey`, `cardNumber`, `cvv`, `ssn`) wherever they appear inside `audit.changes.before` and `audit.changes.after`.
The preset redacts `authorization`, `cookie`, `set-cookie`, and common credential key names (`password`, `token`, `apiKey`, `cardNumber`, `cvv`, `ssn`) **at any nesting depth** — including inside `audit.changes.before` / `audit.changes.after`.

## GDPR vs append-only

Expand Down
2 changes: 2 additions & 0 deletions apps/docs/content/6.reference/1.configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ initLogger({
| `redact` | `boolean \| RedactConfig` | `true` in production | Enabled by default in production. `false` to disable. Object for fine-grained control. See [Auto-Redaction](/learn/redaction) |
| `drain` | `(ctx: DrainContext) => void` | `undefined` | Drain callback for sending events to external services |

`RedactConfig` fields (when `redact` is an object): `keys` (key names at any depth), `keyPatterns` (regex on key names), `paths` (exact dot-notation), `patterns` (regex on string values), `builtins`, `replacement`. Full table in [Auto-Redaction](/learn/redaction#configuration-reference).

### `minLevel` vs sampling

- **`minLevel`** is a **hard threshold** on the simple `log.*` API: levels below the threshold are never emitted. It does **not** apply to wide events from `useLogger` / `createLogger().emit()` — use **`sampling.rates`** (and tail `keep`) for request volume.
Expand Down
4 changes: 2 additions & 2 deletions apps/docs/skills/build-audit-logs/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,14 +282,14 @@ function authorize(actor, action, resource) {

### Step 5 — Redact

Apply `auditRedactPreset` (or merge it into the existing `RedactConfig`). It drops `Authorization` / `Cookie` headers and common credential field names (`password`, `token`, `apiKey`, `cardNumber`, `cvv`, `ssn`) wherever they appear inside `audit.changes.before` / `audit.changes.after`:
Apply `auditRedactPreset` (or merge it into the existing `RedactConfig`). It redacts `authorization`, `cookie`, `set-cookie`, and common credential key names (`password`, `token`, `apiKey`, `cardNumber`, `cvv`, `ssn`) at any nesting depth — including inside `audit.changes.before` / `audit.changes.after`:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

```ts
import { initLogger, auditRedactPreset } from 'evlog'

initLogger({
redact: {
paths: [...(auditRedactPreset.paths ?? []), 'user.password', 'user.token'],
keys: [...(auditRedactPreset.keys ?? [])],
},
})
```
Expand Down
69 changes: 18 additions & 51 deletions packages/evlog/src/audit.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { AuditActor, AuditActionDefinition, AuditFields, AuditPatchOp, AuditTarget, DrainContext, EnrichContext, FieldContext, RedactConfig, RequestLogger, WideEvent } from './types'
import { createLogger } from './logger'
import { buildKeyMatcher, redactValueByKeys } from './redact'
import { getHeader as getSharedHeader } from './shared/headers'

/**
Expand Down Expand Up @@ -323,18 +324,9 @@ export function auditDiff(
options: AuditDiffOptions = {},
): { before?: unknown, after?: unknown, patch: AuditPatchOp[] } {
const replacement = options.replacement ?? '[REDACTED]'
const redactSet = new Set((options.redactPaths ?? []).map(p => p))
const keyMatcher = buildKeyMatcher(options.redactPaths) ?? (() => false)
const patch: AuditPatchOp[] = []

function isRedacted(path: string): boolean {
if (redactSet.size === 0) return false
if (redactSet.has(path)) return true
for (const p of redactSet) {
if (path.endsWith(`.${p}`)) return true
}
return false
}

function diff(a: unknown, b: unknown, path: string): void {
if (a === b) return

Expand Down Expand Up @@ -363,20 +355,7 @@ export function auditDiff(
}

function redactValue(value: unknown, path: string): unknown {
if (value === null || typeof value !== 'object') {
const segs = path.split('/').filter(Boolean)
const last = segs[segs.length - 1]
if (last && isRedacted(last)) return replacement
return value
}
if (Array.isArray(value)) {
return value.map((v, i) => redactValue(v, `${path}/${i}`))
}
const out: Record<string, unknown> = {}
for (const [k, v] of Object.entries(value as Record<string, unknown>)) {
out[k] = isRedacted(k) ? replacement : redactValue(v, `${path}/${k}`)
}
return out
return redactValueByKeys(value, keyMatcher, replacement, path)
}

diff(before, after, '')
Expand Down Expand Up @@ -867,7 +846,7 @@ function stripIntegrity(event: WideEvent): WideEvent {
* Strict redact preset for audit events.
*
* Combine with the user's existing redact configuration via spread:
* `initLogger({ redact: { paths: [...auditRedactPreset.paths!, ...mine] } })`.
* `initLogger({ redact: { keys: [...auditRedactPreset.keys!, ...mine] } })`.
*
* Hardens PII handling:
* - Drops `Authorization` and `Cookie` headers anywhere they appear.
Expand All @@ -880,31 +859,19 @@ function stripIntegrity(event: WideEvent): WideEvent {
* enough signal to be useful.
*/
export const auditRedactPreset: RedactConfig = {
paths: [
'audit.changes.before.password',
'audit.changes.before.passwordHash',
'audit.changes.before.token',
'audit.changes.before.apiKey',
'audit.changes.before.secret',
'audit.changes.before.accessToken',
'audit.changes.before.refreshToken',
'audit.changes.before.cardNumber',
'audit.changes.before.cvv',
'audit.changes.before.ssn',
'audit.changes.after.password',
'audit.changes.after.passwordHash',
'audit.changes.after.token',
'audit.changes.after.apiKey',
'audit.changes.after.secret',
'audit.changes.after.accessToken',
'audit.changes.after.refreshToken',
'audit.changes.after.cardNumber',
'audit.changes.after.cvv',
'audit.changes.after.ssn',
'headers.authorization',
'headers.cookie',
'headers.set-cookie',
'audit.context.headers.authorization',
'audit.context.headers.cookie',
keys: [
'password',
'passwordHash',
'token',
'apiKey',
'secret',
'accessToken',
'refreshToken',
'cardNumber',
'cvv',
'ssn',
'authorization',
'cookie',
'set-cookie',
],
}
125 changes: 112 additions & 13 deletions packages/evlog/src/redact.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,87 @@ const DEFAULT_REPLACEMENT = '[REDACTED]'

export type Masker = [RegExp, (match: string) => string]

/** Predicate that returns whether an object key name should be fully redacted. */
export type KeyMatcher = (key: string) => boolean

/**
* Build a matcher from exact key names and regex patterns on key names.
* Returns `undefined` when both inputs are empty.
*/
export function buildKeyMatcher(keys?: string[], keyPatterns?: RegExp[]): KeyMatcher | undefined {
const keySet = new Set(keys ?? [])
const patterns = (keyPatterns ?? []).map(cloneRegex)
if (keySet.size === 0 && patterns.length === 0) return undefined

return (key: string) => {
if (keySet.has(key)) return true
for (const pattern of patterns) {
pattern.lastIndex = 0
if (pattern.test(key)) return true
}
return false
}
}

/**
* Redact values whose key names match `matcher`, recursively at any depth.
* Mutates `obj` in place (intended for use on a clone).
*/
export function redactKeysInTree(obj: unknown, matcher: KeyMatcher, replacement: string): void {
if (obj === null || obj === undefined) return

if (Array.isArray(obj)) {
for (const item of obj) {
redactKeysInTree(item, matcher, replacement)
}
return
}

if (typeof obj === 'object') {
const record = obj as Record<string, unknown>
for (const key in record) {
if (matcher(key)) {
record[key] = replacement
} else {
redactKeysInTree(record[key], matcher, replacement)
}
}
}
}

/**
* Return a copy of `value` with key-name matches replaced by `replacement`.
* Used by audit diffs; does not mutate the input.
*
* When `value` is a scalar and `path` is provided, the last JSON Pointer
* segment is checked against `matcher` (for patch leaf values).
*/
export function redactValueByKeys(
value: unknown,
matcher: KeyMatcher,
replacement: string,
path?: string,
): unknown {
if (value === null || typeof value !== 'object') {
if (path) {
const last = path.split('/').filter(Boolean).at(-1)
if (last && matcher(last)) return replacement
}
return value
}

if (Array.isArray(value)) {
return value.map((v, i) => redactValueByKeys(v, matcher, replacement, path ? `${path}/${i}` : undefined))
}

const out: Record<string, unknown> = {}
for (const [k, v] of Object.entries(value as Record<string, unknown>)) {
const childPath = path ? `${path}/${k}` : k
out[k] = matcher(k) ? replacement : redactValueByKeys(v, matcher, replacement, childPath)
}
return out
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}

/**
* Built-in PII detection patterns with smart masking.
* Each builtin preserves just enough signal for debugging while scrubbing PII.
Expand Down Expand Up @@ -148,10 +229,11 @@ function cloneForRedaction(event: Record<string, unknown>): Record<string, unkno
/**
* Redact sensitive data from a wide event without mutating the input.
*
* Returns a deep clone with redaction applied. Three strategies run in order:
* 1. **Path-based**: dot-notation paths — the leaf value is replaced with `replacement`.
* 2. **Masker-based**: built-in patterns with smart partial masking (e.g. `****1111`).
* 3. **Pattern-based**: custom RegExp patterns replaced with `replacement`.
* Returns a deep clone with redaction applied. Four strategies run in order:
* 1. **Key-based**: object key names (and `keyPatterns`) at any nesting depth — full value replacement.
* 2. **Path-based**: exact dot-notation paths — the leaf value is replaced with `replacement`.
* 3. **Masker-based**: built-in patterns with smart partial masking (e.g. `****1111`).
* 4. **Pattern-based**: custom RegExp patterns on string values replaced with `replacement`.
*
* @param event - The wide event object (not mutated).
* @param config - Redaction configuration.
Expand All @@ -161,6 +243,11 @@ export function redactEvent(event: Record<string, unknown>, config: RedactConfig
const clone = cloneForRedaction(event)
const replacement = config.replacement ?? DEFAULT_REPLACEMENT

const keyMatcher = buildKeyMatcher(config.keys, config.keyPatterns)
if (keyMatcher) {
redactKeysInTree(clone, keyMatcher, replacement)
}

if (config.paths?.length) {
for (const path of config.paths) {
redactPath(clone, path.split('.'), replacement)
Expand Down Expand Up @@ -280,6 +367,10 @@ export function normalizeRedactConfig(raw: boolean | Record<string, unknown> | u
config.paths = raw.paths as string[]
}

if (Array.isArray(raw.keys)) {
config.keys = raw.keys as string[]
}

if (typeof raw.replacement === 'string') {
config.replacement = raw.replacement
}
Expand All @@ -291,16 +382,24 @@ export function normalizeRedactConfig(raw: boolean | Record<string, unknown> | u
}

if (Array.isArray(raw.patterns)) {
config.patterns = (raw.patterns as unknown[]).map((p) => {
if (p instanceof RegExp) return p
if (typeof p === 'string') return new RegExp(p, 'g')
if (typeof p === 'object' && p !== null) {
const obj = p as Record<string, string>
return new RegExp(obj.source, obj.flags ?? 'g')
}
return null
}).filter((p): p is RegExp => p !== null)
config.patterns = deserializeRegexList(raw.patterns)
}

if (Array.isArray(raw.keyPatterns)) {
config.keyPatterns = deserializeRegexList(raw.keyPatterns)
}

return resolveRedactConfig(config)
}

function deserializeRegexList(raw: unknown[]): RegExp[] {
return raw.map((p) => {
if (p instanceof RegExp) return p
if (typeof p === 'string') return new RegExp(p, 'g')
if (typeof p === 'object' && p !== null) {
const obj = p as Record<string, string>
return new RegExp(obj.source, obj.flags ?? 'g')
}
return null
}).filter((p): p is RegExp => p !== null)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
}
Loading
Loading