Skip to content

Feat/495 ttl drift alerting v2 - #664

Closed
Gabugo-tech wants to merge 2 commits into
TegoLabs:mainfrom
Gabugo-tech:feat/495-ttl-drift-alerting-v2
Closed

Gabugo-tech wants to merge 2 commits into
TegoLabs:mainfrom
Gabugo-tech:feat/495-ttl-drift-alerting-v2

Conversation

@Gabugo-tech

Copy link
Copy Markdown

feat(core): implement TTL-drift alerting when actual TTL diverges from policy target
Closes #495

Summary
After every successful auto-extension, the actual post-extension TTL is compared against the policy's target_ttl_ledgers. When the absolute delta exceeds a configurable tolerance, exactly one ttl_drift alert is fired per extension through the existing dispatcher — no new delivery infrastructure required.

Changes
types.ts
Added TTLDriftAlertEvent interface (type: "ttl_drift") with fields: targetTTLLedgers, actualTTLLedgers, driftLedgers, toleranceLedgers, txHash, detectedAtLedger
Added buildTTLDriftAlertEvent builder function
Extended AlertEvent union and AlertEventType to include "ttl_drift"
discord.ts
/ slack.ts / telegram.ts / pagerduty.ts / opsgenie.ts / templates.ts
Added explicit ttl_drift branches in all channel formatters so the new event type renders correctly and TypeScript does not error on missing entry/threshold fields
extension.ts
Added DEFAULT_DRIFT_TOLERANCE_LEDGERS = 100 constant (exported)
Added driftToleranceLedgers param to runAutoExtensions (default: 100)
Added optional channels param to runAutoExtensions for test injection
Added optional driftLedgers?: number to ExtensionResult and AutoExtensionResult
Drift is computed inside extendEntries right after getEntryTTLs and persisted with the extension record
Post-extension drift-check in runAutoExtensions fires TTLDriftAlertEvent via deliverSingleAlert when |drift| > tolerance, isolated in its own try/catch so delivery failures never roll back a completed extension
repositories.ts
Added drift_ledgers?: number | null to the recordExtension contract and INSERT statement
002_extension_history_drift_ledgers.sql
ALTER TABLE extension_history ADD COLUMN drift_ledgers INTEGER
extension.test.ts
Added insertAlertConfig import
Two new tests in the runAutoExtensions describe block:
TTL within tolerance (drift=50, tolerance=100) → no send call
TTL outside tolerance (drift=−2000, tolerance=100) → exactly one send call with correct ttl_drift payload
Acceptance Criteria
A resulting TTL within tolerance of the target does not fire a drift alert
A resulting TTL outside tolerance fires exactly one drift alert per extension
Scope
Only the files described in issue #495 were touched.
dispatcher.ts
and
registry.ts
were not modified — drift alerts flow through the existing dispatcher unchanged.

closes #495

@coderabbitai

coderabbitai Bot commented Aug 4, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Gabugo-tech, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 30 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b76f5eb3-5b99-4d6e-b758-798a1471fe09

📥 Commits

Reviewing files that changed from the base of the PR and between 444c2b9 and 38be2f3.

📒 Files selected for processing (3)
  • src/alerts/templates.ts
  • src/core/extension.ts
  • tests/core/extension.test.ts
📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Added TTL-drift monitoring for automatic extensions.
    • Extension results now show the difference between target and actual TTL.
    • Configured alert channels can notify you when drift exceeds the allowed tolerance.
    • Added warning alerts with transaction, ledger, target, actual, and tolerance details.
  • Bug Fixes
    • Alert delivery failures are isolated so they do not interrupt extension processing.
  • Data Updates
    • Extension history now records TTL-drift information for future review.

Walkthrough

Changes

The extension workflow now computes signed TTL drift after successful extensions, stores the value in extension_history, and includes it in results. Configurable tolerance controls warning alert creation through the existing alert channels. Tests cover tolerated and excessive drift.

TTL drift alerting

Layer / File(s) Summary
TTL drift alert contract
src/alerts/types.ts
Adds the ttl_drift event type, event interface, and warning-severity event builder.
Drift computation and persistence
src/core/extension.ts, src/db/migrations/002_extension_history_drift_ledgers.sql, src/db/repositories.ts
Computes signed drift, returns it in extension results, and persists nullable drift_ledgers values.
Tolerance checks and alert dispatch
src/core/extension.ts, tests/core/extension.test.ts
Applies configurable tolerance, dispatches configured webhook alerts for excessive drift, and validates both alert and non-alert cases.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant runAutoExtensions
  participant extendEntries
  participant ExtensionHistory
  participant AlertDispatcher
  participant Webhook
  runAutoExtensions->>extendEntries: execute extension and fetch fresh TTL
  extendEntries->>extendEntries: calculate signed TTL drift
  extendEntries->>ExtensionHistory: persist drift_ledgers
  extendEntries-->>runAutoExtensions: return driftLedgers
  runAutoExtensions->>runAutoExtensions: compare drift with tolerance
  runAutoExtensions->>AlertDispatcher: dispatch ttl_drift event when excessive
  AlertDispatcher->>Webhook: send configured alert
Loading

Possibly related PRs

Suggested reviewers: abdulmalikalayande

Poem

A rabbit checks the ledger’s trail,
Finds TTL drift beyond the scale.
It stores the gap, then sounds the call,
While steady extensions alert none at all.
“Hop safely onward,” says the hare.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The pull request changes src/db/repositories.ts, which is outside the files explicitly allowed by issue #495. Limit changes to the files allowed by issue #495, or obtain maintainer approval and update the issue scope before merging.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies TTL-drift alerting and references issue 495, matching the primary change.
Description check ✅ Passed The description directly explains TTL-drift detection, persistence, alert dispatch, configuration, and tests.
Linked Issues check ✅ Passed The implementation covers configurable drift detection, persistence, alert dispatch, and both required acceptance tests for issue #495.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…m policy target (TegoLabs#495)

- Add TTLDriftAlertEvent interface and buildTTLDriftAlertEvent builder to alerts/types.ts
- Extend AlertEvent union and AlertEventType with 'ttl_drift'
- Add DEFAULT_DRIFT_TOLERANCE_LEDGERS = 100 constant to extension.ts
- Add driftToleranceLedgers + channels params to runAutoExtensions
- Add driftLedgers field to ExtensionResult and AutoExtensionResult
- Compute and persist drift_ledgers in extendEntries alongside extension record
- Add drift_ledgers to recordExtension in repositories.ts
- Post-extension drift check fires TTLDriftAlertEvent when abs(drift) > tolerance
- Migration 002: ALTER TABLE extension_history ADD COLUMN drift_ledgers INTEGER
- Tests: within-tolerance no alert; outside-tolerance exactly one alert per config
@Gabugo-tech
Gabugo-tech force-pushed the feat/495-ttl-drift-alerting-v2 branch from 2cb4e66 to 444c2b9 Compare August 4, 2026 11:24
@Gabugo-tech

Gabugo-tech commented Aug 4, 2026 •

Copy link
Copy Markdown
Author

@AbdulmalikAlayande please come and review this pr and merge it

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/alerts/types.ts`:
- Line 4: Update the alert routing and rendering logic for the "ttl_drift"
AlertEventType, especially deliverSingleAlert and the relevant helpers in
types.ts and templates.ts, so TTLDriftAlertEvent uses its drift-specific entry
label, template flags, isTTLAlert classification, and custom/template fields
instead of generic fallbacks. Preserve existing behavior for all other alert
event types and ensure every configured channel receives the populated TTL-drift
data.

In `@src/core/extension.ts`:
- Around line 254-260: Update the drift handling in extendEntries: calculate and
persist each entry’s signed drift against extendToLedgers in its corresponding
history row instead of using the aggregate Math.max result. Track the per-entry
drift with the largest absolute value solely for the extension-level alert,
preserving signed values in storage. Add a mixed-entry regression test verifying
both per-entry stored drifts and that the alert is emitted.

In `@tests/core/extension.test.ts`:
- Around line 887-1003: The drift-alert tests only cover successful delivery;
add a rejected-delivery case around runAutoExtensions using the existing setup
and mockChannel, with send configured to reject. Assert the completed extension
remains successful by checking contractsExtended, entriesExtended, and
extensions still report the extension, while preserving the existing drift
behavior.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d2c1c3ed-6972-4231-999d-f532c4f3e79e

📥 Commits

Reviewing files that changed from the base of the PR and between 1e4a880 and 444c2b9.

📒 Files selected for processing (5)
  • src/alerts/types.ts
  • src/core/extension.ts
  • src/db/migrations/002_extension_history_drift_ledgers.sql
  • src/db/repositories.ts
  • tests/core/extension.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: GitGuardian Security Checks
🧰 Additional context used
🪛 Squawk (2.61.0)
src/db/migrations/002_extension_history_drift_ledgers.sql

[warning] 7-7: Using 32-bit integer fields can result in hitting the max int limit. Use 64-bit integer values instead to prevent hitting this limit.

(prefer-bigint-over-int)

🔇 Additional comments (7)
src/alerts/types.ts (1)

199-256: LGTM!

src/core/extension.ts (3)

16-34: LGTM!

Also applies to: 90-91, 106-106


318-319: 🗄️ Data Integrity & Integration

No change needed.

Current callers do not pass a channel map in the driftToleranceLedgers position; channel maps are passed after that argument, so existing call sites do not have this drift-alert suppression path.

			> Likely an incorrect or invalid review comment.

479-489: 🔒 Security & Privacy

Sensitive Data Exposure (CWE-319): Cleartext Transmission of Sensitive Information

Reachability path
● Entry
  tests/core/extension.test.ts
│
▼
● Sink
  src/core/extension.ts

Verify secret-safe delivery for the new alert path.

cfg.channel_target and cfg.webhook_secret flow from alert_configs through deliverSingleAlert to AlertChannel.send. If an attacker can configure an alert and a channel permits HTTP or forwards the secret after a redirect, that endpoint receives the secret. The only supplied control resolves a channel type. Verify destination authorization, HTTPS enforcement, redirect handling, and secret forwarding in each channel implementation.

#!/bin/bash
set -euo pipefail

ast-grep outline src/alerts --items all --type class,function

rg -nP -C 8 \
  '\b(send|channel_target|webhook_secret|redirect|Authorization|signature)\b|https?://' \
  src/alerts

rg -nP -C 8 \
  '\b(insertAlertConfig|updateAlertConfig|deleteAlertConfig|channel_target|webhook_secret)\b' \
  src
tests/core/extension.test.ts (1)

6-6: LGTM!

src/db/migrations/002_extension_history_drift_ledgers.sql (1)

7-7: 🩺 Stability & Availability

No change needed.

Fresh and test databases apply getDatabaseForTesting() and migrate through the migrations directory before recordExtension can write. Existing databases also run migrations from that directory before auto-extension writes.

src/db/repositories.ts (1)

410-421: 🗄️ Data Integrity & Integration

No read-contract change needed.

getExtensionHistory() returns raw extension-history rows via SELECT *, and ExtensionRecord already contains all existing columns, including drift_ledgers.

Comment thread src/alerts/types.ts
Comment thread src/core/extension.ts Outdated
Comment on lines +254 to +260
// Compute drift here so it can be persisted alongside the extension record.
let driftLedgers: number | undefined;
if (freshTTLs.entries.length > 0) {
const maxActualTTL = Math.max(...freshTTLs.entries.map(e => e.remainingTTL));
driftLedgers = maxActualTTL - extendToLedgers;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Compute and persist drift per entry before aggregate alerting.

extendEntries can process multiple entries. Math.max selects the healthy entry TTL and writes its aggregate drift to every history row. If one entry is 1,000 ledgers short and another reaches the target, this code stores 0 for both and sends no alert. Calculate each row’s signed drift. Use the largest absolute per-entry drift only for the one extension-level alert. Add a mixed-entry regression test that checks both stored values and the alert.

Proposed fix
-    let driftLedgers: number | undefined;
-    if (freshTTLs.entries.length > 0) {
-        const maxActualTTL = Math.max(...freshTTLs.entries.map(e => e.remainingTTL));
-        driftLedgers = maxActualTTL - extendToLedgers;
-    }
+    const driftLedgers = freshTTLs.entries.reduce<number | undefined>(
+        (worst, freshEntry) => {
+            const candidate = freshEntry.remainingTTL - extendToLedgers;
+            return worst === undefined || Math.abs(candidate) > Math.abs(worst)
+                ? candidate
+                : worst;
+        },
+        undefined,
+    );
@@
-                drift_ledgers: driftLedgers ?? null,
+                drift_ledgers: freshEntry.remainingTTL - extendToLedgers,

Also applies to: 280-280, 309-309

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/core/extension.ts` around lines 254 - 260, Update the drift handling in
extendEntries: calculate and persist each entry’s signed drift against
extendToLedgers in its corresponding history row instead of using the aggregate
Math.max result. Track the per-entry drift with the largest absolute value
solely for the extension-level alert, preserving signed values in storage. Add a
mixed-entry regression test verifying both per-entry stored drifts and that the
alert is emitted.

Comment thread tests/core/extension.test.ts
)

- templates.ts: add ttl_drift branch for entryLabel, dedupKey, customDetails,
  and isDriftAlert template flag; guard entryLabel assignment with type check
- extension.ts: compute per-entry drift in recordExtension rows instead of
  aggregate Math.max; track largest-absolute-value drift for the alert
- tests: add mixed-entry regression test verifying per-entry stored drifts
  and correct alert drift value; add rejected-delivery test verifying that
  a failed alert dispatch does not surface as an extension failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(core): implement TTL-drift alerting when actual TTL diverges from policy target unexpectedly

1 participant