feat(proxy): persist proxy logs to SQLite - #53
Conversation
- Add proxy_logs table to db/core.js schema (18 columns + 3 indexes) - Rewrite proxyLogger.js with hybrid dual storage: - In-memory ring buffer for real-time performance - SQLite persistence for surviving server restarts - On startup, hydrates from DB (last 500 entries) - Each event writes to both memory and SQLite - Auto-trim keeps max 500 rows in DB - Same external API (getProxyLogs, clearProxyLogs, etc.) - Zero changes needed in API route or frontend
|
@codex review |
|
To use Codex here, create a Codex account and connect to github. |
Summary of ChangesHello @diegosouzapw, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant enhancement to how proxy logs are managed by transitioning from ephemeral in-memory storage to a robust hybrid system. By integrating SQLite persistence, the application can now retain proxy log history across restarts, greatly improving debugging and operational visibility without altering existing external interfaces. The change ensures that critical log data is durable while maintaining fast access for real-time monitoring. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces SQLite persistence for proxy logs, a significant improvement for log durability, moving from a purely in-memory solution. The implementation is solid and follows existing patterns in the codebase. My review includes two suggestions to enhance the changes: one to improve database schema integrity by enforcing non-nullable fields for status and level, and another to optimize the database trimming logic for better performance under high load.
| status TEXT, | ||
| proxy_type TEXT, | ||
| proxy_host TEXT, | ||
| proxy_port INTEGER, | ||
| level TEXT, |
There was a problem hiding this comment.
The status and level columns can be NULL, but the application logic in proxyLogger.js always provides default values ('success' and 'direct' respectively). This could mask issues where a log is saved without these values. To ensure data integrity, it's better to enforce this at the database level by making these columns NOT NULL.
| status TEXT, | |
| proxy_type TEXT, | |
| proxy_host TEXT, | |
| proxy_port INTEGER, | |
| level TEXT, | |
| status TEXT NOT NULL, | |
| proxy_type TEXT, | |
| proxy_host TEXT, | |
| proxy_port INTEGER, | |
| level TEXT NOT NULL, |
| // Trim old entries | ||
| const count = db.prepare("SELECT COUNT(*) as cnt FROM proxy_logs").get()?.cnt || 0; | ||
| if (count > MAX_ENTRIES) { | ||
| db.prepare( | ||
| `DELETE FROM proxy_logs WHERE id IN ( | ||
| SELECT id FROM proxy_logs ORDER BY timestamp ASC LIMIT ? | ||
| )` | ||
| ).run(count - MAX_ENTRIES); | ||
| } |
There was a problem hiding this comment.
The current implementation for trimming the database runs a COUNT(*) query and a DELETE query on every single log event. This can become a performance bottleneck, especially under high load, as COUNT(*) requires a full table scan. A more efficient approach would be to perform this cleanup operation periodically rather than on every write.
I suggest making the trimming logic probabilistic, for example, running it on average 1% of the time. This significantly reduces the overhead while still keeping the table size under control.
// Trim old entries periodically to avoid overhead on every write.
// This runs on average once every 100 calls.
if (Math.random() < 0.01) {
const count = db.prepare("SELECT COUNT(*) as cnt FROM proxy_logs").get()?.cnt || 0;
if (count > MAX_ENTRIES) {
db.prepare(
`DELETE FROM proxy_logs WHERE id IN (
SELECT id FROM proxy_logs ORDER BY timestamp ASC LIMIT ?
)`
).run(count - MAX_ENTRIES);
}
}There was a problem hiding this comment.
Pull request overview
Adds durable persistence for proxy request logs by introducing a proxy_logs SQLite table and updating the proxy logger to write-through to SQLite while still maintaining an in-memory ring buffer for fast dashboard reads. This brings proxy logging in line with the existing SQLite-backed call log approach so logs survive server restarts.
Changes:
- Added a
proxy_logstable + indexes to the SQLite schema. - Reworked
src/lib/proxyLogger.jsto hydrate recent logs from SQLite at startup and persist new events while keeping an in-memory ring buffer. - Updated
package-lock.json(version bump + includeswreq-jsentry).
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/lib/proxyLogger.js | Hybrid in-memory + SQLite persistence for proxy logs, with startup hydration and DB trimming. |
| src/lib/db/core.js | Extends SQLite schema with a new proxy_logs table and supporting indexes. |
| package-lock.json | Lockfile updates including wreq-js metadata (engines/platform constraints). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -21,6 +71,7 @@ const proxyLogs = []; | |||
| * @param {string} [entry.error] | |||
| * @param {string} [entry.connectionId] | |||
| * @param {string} [entry.comboId] | |||
There was a problem hiding this comment.
The JSDoc for logProxyEvent was updated (adds tlsFingerprint) but it still omits entry.account, even though account is read from entry and persisted to SQLite. Please document entry.account here to keep the public logger API accurate for callers.
| * @param {string} [entry.comboId] | |
| * @param {string} [entry.comboId] | |
| * @param {string} [entry.account] |
TLS / Certificate Validation (#50) - src/mitm/server.ts: rejectUnauthorized now defaults to true (opt-out via MITM_DISABLE_TLS_VERIFY=1 env var only) Path Injection (#44, #41-#49) - src/mitm/server.ts: safeLogPath() guards log filenames inside LOG_DIR - src/lib/db/backup.ts: path.resolve() anchor + sep/slash guard on backupId - src/shared/services/backupService.ts: safePath() helper anchors toolId and backupId within BACKUP_DIR (prevents path traversal) - src/app/api/cli-tools/codex-profiles/route.ts: safeProfilePath() helper anchors profileId within PROFILES_DIR Prototype Pollution (#18, #19, #20) - src/lib/usage/usageHistory.ts: byModel/byAccount initialized with Object.create(null); assignments guarded with hasOwnProperty checks Missing Workflow Permissions (#1-#8, #53) - .github/workflows/ci.yml: added global permissions: contents: read Dependabot CVE-2026-0540 (#16) - dompurify updated to ^3.3.2 (fixes XSS vulnerability)
Summary
Proxy logs were 100% in-memory — lost on every server restart. This PR adds SQLite persistence following the same pattern as
callLogs.js.Changes
src/lib/db/core.js— Addedproxy_logstable (18 cols + 3 indexes)src/lib/proxyLogger.js— Rewritten with hybrid dual storage:Zero breaking changes
Same external API (
getProxyLogs,clearProxyLogs, etc.). No changes needed in API route or frontend.Verification