Skip to content

fix provider limits sync cadence #888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rdself wants to merge 1 commit into from
Closed

fix provider limits sync cadence #888

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ ENABLE_REQUEST_LOGS=false
AUTH_COOKIE_SECURE=false
REQUIRE_API_KEY=false
ALLOW_API_KEY_REVEAL=false
PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES=70

# Input Sanitizer (FASE-01 — prompt injection & PII protection)
# INPUT_SANITIZER_ENABLED=true
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ Developers pay $20–200/month for Claude Pro, Codex Pro, or GitHub Copilot. Eve
**How OmniRoute solves it:**

- **Smart 4-Tier Fallback** — If subscription quota runs out, automatically redirects to API Key → Cheap → Free with zero manual intervention
- **Real-Time Quota Tracking** — Shows token consumption in real-time with reset countdown (5h, daily, weekly)
- **Provider Limits Tracking** — Cached quota snapshots refresh on a server-side schedule (default `PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES=70`) with manual refresh available in the UI
- **Multi-Account Support** — Multiple accounts per provider with auto round-robin — when one runs out, switches to the next
- **Custom Combos** — Customizable fallback chains with 6 balancing strategies (fill-first, round-robin, P2C, random, least-used, cost-optimized)
- **Codex Business Quotas** — Business/Team workspace quota monitoring directly in the dashboard
Expand Down
49 changes: 25 additions & 24 deletions docs/USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -507,26 +507,27 @@ post_install() {

### Environment Variables

| Variable | Default | Description |
| -------------------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------ |
| `JWT_SECRET` | `omniroute-default-secret-change-me` | JWT signing secret (**change in production**) |
| `INITIAL_PASSWORD` | `123456` | First login password |
| `DATA_DIR` | `~/.omniroute` | Data directory (db, usage, logs) |
| `PORT` | framework default | Service port (`20128` in examples) |
| `HOSTNAME` | framework default | Bind host (Docker defaults to `0.0.0.0`) |
| `NODE_ENV` | runtime default | Set `production` for deploy |
| `BASE_URL` | `http://localhost:20128` | Server-side internal base URL |
| `CLOUD_URL` | `https://omniroute.dev` | Cloud sync endpoint base URL |
| `API_KEY_SECRET` | `endpoint-proxy-api-key-secret` | HMAC secret for generated API keys |
| `REQUIRE_API_KEY` | `false` | Enforce Bearer API key on `/v1/*` |
| `ALLOW_API_KEY_REVEAL` | `false` | Allow Api Manager to copy full API keys on demand |
| `DISABLE_SQLITE_AUTO_BACKUP` | `false` | Disable automatic SQLite snapshots before writes/import/restore; manual backups still work |
| `ENABLE_REQUEST_LOGS` | `false` | Enables request/response logs |
| `AUTH_COOKIE_SECURE` | `false` | Force `Secure` auth cookie (behind HTTPS reverse proxy) |
| `CLOUDFLARED_BIN` | unset | Use an existing `cloudflared` binary instead of managed download |
| `OMNIROUTE_MEMORY_MB` | `512` | Node.js heap limit in MB |
| `PROMPT_CACHE_MAX_SIZE` | `50` | Max prompt cache entries |
| `SEMANTIC_CACHE_MAX_SIZE` | `100` | Max semantic cache entries |
| Variable | Default | Description |
| --------------------------------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------- |
| `JWT_SECRET` | `omniroute-default-secret-change-me` | JWT signing secret (**change in production**) |
| `INITIAL_PASSWORD` | `123456` | First login password |
| `DATA_DIR` | `~/.omniroute` | Data directory (db, usage, logs) |
| `PORT` | framework default | Service port (`20128` in examples) |
| `HOSTNAME` | framework default | Bind host (Docker defaults to `0.0.0.0`) |
| `NODE_ENV` | runtime default | Set `production` for deploy |
| `BASE_URL` | `http://localhost:20128` | Server-side internal base URL |
| `CLOUD_URL` | `https://omniroute.dev` | Cloud sync endpoint base URL |
| `API_KEY_SECRET` | `endpoint-proxy-api-key-secret` | HMAC secret for generated API keys |
| `REQUIRE_API_KEY` | `false` | Enforce Bearer API key on `/v1/*` |
| `ALLOW_API_KEY_REVEAL` | `false` | Allow Api Manager to copy full API keys on demand |
| `PROVIDER_LIMITS_SYNC_INTERVAL_MINUTES` | `70` | Server-side refresh cadence for cached Provider Limits data; UI refresh buttons still trigger manual sync |
| `DISABLE_SQLITE_AUTO_BACKUP` | `false` | Disable automatic SQLite snapshots before writes/import/restore; manual backups still work |
| `ENABLE_REQUEST_LOGS` | `false` | Enables request/response logs |
| `AUTH_COOKIE_SECURE` | `false` | Force `Secure` auth cookie (behind HTTPS reverse proxy) |
| `CLOUDFLARED_BIN` | unset | Use an existing `cloudflared` binary instead of managed download |
| `OMNIROUTE_MEMORY_MB` | `512` | Node.js heap limit in MB |
| `PROMPT_CACHE_MAX_SIZE` | `50` | Max prompt cache entries |
| `SEMANTIC_CACHE_MAX_SIZE` | `100` | Max semantic cache entries |

For the full environment variable reference, see the [README](../README.md).

Expand Down Expand Up @@ -769,10 +770,10 @@ OmniRoute implements provider-level resilience with four components:

Manage database backups in **Dashboard → Settings → System & Storage**.

| Action | Description |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Export Database** | Downloads the current SQLite database as a `.sqlite` file |
| **Export All (.tar.gz)** | Downloads a full backup archive including: database, settings, combos, provider connections (no credentials), API key metadata |
| Action | Description |
| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| **Export Database** | Downloads the current SQLite database as a `.sqlite` file |
| **Export All (.tar.gz)** | Downloads a full backup archive including: database, settings, combos, provider connections (no credentials), API key metadata |
| **Import Database** | Upload a `.sqlite` file to replace the current database. A pre-import backup is automatically created unless `DISABLE_SQLITE_AUTO_BACKUP=true` |

```bash
Expand Down
139 changes: 56 additions & 83 deletions src/app/(dashboard)/dashboard/usage/components/ProviderLimits/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,8 @@ import { CardSkeleton } from "@/shared/components/Loading";
import { USAGE_SUPPORTED_PROVIDERS } from "@/shared/constants/providers";

const LS_GROUP_BY = "omniroute:limits:groupBy";
const LS_AUTO_REFRESH = "omniroute:limits:autoRefresh";
const LS_EXPANDED_GROUPS = "omniroute:limits:expandedGroups";

const REFRESH_INTERVAL_MS = 120000;
const MIN_FETCH_INTERVAL_MS = 30000; // Debounce per-connection fetches
const QUOTA_BAR_GREEN_THRESHOLD = 50;
const QUOTA_BAR_YELLOW_THRESHOLD = 20;
Expand Down Expand Up @@ -84,13 +82,8 @@ export default function ProviderLimits() {
const [quotaData, setQuotaData] = useState({});
const [loading, setLoading] = useState({});
const [errors, setErrors] = useState({});
const [autoRefresh, setAutoRefresh] = useState(() => {
if (typeof window === "undefined") return false;
return localStorage.getItem(LS_AUTO_REFRESH) === "true";
});
const [lastRefreshedAt, setLastRefreshedAt] = useState<Record<string, string>>({});
const [refreshingAll, setRefreshingAll] = useState(false);
const [countdown, setCountdown] = useState(120);
const [initialLoading, setInitialLoading] = useState(true);
const [tierFilter, setTierFilter] = useState("all");
const [groupBy, setGroupBy] = useState<"none" | "environment">(() => {
Expand All @@ -109,8 +102,6 @@ export default function ProviderLimits() {
}
});

const intervalRef = useRef(null);
const countdownRef = useRef(null);
const lastFetchTimeRef = useRef({});
const staleProbeRef = useRef({});

Expand All @@ -128,6 +119,41 @@ export default function ProviderLimits() {
}
}, []);

const applyCachedQuotaState = useCallback((connectionList, caches) => {
const nextQuotaData = {};
const nextLastRefreshedAt = {};

for (const conn of connectionList) {
const cached = caches?.[conn.id];
if (!cached) continue;

nextQuotaData[conn.id] = {
quotas: parseQuotaData(conn.provider, cached),
plan: cached.plan || null,
message: cached.message || null,
Comment on lines +132 to +133

Copilot AI Apr 1, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plan: cached.plan || null will coerce falsy-but-valid plan values (e.g., 0, false, or empty string) to null. If you only want to default when the field is missing, use nullish coalescing (cached.plan ?? null). Same applies to cached.message if you want to preserve empty strings.

Suggested change
plan: cached.plan || null,
message: cached.message || null,
plan: cached.plan ?? null,
message: cached.message ?? null,

Copilot uses AI. Check for mistakes.
raw: cached,
};

if (cached.fetchedAt) {
nextLastRefreshedAt[conn.id] = cached.fetchedAt;
}
}

setQuotaData(nextQuotaData);
setLastRefreshedAt(nextLastRefreshedAt);
}, []);

const fetchCachedProviderLimits = useCallback(async () => {
try {
const response = await fetch("/api/usage/provider-limits");
if (!response.ok) throw new Error("Failed");
const data = await response.json();
return data.caches || {};
} catch {
return {};
}
}, []);

const fetchQuota = useCallback(
async (connectionId, provider, options: { force?: boolean } = {}) => {
const force = options?.force === true;
Expand Down Expand Up @@ -207,72 +233,39 @@ export default function ProviderLimits() {
const refreshAll = useCallback(async () => {
if (refreshingAll) return;
setRefreshingAll(true);
setCountdown(120);
try {
const conns = await fetchConnections();

// Show table layout immediately once connections are loaded (Issue #784)
setInitialLoading(false);

const usageConnections = conns.filter(
(conn) =>
USAGE_SUPPORTED_PROVIDERS.includes(conn.provider) &&
(conn.authType === "oauth" || conn.authType === "apikey")
);
// Fix: Fetch quotas in chunks of 5 to avoid spamming the backend/provider APIs and hanging the UI.
const chunkSize = 5;
for (let i = 0; i < usageConnections.length; i += chunkSize) {
const chunk = usageConnections.slice(i, i + chunkSize);
await Promise.all(chunk.map((conn) => fetchQuota(conn.id, conn.provider, { force: true })));
const response = await fetch("/api/usage/provider-limits", { method: "POST" });
if (!response.ok) {
const errorData = await response.json().catch(() => ({}));
const errorMsg = errorData.error || response.statusText;
throw new Error(errorMsg);
}

const data = await response.json();
const connectionList = await fetchConnections();
applyCachedQuotaState(connectionList, data.caches || {});
setErrors(data.errors || {});
} catch (error) {
console.error("Error refreshing all:", error);
} finally {
setRefreshingAll(false);
setInitialLoading(false); // Fallback to ensure skeleton is cleared
}
}, [refreshingAll, fetchConnections, fetchQuota]);
}, [refreshingAll, applyCachedQuotaState, fetchConnections]);

useEffect(() => {
const init = async () => {
setInitialLoading(true);
// No longer await refreshAll here so we don't block the UI
refreshAll();
};
init();
}, []); // eslint-disable-line react-hooks/exhaustive-deps

useEffect(() => {
if (!autoRefresh) {
if (intervalRef.current) clearInterval(intervalRef.current);
if (countdownRef.current) clearInterval(countdownRef.current);
return;
}
intervalRef.current = setInterval(refreshAll, REFRESH_INTERVAL_MS);
countdownRef.current = setInterval(() => {
setCountdown((prev) => (prev <= 1 ? 120 : prev - 1));
}, 1000);
return () => {
if (intervalRef.current) clearInterval(intervalRef.current);
if (countdownRef.current) clearInterval(countdownRef.current);
};
}, [autoRefresh, refreshAll]);

useEffect(() => {
const handler = () => {
if (document.hidden) {
if (intervalRef.current) clearInterval(intervalRef.current);
if (countdownRef.current) clearInterval(countdownRef.current);
} else if (autoRefresh) {
intervalRef.current = setInterval(refreshAll, REFRESH_INTERVAL_MS);
countdownRef.current = setInterval(() => {
setCountdown((prev) => (prev <= 1 ? 120 : prev - 1));
}, 1000);
}
const [connectionList, caches] = await Promise.all([
fetchConnections(),
fetchCachedProviderLimits(),
]);
applyCachedQuotaState(connectionList, caches);
setInitialLoading(false);
};
document.addEventListener("visibilitychange", handler);
return () => document.removeEventListener("visibilitychange", handler);
}, [autoRefresh, refreshAll]);
init().catch(() => {
setInitialLoading(false);
});
}, [applyCachedQuotaState, fetchCachedProviderLimits, fetchConnections]);

const filteredConnections = useMemo(
() =>
Expand Down Expand Up @@ -462,26 +455,6 @@ export default function ProviderLimits() {
</button>
</div>

<button
onClick={() => {
const next = !autoRefresh;
setAutoRefresh(next);
localStorage.setItem(LS_AUTO_REFRESH, String(next));
}}
className="flex items-center gap-1.5 px-3 py-1.5 rounded-lg border border-border bg-transparent cursor-pointer text-text-main text-[13px]"
>
<span
className="material-symbols-outlined text-[18px]"
style={{
color: autoRefresh ? "#22c55e" : "var(--text-muted)",
}}
>
{autoRefresh ? "toggle_on" : "toggle_off"}
</span>
{t("autoRefresh")}
{autoRefresh && <span className="text-xs text-text-muted">({countdown}s)</span>}
</button>

<button
onClick={refreshAll}
disabled={refreshingAll}
Expand Down
Loading