-
Notifications
You must be signed in to change notification settings - Fork 611
feat: ratelimit logs #2866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
feat: ratelimit logs #2866
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
5ad0138
feat: init
ogzhanolguncu 1140dc3
chore: Add missing definitions
ogzhanolguncu 061cda8
fix: table border color
ogzhanolguncu d5a396f
fix: add missing items
ogzhanolguncu 61b75fe
fix: add missing elements
ogzhanolguncu cf75f7d
chore: format
ogzhanolguncu 6fb4c1d
fix: add placeholder for empty namespaces
ogzhanolguncu 1ee9bfe
fix: rename rejected to blocked
ogzhanolguncu 0ba37f0
fix: condition for since
ogzhanolguncu 813f6ab
fix: filters and add datetime to ratelimit logs
ogzhanolguncu edb9254
Merge branch 'main' into feat/ratelimit-logs
ogzhanolguncu 9a96d6a
fix: sql security issue
ogzhanolguncu ab0abf6
fix: coderabbit comments
ogzhanolguncu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
23 changes: 23 additions & 0 deletions
23
...app/(app)/ratelimits/[namespaceId]/logs/components/charts/components/logs-chart-error.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| import { ResponsiveContainer } from "recharts"; | ||
|
|
||
| export const LogsChartError = () => { | ||
| return ( | ||
| <div className="w-full relative"> | ||
| <div className="px-2 text-accent-11 font-mono absolute top-0 text-xxs w-full flex justify-between opacity-50"> | ||
| {Array(5) | ||
| .fill(0) | ||
| .map((_, i) => ( | ||
| // biome-ignore lint/suspicious/noArrayIndexKey: it's okay | ||
| <div key={i} className="z-10"> | ||
| --:-- | ||
| </div> | ||
| ))} | ||
| </div> | ||
| <ResponsiveContainer height={50} className="border-b border-gray-4" width="100%"> | ||
| <div className="h-full w-full flex items-center justify-center"> | ||
| <span className="text-xs text-error-11 font-mono">Could not retrieve logs</span> | ||
| </div> | ||
| </ResponsiveContainer> | ||
| </div> | ||
| ); | ||
| }; |
28 changes: 28 additions & 0 deletions
28
...p/(app)/ratelimits/[namespaceId]/logs/components/charts/components/logs-chart-loading.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| import { Bar, BarChart, ResponsiveContainer, YAxis } from "recharts"; | ||
| import { calculateTimePoints } from "../utils/calculate-timepoints"; | ||
| import { formatTimestampLabel } from "../utils/format-timestamp"; | ||
|
|
||
| export const LogsChartLoading = () => { | ||
| const mockData = Array.from({ length: 100 }).map(() => ({ | ||
| success: Math.random() * 0.5 + 0.5, // Random values between 0.5 and 1 | ||
| })); | ||
|
|
||
| return ( | ||
| <div className="w-full relative animate-pulse"> | ||
| <div className="px-2 text-accent-11 font-mono absolute top-0 text-xxs w-full flex justify-between"> | ||
| {calculateTimePoints(Date.now(), Date.now()).map((time, i) => ( | ||
| // biome-ignore lint/suspicious/noArrayIndexKey: it's safe to use index here | ||
| <div key={i} className="z-10"> | ||
| {formatTimestampLabel(time)} | ||
| </div> | ||
| ))} | ||
| </div> | ||
| <ResponsiveContainer height={50} className="border-b border-gray-4" width="100%"> | ||
| <BarChart margin={{ top: 0, right: -20, bottom: 0, left: -20 }} barGap={0} data={mockData}> | ||
| <YAxis domain={[0, (dataMax: number) => dataMax * 2]} hide /> | ||
| <Bar dataKey="success" fill="hsl(var(--accent-3))" /> | ||
| </BarChart> | ||
| </ResponsiveContainer> | ||
| </div> | ||
| ); | ||
| }; | ||
95 changes: 95 additions & 0 deletions
95
...d/app/(app)/ratelimits/[namespaceId]/logs/components/charts/hooks/use-fetch-timeseries.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| import { trpc } from "@/lib/trpc/client"; | ||
| import type { TimeseriesGranularity } from "@/lib/trpc/routers/ratelimit/query-timeseries/utils"; | ||
| import { addMinutes, format } from "date-fns"; | ||
| import { useMemo } from "react"; | ||
| import { useFilters } from "../../../hooks/use-filters"; | ||
| import type { RatelimitQueryTimeseriesPayload } from "../query-timeseries.schema"; | ||
|
|
||
| // Duration in milliseconds for historical data fetch window (1 hours) | ||
| const TIMESERIES_DATA_WINDOW = 60 * 60 * 1000; | ||
|
|
||
| const formatTimestamp = (value: string | number, granularity: TimeseriesGranularity) => { | ||
| const date = new Date(value); | ||
| const offset = new Date().getTimezoneOffset() * -1; | ||
| const localDate = addMinutes(date, offset); | ||
|
|
||
| switch (granularity) { | ||
| case "perMinute": | ||
| return format(localDate, "HH:mm:ss"); | ||
| case "perHour": | ||
| return format(localDate, "MMM d, HH:mm"); | ||
| case "perDay": | ||
| return format(localDate, "MMM d"); | ||
| case "perMonth": | ||
| return format(localDate, "MMM yyyy"); | ||
| default: | ||
| return format(localDate, "Pp"); | ||
| } | ||
| }; | ||
|
|
||
| export const useFetchRatelimitTimeseries = (namespaceId: string) => { | ||
| const { filters } = useFilters(); | ||
| const dateNow = useMemo(() => Date.now(), []); | ||
|
|
||
| const queryParams = useMemo(() => { | ||
| const params: RatelimitQueryTimeseriesPayload = { | ||
| namespaceId, | ||
| startTime: dateNow - TIMESERIES_DATA_WINDOW, | ||
| endTime: dateNow, | ||
| identifiers: { filters: [] }, | ||
| since: "", | ||
| }; | ||
|
|
||
| filters.forEach((filter) => { | ||
| switch (filter.field) { | ||
| case "identifiers": { | ||
| if (typeof filter.value !== "string") { | ||
| console.error("Identifier filter value type has to be 'string'"); | ||
| return; | ||
| } | ||
| params.identifiers?.filters.push({ | ||
| operator: filter.operator as "is" | "contains", | ||
| value: filter.value, | ||
| }); | ||
| break; | ||
| } | ||
| case "startTime": | ||
| case "endTime": { | ||
| if (typeof filter.value !== "number") { | ||
| console.error(`${filter.field} filter value type has to be 'number'`); | ||
| return; | ||
| } | ||
| params[filter.field] = filter.value; | ||
| break; | ||
| } | ||
| case "since": { | ||
| if (typeof filter.value !== "string") { | ||
| console.error("Since filter value type has to be 'string'"); | ||
| return; | ||
| } | ||
| params.since = filter.value; | ||
| break; | ||
| } | ||
| } | ||
| }); | ||
ogzhanolguncu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| return params; | ||
| }, [filters, dateNow, namespaceId]); | ||
|
|
||
| const { data, isLoading, isError } = trpc.ratelimit.logs.queryRatelimitTimeseries.useQuery( | ||
| queryParams, | ||
| { | ||
| refetchInterval: queryParams.endTime ? false : 10_000, | ||
| }, | ||
| ); | ||
|
|
||
| const timeseries = data?.timeseries.map((ts) => ({ | ||
| displayX: formatTimestamp(ts.x, data.granularity), | ||
| originalTimestamp: ts.x, | ||
| success: ts.y.passed, | ||
| error: ts.y.total - ts.y.passed, | ||
| total: ts.y.total, | ||
| })); | ||
|
|
||
| return { timeseries, isLoading, isError }; | ||
| }; | ||
142 changes: 142 additions & 0 deletions
142
apps/dashboard/app/(app)/ratelimits/[namespaceId]/logs/components/charts/index.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,142 @@ | ||
| "use client"; | ||
| import { | ||
| type ChartConfig, | ||
| ChartContainer, | ||
| ChartTooltip, | ||
| ChartTooltipContent, | ||
| } from "@/components/ui/chart"; | ||
| import { Grid } from "@unkey/icons"; | ||
| import { useEffect, useRef } from "react"; | ||
| import { Bar, BarChart, ResponsiveContainer, YAxis } from "recharts"; | ||
| import { useRatelimitLogsContext } from "../../context/logs"; | ||
| import { LogsChartError } from "./components/logs-chart-error"; | ||
| import { LogsChartLoading } from "./components/logs-chart-loading"; | ||
| import { useFetchRatelimitTimeseries } from "./hooks/use-fetch-timeseries"; | ||
| import { calculateTimePoints } from "./utils/calculate-timepoints"; | ||
| import { formatTimestampLabel, formatTimestampTooltip } from "./utils/format-timestamp"; | ||
|
|
||
| const chartConfig = { | ||
| success: { | ||
| label: "Passed", | ||
| subLabel: "Passed", | ||
| color: "hsl(var(--accent-4))", | ||
| }, | ||
| error: { | ||
| label: "Blocked", | ||
| subLabel: "Blocked", | ||
| color: "hsl(var(--warning-9))", | ||
| }, | ||
| } satisfies ChartConfig; | ||
|
|
||
| export function RatelimitLogsChart({ | ||
| onMount, | ||
| }: { | ||
| onMount: (distanceToTop: number) => void; | ||
| }) { | ||
| const chartRef = useRef<HTMLDivElement>(null); | ||
|
|
||
| const { namespaceId } = useRatelimitLogsContext(); | ||
| const { timeseries, isLoading, isError } = useFetchRatelimitTimeseries(namespaceId); | ||
| // biome-ignore lint/correctness/useExhaustiveDependencies: We need this to re-trigger distanceToTop calculation | ||
| useEffect(() => { | ||
| const distanceToTop = chartRef.current?.getBoundingClientRect().top ?? 0; | ||
| onMount(distanceToTop); | ||
| }, [onMount, isLoading, isError]); | ||
|
|
||
| if (isError) { | ||
| return <LogsChartError />; | ||
| } | ||
|
|
||
| if (isLoading) { | ||
| return <LogsChartLoading />; | ||
| } | ||
|
|
||
| return ( | ||
| <div className="w-full relative" ref={chartRef}> | ||
| <div className="px-2 text-accent-11 font-mono absolute top-0 text-xxs w-full flex justify-between"> | ||
| {timeseries | ||
| ? calculateTimePoints( | ||
| timeseries[0].originalTimestamp ?? Date.now(), | ||
| timeseries.at(-1)?.originalTimestamp ?? Date.now(), | ||
| ).map((time, i) => ( | ||
| // biome-ignore lint/suspicious/noArrayIndexKey: use of index is acceptable here. | ||
| <div key={i} className="z-10"> | ||
| {formatTimestampLabel(time)} | ||
| </div> | ||
| )) | ||
| : null} | ||
| </div> | ||
| <ResponsiveContainer width="100%" height={50} className="border-b border-gray-4"> | ||
| <ChartContainer config={chartConfig}> | ||
| <BarChart | ||
| data={timeseries} | ||
| barGap={2} | ||
| barSize={8} | ||
| margin={{ top: 0, right: 0, bottom: 0, left: 0 }} | ||
| > | ||
| <YAxis domain={[0, (dataMax: number) => dataMax * 1.5]} hide /> | ||
| <ChartTooltip | ||
| position={{ y: 50 }} | ||
| isAnimationActive | ||
| wrapperStyle={{ zIndex: 1000 }} | ||
| cursor={{ | ||
| fill: "hsl(var(--accent-3))", | ||
| strokeWidth: 1, | ||
| strokeDasharray: "5 5", | ||
| strokeOpacity: 0.7, | ||
| }} | ||
| content={({ active, payload, label }) => { | ||
| if (!active || !payload?.length || payload?.[0]?.payload.total === 0) { | ||
| return null; | ||
| } | ||
|
|
||
| return ( | ||
| <ChartTooltipContent | ||
| payload={payload} | ||
| label={label} | ||
| active={active} | ||
| bottomExplainer={ | ||
| <div className="grid gap-1.5 pt-2 border-t border-gray-4"> | ||
| <div className="flex w-full [&>svg]:size-4 gap-4 px-4 items-center"> | ||
| <Grid className="text-gray-6" /> | ||
| <div className="flex gap-4 leading-none justify-between w-full py-1 items-center"> | ||
| <div className="flex gap-4 items-center min-w-[80px]"> | ||
| <span className="capitalize text-accent-9 text-xs w-[2ch] inline-block"> | ||
| All | ||
| </span> | ||
| <span className="capitalize text-accent-12 text-xs">Total</span> | ||
| </div> | ||
| <div className="ml-auto"> | ||
| <span className="font-mono tabular-nums text-accent-12"> | ||
| {payload[0]?.payload?.total} | ||
| </span> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| } | ||
| className="rounded-lg shadow-lg border border-gray-4" | ||
| labelFormatter={(_, tooltipPayload) => { | ||
| const originalTimestamp = tooltipPayload[0]?.payload?.originalTimestamp; | ||
| return originalTimestamp ? ( | ||
| <div> | ||
| <span className="font-mono text-accent-9 text-xs px-4"> | ||
| {formatTimestampTooltip(originalTimestamp)} | ||
| </span> | ||
| </div> | ||
| ) : ( | ||
| "" | ||
| ); | ||
| }} | ||
| /> | ||
| ); | ||
| }} | ||
| /> | ||
| <Bar dataKey="success" stackId="a" fill={chartConfig.success.color} /> | ||
| <Bar dataKey="error" stackId="a" fill={chartConfig.error.color} /> | ||
| </BarChart> | ||
| </ChartContainer> | ||
| </ResponsiveContainer> | ||
| </div> | ||
| ); | ||
| } |
21 changes: 21 additions & 0 deletions
21
...oard/app/(app)/ratelimits/[namespaceId]/logs/components/charts/query-timeseries.schema.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| import { z } from "zod"; | ||
| import { filterOperatorEnum } from "../../filters.schema"; | ||
|
|
||
| export const ratelimitQueryTimeseriesPayload = z.object({ | ||
| startTime: z.number().int(), | ||
| endTime: z.number().int(), | ||
| since: z.string(), | ||
| namespaceId: z.string(), | ||
| identifiers: z | ||
| .object({ | ||
| filters: z.array( | ||
| z.object({ | ||
| operator: filterOperatorEnum, | ||
| value: z.string(), | ||
| }), | ||
| ), | ||
| }) | ||
| .nullable(), | ||
| }); | ||
ogzhanolguncu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| export type RatelimitQueryTimeseriesPayload = z.infer<typeof ratelimitQueryTimeseriesPayload>; | ||
9 changes: 9 additions & 0 deletions
9
...d/app/(app)/ratelimits/[namespaceId]/logs/components/charts/utils/calculate-timepoints.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| export const calculateTimePoints = (startTime: number, endTime: number) => { | ||
| const points = 6; | ||
| const timeRange = endTime - startTime; | ||
| const step = Math.floor(timeRange / (points - 1)); | ||
|
|
||
| return Array.from({ length: points }, (_, i) => new Date(startTime + step * i)).filter( | ||
| (date) => date.getTime() <= endTime, | ||
| ); | ||
| }; |
13 changes: 13 additions & 0 deletions
13
...board/app/(app)/ratelimits/[namespaceId]/logs/components/charts/utils/format-timestamp.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| import { addMinutes, format } from "date-fns"; | ||
|
|
||
| export const formatTimestampTooltip = (value: string | number) => { | ||
| const date = new Date(value); | ||
| const offset = new Date().getTimezoneOffset() * -1; | ||
| const localDate = addMinutes(date, offset); | ||
| return format(localDate, "MMM dd HH:mm aa"); | ||
| }; | ||
|
|
||
| export const formatTimestampLabel = (timestamp: string | number | Date) => { | ||
| const date = new Date(timestamp); | ||
| return format(date, "MMM dd, h:mma").toUpperCase(); | ||
| }; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.