diff --git a/apps/docs/analytics/getting-started.mdx b/apps/docs/analytics/getting-started.mdx index 0dd6d95baf..1ea887bceb 100644 --- a/apps/docs/analytics/getting-started.mdx +++ b/apps/docs/analytics/getting-started.mdx @@ -39,49 +39,58 @@ Once you have access, execute your first analytics query using the `/v2/analytic ### Count Total Verifications -```sql -SELECT COUNT(*) as total -FROM key_verifications_v1 +Count the total number of key verifications in the last 7 days across all your APIs to get a high-level view of your overall usage volume. + + +```sql SQL +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY ``` -Execute this query with curl: - -```bash +```bash cURL curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ - "query": "SELECT COUNT(*) as total FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY" + "query": "SELECT SUM(count) as total FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY" }' ``` + + ### Break Down by Outcome -```sql +Group verifications by their outcome (`VALID`, `RATE_LIMITED`, `USAGE_EXCEEDED`, etc.) over the last 24 hours to understand the distribution of successful vs. failed requests. + + +```sql SQL SELECT outcome, - COUNT(*) as count -FROM key_verifications_v1 + SUM(count) as count +FROM key_verifications_per_hour_v1 WHERE time >= now() - INTERVAL 24 HOUR GROUP BY outcome ORDER BY count DESC ``` -Execute this query with curl: - -```bash +```bash cURL curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d '{ - "query": "SELECT outcome, COUNT(*) as count FROM key_verifications_v1 WHERE time >= now() - INTERVAL 24 HOUR GROUP BY outcome ORDER BY count DESC" + "query": "SELECT outcome, SUM(count) as count FROM key_verifications_per_hour_v1 WHERE time >= now() - INTERVAL 24 HOUR GROUP BY outcome ORDER BY count DESC" }' ``` + + ### Top Users by Usage -```sql +Identify your most active users by counting their total verifications over the last 30 days to spot power users or potential abuse patterns. + + +```sql SQL SELECT external_id, SUM(count) as verifications @@ -92,9 +101,7 @@ ORDER BY verifications DESC LIMIT 10 ``` -Execute this query with curl: - -```bash +```bash cURL curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ @@ -103,6 +110,8 @@ curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ }' ``` + + **Performance tip:** For longer time ranges, use pre-aggregated tables instead of the raw table: - `key_verifications_per_minute_v1` - For queries spanning hours @@ -147,25 +156,3 @@ You can filter queries to specific APIs or users. Use `key_space_id` to filter b and quota). See [Query Restrictions](/analytics/query-restrictions) for complete details on limits and error codes. - -## Next Steps - - - - Explore common SQL patterns for analytics and billing - - - Browse available tables, columns, and data types - - - View limits, quotas, and permissions - - diff --git a/apps/docs/analytics/overview.mdx b/apps/docs/analytics/overview.mdx index 2d5ba1ed10..a7653f80fa 100644 --- a/apps/docs/analytics/overview.mdx +++ b/apps/docs/analytics/overview.mdx @@ -6,7 +6,8 @@ description: "Query your verification data with SQL" **Analytics is currently in private beta and available by request only.** - See [Getting Started](/analytics/getting-started) for access instructions. +See [Getting Started](/analytics/getting-started) for access instructions. + ## What is Unkey Analytics? @@ -32,6 +33,7 @@ graph LR ``` You can query these tables using standard SQL to: + - Aggregate verification counts by time period - Group by API, user, or outcome - Filter by region, tags, or custom criteria @@ -54,23 +56,28 @@ Every verification event contains: | `tags` | Array(String) | Custom tags added during verification | | `spent_credits` | Int64 | Number of credits spent on this verification (0 if no credits were spent) | -## Next Steps +## Use Cases - - - Learn how to request access and execute your first query - - - Explore common SQL patterns for analytics and billing + + + Usage-based billing and credit tracking - Browse available tables, columns, and data types + API health and performance monitoring - - View limits, quotas, and permissions + + User behavior and engagement insights diff --git a/apps/docs/analytics/query-examples.mdx b/apps/docs/analytics/query-examples.mdx index 934ab438d4..51811d5935 100644 --- a/apps/docs/analytics/query-examples.mdx +++ b/apps/docs/analytics/query-examples.mdx @@ -9,22 +9,24 @@ This guide provides SQL query examples for common analytics scenarios covering a When making API requests, you need to format the SQL query as a JSON string on a single line. Here's how: -**Readable format (for documentation):** - -```sql + +```sql SQL SELECT COUNT(*) as total FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY ``` -**JSON format (for API requests):** - -```json -{ - "query": "SELECT COUNT(*) as total FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT COUNT(*) as total FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY" + }' ``` + + Each example below shows both the readable multi-line SQL and the single-line JSON format you can copy directly into your API requests. @@ -32,51 +34,64 @@ WHERE time >= now() - INTERVAL 7 DAY ## Usage Analytics - +**Use this for:** High-level usage metrics, health monitoring, and trend analysis. - +**Key patterns:** Total counts, outcome breakdowns, time series analysis. -```sql -SELECT COUNT(*) as total_verifications -FROM key_verifications_v1 +### Total verifications in the last 7 days + +Count total verifications across all APIs in the last 7 days. + + +```sql SQL +SELECT SUM(count) as total_verifications +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total_verifications FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total_verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY" + }' ``` - + - +### Verifications by outcome -```sql +Break down verifications by outcome to understand success vs failure rates. + + +```sql SQL SELECT outcome, - COUNT(*) as count -FROM key_verifications_v1 + SUM(count) as count +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY outcome ORDER BY count DESC ``` -**JSON format:** - -```json -{ - "query": "SELECT outcome, COUNT(*) as count FROM key_verifications_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY outcome ORDER BY count DESC" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT outcome, SUM(count) as count FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY outcome ORDER BY count DESC" + }' ``` - + - +### Daily verification trend -```sql +Track daily verification patterns over the last 30 days. + + +```sql SQL SELECT time as date, SUM(count) as verifications @@ -86,19 +101,23 @@ GROUP BY date ORDER BY date ``` -**JSON format:** - -```json -{ - "query": "SELECT time as date, SUM(count) as verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT time as date, SUM(count) as verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date" + }' ``` - + - +### Hourly breakdown for today -```sql +Analyze hourly verification patterns for today with outcome breakdown. + + +```sql SQL SELECT time as hour, outcome, @@ -109,252 +128,292 @@ GROUP BY time, outcome ORDER BY time, outcome ``` -**JSON format:** - -```json -{ - "query": "SELECT time as hour, outcome, SUM(count) as verifications FROM key_verifications_per_hour_v1 WHERE time >= toStartOfDay(now()) GROUP BY time, outcome ORDER BY time, outcome" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT time as hour, outcome, SUM(count) as verifications FROM key_verifications_per_hour_v1 WHERE time >= toStartOfDay(now()) GROUP BY time, outcome ORDER BY time, outcome" + }' ``` - - - + ## Usage by User - +**Use this for:** Understanding user behavior, identifying power users, tracking user activity over time. + +**Key patterns:** User ranking, activity trends, specific user analysis. - +### All users ranked by usage -```sql +Rank all users by their total verification usage over the last 30 days. + + +```sql SQL SELECT external_id, - COUNT(*) as total_verifications, - countIf(outcome = 'VALID') as successful, - countIf(outcome = 'RATE_LIMITED') as rate_limited -FROM key_verifications_v1 + SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful, + SUM(CASE WHEN outcome = 'RATE_LIMITED' THEN count ELSE 0 END) as rate_limited +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY AND external_id != '' - GROUP BY external_id +GROUP BY external_id ORDER BY total_verifications DESC LIMIT 100 ``` -**JSON format:** - -```json -{ - "query": "SELECT external_id, COUNT(*) as total_verifications, countIf(outcome = 'VALID') as successful, countIf(outcome = 'RATE_LIMITED') as rate_limited FROM key_verifications_v1 WHERE time >= now() - INTERVAL 30 DAY AND external_id != '' GROUP BY external_id ORDER BY total_verifications DESC LIMIT 100" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT external_id, SUM(count) as total_verifications, SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) as successful, SUM(CASE WHEN outcome = '\''RATE_LIMITED'\'' THEN count ELSE 0 END) as rate_limited FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY AND external_id != '\'''\'' GROUP BY external_id ORDER BY total_verifications DESC LIMIT 100" + }' ``` - + - +### Usage for a specific user -```sql +Analyze usage patterns for a specific user over the last 30 days. + + +```sql SQL SELECT - COUNT(*) as total_verifications, - countIf(outcome = 'VALID') as successful, - countIf(outcome = 'RATE_LIMITED') as rate_limited -FROM key_verifications_v1 + SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful, + SUM(CASE WHEN outcome = 'RATE_LIMITED' THEN count ELSE 0 END) as rate_limited +FROM key_verifications_per_day_v1 WHERE external_id = 'user_123' AND time >= now() - INTERVAL 30 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total_verifications, countIf(outcome = 'VALID') as successful, countIf(outcome = 'RATE_LIMITED') as rate_limited FROM key_verifications_v1 WHERE external_id = 'user_123' AND time >= now() - INTERVAL 30 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total_verifications, SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) as successful, SUM(CASE WHEN outcome = '\''RATE_LIMITED'\'' THEN count ELSE 0 END) as rate_limited FROM key_verifications_per_day_v1 WHERE external_id = '\''user_123'\'' AND time >= now() - INTERVAL 30 DAY" + }' ``` - + - +### Top 10 users by API usage -```sql +Identify your most active users by verification count. + + +```sql SQL SELECT external_id, - COUNT(*) as total_verifications -FROM key_verifications_v1 + SUM(count) as total_verifications +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY AND external_id != '' - GROUP BY external_id +GROUP BY external_id ORDER BY total_verifications DESC LIMIT 10 ``` -**JSON format:** - -```json -{ - "query": "SELECT external_id, COUNT(*) as total_verifications FROM key_verifications_v1 WHERE time >= now() - INTERVAL 30 DAY AND external_id != '' GROUP BY external_id ORDER BY total_verifications DESC LIMIT 10" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT external_id, SUM(count) as total_verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY AND external_id != '\'''\'' GROUP BY external_id ORDER BY total_verifications DESC LIMIT 10" + }' ``` - + - +### Daily usage per user -```sql +Track daily verification patterns for each user over 30 days. + + +```sql SQL SELECT external_id, - toDate(time) as date, - COUNT(*) as verifications -FROM key_verifications_v1 + time as date, + SUM(count) as verifications +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY external_id, date ORDER BY external_id, date ``` -**JSON format:** - -```json -{ - "query": "SELECT external_id, toDate(time) as date, COUNT(*) as verifications FROM key_verifications_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY external_id, date ORDER BY external_id, date" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT external_id, time as date, SUM(count) as verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY external_id, date ORDER BY external_id, date" + }' ``` - - - + ## API Analytics - +**Use this for:** Comparing API performance, usage across different APIs, API-specific analysis. + +**Key patterns:** API comparison, success rates, per-API breakdowns. - +### Usage per API -```sql +Compare usage across all APIs to identify most active endpoints. + + +```sql SQL SELECT key_space_id, - COUNT(*) as total_verifications, - countIf(outcome = 'VALID') as successful -FROM key_verifications_v1 + SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY key_space_id ORDER BY total_verifications DESC ``` -**JSON format:** - -```json -{ - "query": "SELECT key_space_id, COUNT(*) as total_verifications, countIf(outcome = 'VALID') as successful FROM key_verifications_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY key_space_id ORDER BY total_verifications DESC" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT key_space_id, SUM(count) as total_verifications, SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) as successful FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY key_space_id ORDER BY total_verifications DESC" + }' ``` - + - +### Usage for a specific API -```sql +Analyze detailed usage patterns for a specific API over 30 days. + + +```sql SQL SELECT - COUNT(*) as total_verifications, - countIf(outcome = 'VALID') as successful, - countIf(outcome = 'RATE_LIMITED') as rate_limited, - countIf(outcome = 'INVALID') as invalid -FROM key_verifications_v1 + SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful, + SUM(CASE WHEN outcome = 'RATE_LIMITED' THEN count ELSE 0 END) as rate_limited, + SUM(CASE WHEN outcome = 'INVALID' THEN count ELSE 0 END) as invalid +FROM key_verifications_per_day_v1 WHERE key_space_id = 'ks_1234' AND time >= now() - INTERVAL 30 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total_verifications, countIf(outcome = 'VALID') as successful, countIf(outcome = 'RATE_LIMITED') as rate_limited, countIf(outcome = 'INVALID') as invalid FROM key_verifications_v1 WHERE key_space_id = 'ks_1234' AND time >= now() - INTERVAL 30 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total_verifications, SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) as successful, SUM(CASE WHEN outcome = '\''RATE_LIMITED'\'' THEN count ELSE 0 END) as rate_limited, SUM(CASE WHEN outcome = '\''INVALID'\'' THEN count ELSE 0 END) as invalid FROM key_verifications_per_day_v1 WHERE key_space_id = '\''ks_1234'\'' AND time >= now() - INTERVAL 30 DAY" + }' ``` - + - +### Compare multiple APIs -```sql +Calculate success rates for multiple APIs to compare performance. + + +```sql SQL SELECT key_space_id, - COUNT(*) as verifications, - round(countIf(outcome = 'VALID') / COUNT(*) * 100, 2) as success_rate -FROM key_verifications_v1 + SUM(count) as verifications, + round(SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) / SUM(count) * 100, 2) as success_rate +FROM key_verifications_per_day_v1 WHERE key_space_id IN ('ks_1234', 'ks_5678') AND time >= now() - INTERVAL 7 DAY GROUP BY key_space_id ``` -**JSON format:** - -```json -{ - "query": "SELECT key_space_id, COUNT(*) as verifications, round(countIf(outcome = 'VALID') / COUNT(*) * 100, 2) as success_rate FROM key_verifications_v1 WHERE key_space_id IN ('ks_1234', 'ks_5678') AND time >= now() - INTERVAL 7 DAY GROUP BY key_space_id" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT key_space_id, SUM(count) as verifications, round(SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) / SUM(count) * 100, 2) as success_rate FROM key_verifications_per_day_v1 WHERE key_space_id IN ('\''ks_1234'\'', '\''ks_5678'\'') AND time >= now() - INTERVAL 7 DAY GROUP BY key_space_id" + }' ``` - - - + ## Key Analytics - +**Use this for:** Individual API key analysis, identifying problematic keys, key-specific usage patterns. + +**Key patterns:** Key ranking, error analysis, specific key monitoring. - +### Usage per key -```sql +Identify your most frequently used API keys over the last 30 days. + + +```sql SQL SELECT key_id, - COUNT(*) as total_verifications, - countIf(outcome = 'VALID') as successful -FROM key_verifications_v1 + SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY key_id ORDER BY total_verifications DESC LIMIT 100 ``` -**JSON format:** - -```json -{ - "query": "SELECT key_id, COUNT(*) as total_verifications, countIf(outcome = 'VALID') as successful FROM key_verifications_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY key_id ORDER BY total_verifications DESC LIMIT 100" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT key_id, SUM(count) as total_verifications, SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) as successful FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 30 DAY GROUP BY key_id ORDER BY total_verifications DESC LIMIT 100" + }' ``` - + - +### Usage for a specific key -```sql +Analyze detailed usage patterns for a specific API key. + + +```sql SQL SELECT - COUNT(*) as total_verifications, - countIf(outcome = 'VALID') as successful, - countIf(outcome = 'RATE_LIMITED') as rate_limited -FROM key_verifications_v1 + SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful, + SUM(CASE WHEN outcome = 'RATE_LIMITED' THEN count ELSE 0 END) as rate_limited +FROM key_verifications_per_day_v1 WHERE key_id = 'key_1234' AND time >= now() - INTERVAL 30 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total_verifications, countIf(outcome = 'VALID') as successful, countIf(outcome = 'RATE_LIMITED') as rate_limited FROM key_verifications_v1 WHERE key_id = 'key_1234' AND time >= now() - INTERVAL 30 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total_verifications, SUM(CASE WHEN outcome = '\''VALID'\'' THEN count ELSE 0 END) as successful, SUM(CASE WHEN outcome = '\''RATE_LIMITED'\'' THEN count ELSE 0 END) as rate_limited FROM key_verifications_per_day_v1 WHERE key_id = '\''key_1234'\'' AND time >= now() - INTERVAL 30 DAY" + }' ``` - + - +### Keys with most errors -```sql +Find API keys that are generating the most errors. + + +```sql SQL SELECT key_id, - COUNT(*) as total_errors, + SUM(count) as total_errors, groupArray(DISTINCT outcome) as error_types -FROM key_verifications_v1 +FROM key_verifications_per_day_v1 WHERE outcome != 'VALID' AND time >= now() - INTERVAL 7 DAY GROUP BY key_id @@ -362,107 +421,132 @@ ORDER BY total_errors DESC LIMIT 20 ``` -**JSON format:** - -```json -{ - "query": "SELECT key_id, COUNT(*) as total_errors, groupArray(DISTINCT outcome) as error_types FROM key_verifications_v1 WHERE outcome != 'VALID' AND time >= now() - INTERVAL 7 DAY GROUP BY key_id ORDER BY total_errors DESC LIMIT 20" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT key_id, SUM(count) as total_errors, groupArray(DISTINCT outcome) as error_types FROM key_verifications_per_day_v1 WHERE outcome != '\''VALID'\'' AND time >= now() - INTERVAL 7 DAY GROUP BY key_id ORDER BY total_errors DESC LIMIT 20" + }' ``` - - - + ## Tag-Based Analytics +**Use this for:** Custom metadata filtering, endpoint analysis, user segmentation using tags. + +**Key patterns:** Tag filtering, endpoint breakdowns, custom attribute analysis. + Tags allow you to add custom metadata to verification requests for filtering and aggregation. - +### Filter by single tag - +Count verifications for requests with a specific tag. -```sql -SELECT COUNT(*) as total -FROM key_verifications_v1 + +```sql SQL +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 WHERE has(tags, 'path=/api/v1/users') AND time >= now() - INTERVAL 7 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total FROM key_verifications_v1 WHERE has(tags, 'path=/api/v1/users') AND time >= now() - INTERVAL 7 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total FROM key_verifications_per_day_v1 WHERE has(tags, '\''path=/api/v1/users'\'') AND time >= now() - INTERVAL 7 DAY" + }' ``` - + - +### Filter by multiple tags (OR) -```sql -SELECT COUNT(*) as total -FROM key_verifications_v1 +Count verifications matching any of multiple tags. + + +```sql SQL +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 WHERE hasAny(tags, ['path=/api/v1/users', 'path=/api/v1/posts']) AND time >= now() - INTERVAL 7 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total FROM key_verifications_v1 WHERE hasAny(tags, ['path=/api/v1/users', 'path=/api/v1/posts']) AND time >= now() - INTERVAL 7 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total FROM key_verifications_per_day_v1 WHERE hasAny(tags, ['\''path=/api/v1/users'\'', '\''path=/api/v1/posts'\'']) AND time >= now() - INTERVAL 7 DAY" + }' ``` - + - +### Filter by multiple tags (AND) -```sql -SELECT COUNT(*) as total -FROM key_verifications_v1 +Count verifications matching all specified tags. + + +```sql SQL +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 WHERE hasAll(tags, ['environment=production', 'team=backend']) AND time >= now() - INTERVAL 7 DAY ``` -**JSON format:** - -```json -{ - "query": "SELECT COUNT(*) as total FROM key_verifications_v1 WHERE hasAll(tags, ['environment=production', 'team=backend']) AND time >= now() - INTERVAL 7 DAY" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT SUM(count) as total FROM key_verifications_per_day_v1 WHERE hasAll(tags, ['\''environment=production'\'', '\''team=backend'\'']) AND time >= now() - INTERVAL 7 DAY" + }' ``` - + + +### Group by tag - +Aggregate verifications by individual tags to see usage patterns. -```sql + +```sql SQL SELECT arrayJoin(tags) as tag, - COUNT(*) as verifications -FROM key_verifications_v1 + SUM(count) as verifications +FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY GROUP BY tag ORDER BY verifications DESC LIMIT 20 ``` -**JSON format:** - -```json -{ - "query": "SELECT arrayJoin(tags) as tag, COUNT(*) as verifications FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY GROUP BY tag ORDER BY verifications DESC LIMIT 20" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT arrayJoin(tags) as tag, SUM(count) as verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY GROUP BY tag ORDER BY verifications DESC LIMIT 20" + }' ``` - + + +### Breakdown by endpoint (using path tag) + +Analyze request volume by API endpoint over the last 24 hours. - + + This query uses the raw table for detailed tag analysis. For longer time + ranges, consider using aggregated tables and pre-filtered tags. + -```sql + +```sql SQL SELECT arrayJoin(arrayFilter(x -> startsWith(x, 'path='), tags)) as endpoint, COUNT(*) as requests @@ -472,72 +556,84 @@ GROUP BY endpoint ORDER BY requests DESC ``` -**JSON format:** - -```json -{ - "query": "SELECT arrayJoin(arrayFilter(x -> startsWith(x, 'path='), tags)) as endpoint, COUNT(*) as requests FROM key_verifications_v1 WHERE time >= now() - INTERVAL 24 HOUR GROUP BY endpoint ORDER BY requests DESC" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT arrayJoin(arrayFilter(x -> startsWith(x, '\''path='\''), tags)) as endpoint, COUNT(*) as requests FROM key_verifications_v1 WHERE time >= now() - INTERVAL 24 HOUR GROUP BY endpoint ORDER BY requests DESC" + }' ``` - - - + ## Billing & Usage-Based Pricing - +**Use this for:** Usage-based billing implementation, credit tracking, user tier calculation. + +**Key patterns:** Credit aggregation, billing cycles, tier determination, cost analysis. + +### Monthly credits per user - +Calculate monthly credit consumption per user for billing. -```sql + +```sql SQL SELECT external_id, toStartOfMonth(time) as month, SUM(spent_credits) as total_credits -FROM key_verifications_v1 +FROM key_verifications_per_day_v1 WHERE external_id != '' AND time >= toStartOfMonth(now()) GROUP BY external_id, month ORDER BY total_credits DESC ``` -**JSON format:** - -```json -{ - "query": "SELECT external_id, toStartOfMonth(time) as month, SUM(spent_credits) as total_credits FROM key_verifications_v1 WHERE external_id != '' AND time >= toStartOfMonth(now()) GROUP BY external_id, month ORDER BY total_credits DESC" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT external_id, toStartOfMonth(time) as month, SUM(spent_credits) as total_credits FROM key_verifications_per_day_v1 WHERE external_id != '\'''\'' AND time >= toStartOfMonth(now()) GROUP BY external_id, month ORDER BY total_credits DESC" + }' ``` - + + +### Current billing period credits - +Calculate credit usage for a specific billing period. -```sql + +```sql SQL SELECT external_id, SUM(spent_credits) as credits_this_period -FROM key_verifications_v1 +FROM key_verifications_per_day_v1 WHERE external_id = 'user_123' AND time >= 1704067200000 -- Start of billing period (Unix millis) AND time < 1706745600000 -- End of billing period (Unix millis) GROUP BY external_id ``` -**JSON format:** - -```json -{ - "query": "SELECT external_id, SUM(spent_credits) as credits_this_period FROM key_verifications_v1 WHERE external_id = 'user_123' AND time >= 1704067200000 AND time < 1706745600000 GROUP BY external_id" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT external_id, SUM(spent_credits) as credits_this_period FROM key_verifications_per_day_v1 WHERE external_id = '\''user_123'\'' AND time >= 1704067200000 AND time < 1706745600000 GROUP BY external_id" + }' ``` - + + +### Credit-based tier calculation - +Determine user tiers based on monthly credit consumption. -```sql + +```sql SQL SELECT external_id, SUM(spent_credits) as total_credits, @@ -547,87 +643,99 @@ SELECT WHEN total_credits <= 100000 THEN 'pro' ELSE 'enterprise' END as tier -FROM key_verifications_v1 +FROM key_verifications_per_day_v1 WHERE time >= toStartOfMonth(now()) AND external_id = 'user_123' GROUP BY external_id ``` -**JSON format:** - -```json -{ - "query": "SELECT external_id, SUM(spent_credits) as total_credits, CASE WHEN total_credits <= 1000 THEN 'free' WHEN total_credits <= 10000 THEN 'starter' WHEN total_credits <= 100000 THEN 'pro' ELSE 'enterprise' END as tier FROM key_verifications_v1 WHERE time >= toStartOfMonth(now()) AND external_id = 'user_123' GROUP BY external_id" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT external_id, SUM(spent_credits) as total_credits, CASE WHEN total_credits <= 1000 THEN '\''free'\'' WHEN total_credits <= 10000 THEN '\''starter'\'' WHEN total_credits <= 100000 THEN '\''pro'\'' ELSE '\''enterprise'\'' END as tier FROM key_verifications_per_day_v1 WHERE time >= toStartOfMonth(now()) AND external_id = '\''user_123'\'' GROUP BY external_id" + }' ``` - + + +### Daily credit usage and cost - +Track daily credit consumption and calculate estimated costs. -```sql + +```sql SQL SELECT - toDate(time) as date, + time as date, SUM(spent_credits) as credits_used, credits_used * 0.001 as estimated_cost -- $0.001 per credit -FROM key_verifications_v1 +FROM key_verifications_per_day_v1 WHERE external_id = 'user_123' AND time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date ``` -**JSON format:** - -```json -{ - "query": "SELECT toDate(time) as date, SUM(spent_credits) as credits_used, credits_used * 0.001 as estimated_cost FROM key_verifications_v1 WHERE external_id = 'user_123' AND time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT time as date, SUM(spent_credits) as credits_used, credits_used * 0.001 as estimated_cost FROM key_verifications_per_day_v1 WHERE external_id = '\''user_123'\'' AND time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date" + }' ``` - - - + ## Advanced Queries - +**Use this for:** Complex analytical patterns, cohort analysis, moving averages, advanced insights. + +**Key patterns:** User retention, trend smoothing, complex joins, window functions. - +### Cohort analysis: New vs returning users -```sql +Perform cohort analysis to understand user retention patterns. + + +```sql SQL WITH first_seen AS ( SELECT external_id, min(time) as first_verification - FROM key_verifications_v1 + FROM key_verifications_per_day_v1 WHERE external_id != '' GROUP BY external_id ) SELECT toDate(kv.time) as date, - countIf(kv.time = fs.first_verification) as new_users, - countIf(kv.time > fs.first_verification) as returning_users -FROM key_verifications_v1 kv + SUM(CASE WHEN kv.time = fs.first_verification THEN kv.count ELSE 0 END) as new_users, + SUM(CASE WHEN kv.time > fs.first_verification THEN kv.count ELSE 0 END) as returning_users +FROM key_verifications_per_day_v1 kv JOIN first_seen fs ON kv.external_id = fs.external_id WHERE kv.time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date ``` -**JSON format:** - -```json -{ - "query": "WITH first_seen AS ( SELECT external_id, min(time) as first_verification FROM key_verifications_v1 WHERE external_id != '' GROUP BY external_id ) SELECT toDate(kv.time) as date, countIf(kv.time = fs.first_verification) as new_users, countIf(kv.time > fs.first_verification) as returning_users FROM key_verifications_v1 kv JOIN first_seen fs ON kv.external_id = fs.external_id WHERE kv.time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "WITH first_seen AS ( SELECT external_id, min(time) as first_verification FROM key_verifications_per_day_v1 WHERE external_id != '\'''\'' GROUP BY external_id ) SELECT toDate(kv.time) as date, SUM(CASE WHEN kv.time = fs.first_verification THEN kv.count ELSE 0 END) as new_users, SUM(CASE WHEN kv.time > fs.first_verification THEN kv.count ELSE 0 END) as returning_users FROM key_verifications_per_day_v1 kv JOIN first_seen fs ON kv.external_id = fs.external_id WHERE kv.time >= now() - INTERVAL 30 DAY GROUP BY date ORDER BY date" + }' ``` - + + +### Moving average (7-day) - +Calculate 7-day moving average to smooth out daily fluctuations. -```sql + +```sql SQL SELECT date, verifications, @@ -646,27 +754,27 @@ FROM ( ORDER BY date ``` -**JSON format:** - -```json -{ - "query": "SELECT date, verifications, avg(verifications) OVER ( ORDER BY date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW ) as moving_avg_7d FROM ( SELECT time as date, SUM(count) as verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 60 DAY GROUP BY date ) ORDER BY date" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT date, verifications, avg(verifications) OVER ( ORDER BY date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW ) as moving_avg_7d FROM ( SELECT time as date, SUM(count) as verifications FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 60 DAY GROUP BY date ) ORDER BY date" + }' ``` - - - + ## Using Aggregated Tables For better performance on large time ranges, use pre-aggregated tables: - +### Hourly aggregates - +Query hourly verification counts for the last 7 days. -```sql + +```sql SQL SELECT time, SUM(count) as total @@ -676,19 +784,23 @@ GROUP BY time ORDER BY time ``` -**JSON format:** - -```json -{ - "query": "SELECT time, SUM(count) as total FROM key_verifications_per_hour_v1 WHERE time >= toStartOfHour(now() - INTERVAL 7 DAY) GROUP BY time ORDER BY time" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT time, SUM(count) as total FROM key_verifications_per_hour_v1 WHERE time >= toStartOfHour(now() - INTERVAL 7 DAY) GROUP BY time ORDER BY time" + }' ``` - + + +### Daily aggregates - +Query daily verification counts for the last 30 days. -```sql + +```sql SQL SELECT time, SUM(count) as total @@ -698,19 +810,23 @@ GROUP BY time ORDER BY time ``` -**JSON format:** - -```json -{ - "query": "SELECT time, SUM(count) as total FROM key_verifications_per_day_v1 WHERE time >= toStartOfDay(now() - INTERVAL 30 DAY) GROUP BY time ORDER BY time" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT time, SUM(count) as total FROM key_verifications_per_day_v1 WHERE time >= toStartOfDay(now() - INTERVAL 30 DAY) GROUP BY time ORDER BY time" + }' ``` - + + +### Monthly aggregates - +Query monthly verification counts for the last year. -```sql + +```sql SQL SELECT time, SUM(count) as total @@ -720,17 +836,16 @@ GROUP BY time ORDER BY time ``` -**JSON format:** - -```json -{ - "query": "SELECT time, SUM(count) as total FROM key_verifications_per_month_v1 WHERE time >= toStartOfMonth(now() - INTERVAL 12 MONTH) GROUP BY time ORDER BY time" -} +```bash cURL +curl -X POST https://api.unkey.com/v2/analytics.getVerifications \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{ + "query": "SELECT time, SUM(count) as total FROM key_verifications_per_month_v1 WHERE time >= toStartOfMonth(now() - INTERVAL 12 MONTH) GROUP BY time ORDER BY time" + }' ``` - - - + ## Tips for Efficient Queries @@ -738,25 +853,3 @@ ORDER BY time 2. **Use aggregated tables** - Hourly/daily/monthly tables for longer ranges 3. **Add LIMIT clauses** - Prevent returning too much data 4. **Filter before grouping** - Use WHERE instead of HAVING when possible - -## Next Steps - - - - Browse available tables, columns, and data types - - - View limits, quotas, and permissions - - - Explore ClickHouse SQL functions (external link) - - diff --git a/apps/docs/analytics/query-restrictions.mdx b/apps/docs/analytics/query-restrictions.mdx index 23bd43e8c8..e391df8a3e 100644 --- a/apps/docs/analytics/query-restrictions.mdx +++ b/apps/docs/analytics/query-restrictions.mdx @@ -5,12 +5,6 @@ description: "Limits, quotas, and permissions for analytics queries" This page explains the restrictions, resource limits, and permissions for analytics queries. -## Workspace Isolation - -Every query is automatically scoped to your workspace. You can only access your own verification data - it's impossible to view data from other workspaces. - -## Query Restrictions - ### Only SELECT Allowed Only `SELECT` queries are permitted. All other SQL statement types return a `query_not_supported` error. @@ -40,31 +34,35 @@ Only explicitly approved functions are allowed. Any function not on this list wi -`count`, `sum`, `avg`, `min`, `max`, `any`, `groupArray`, `groupUniqArray`, `uniq`, `uniqExact`, `quantile`, `countIf` + `count`, `sum`, `avg`, `min`, `max`, `any`, `groupArray`, `groupUniqArray`, + `uniq`, `uniqExact`, `quantile`, `countIf` -`now`, `now64`, `today`, `toDate`, `toDateTime`, `toDateTime64`, `toStartOfDay`, `toStartOfWeek`, `toStartOfMonth`, `toStartOfYear`, `toStartOfHour`, `toStartOfMinute`, `date_trunc`, `formatDateTime`, `fromUnixTimestamp64Milli`, `toUnixTimestamp64Milli`, `toIntervalDay`, `toIntervalWeek`, `toIntervalMonth`, `toIntervalYear`, `toIntervalHour`, `toIntervalMinute`, `toIntervalSecond` + `now`, `now64`, `today`, `toDate`, `toDateTime`, `toDateTime64`, + `toStartOfDay`, `toStartOfWeek`, `toStartOfMonth`, `toStartOfYear`, + `toStartOfHour`, `toStartOfMinute`, `date_trunc`, `formatDateTime`, + `fromUnixTimestamp64Milli`, `toUnixTimestamp64Milli`, `toIntervalDay`, + `toIntervalWeek`, `toIntervalMonth`, `toIntervalYear`, `toIntervalHour`, + `toIntervalMinute`, `toIntervalSecond`, `toIntervalMillisecond`, + `toIntervalMicrosecond`, `toIntervalNanosecond`, `toIntervalQuarter` -`lower`, `upper`, `substring`, `concat`, `length`, `trim`, `startsWith`, `endsWith` + `lower`, `upper`, `substring`, `concat`, `length`, `trim`, `startsWith`, + `endsWith` - -`round`, `floor`, `ceil`, `abs` - +`round`, `floor`, `ceil`, `abs` - -`if`, `case`, `coalesce` - +`if`, `case`, `coalesce` -`toString`, `toInt32`, `toInt64`, `toFloat64` + `toString`, `toInt32`, `toInt64`, `toFloat64` -`has`, `hasAny`, `hasAll`, `arrayJoin`, `arrayFilter`, `length` + `has`, `hasAny`, `hasAll`, `arrayJoin`, `arrayFilter` @@ -86,7 +84,6 @@ To ensure fair usage and prevent abuse, queries are subject to resource limits: | Max execution time | 30 seconds | Prevent long-running queries | | Max execution time (per window) | 1800 seconds (30 min) | Total execution time per hour | | Max memory usage | 1 GB | Prevent memory exhaustion | -| Max rows to read | 10 million | Limit data scanned | | Max result rows | 10 million | Limit result set size | ### Query Quotas @@ -97,7 +94,8 @@ To ensure fair usage and prevent abuse, queries are subject to resource limits: If you need higher limits for your use case, please contact us at - [support@unkey.dev](mailto:support@unkey.dev). + [support@unkey.dev](mailto:support@unkey.dev) with details about your specific + requirements and expected query volume. ### Error Codes @@ -108,60 +106,5 @@ When limits are exceeded, you'll receive specific error codes: | ----------------------------- | --------------------------------- | ---------------------------------------------------------- | | `query_execution_timeout` | Query took longer than 30 seconds | Add more filters, reduce time range, use aggregated tables | | `query_memory_limit_exceeded` | Query used more than 1GB memory | Reduce result set size, add LIMIT clause, use aggregation | -| `query_rows_limit_exceeded` | Query scanned more than 10M rows | Add time filters, use aggregated tables (hour/day/month) | +| `query_result_rows_exceeded` | Query returned more than 10M rows | Add LIMIT clause, use aggregation, reduce time range | | `query_quota_exceeded` | Exceeded 1000 queries per hour | Wait for quota to reset, optimize query frequency | - -## Authentication - -Analytics queries require a root key with specific permissions: - -### Required Permissions - -You need to grant analytics access for per API or for all APIs: - -**Workspace-level access** (all APIs): - -``` -api.*.read_analytics -``` - -**Per-API access** (specific API): - -``` -api..read_analytics -``` - -Choose workspace-level for broad access or per-API for fine-grained control. - -### Root Key Best Practices - -1. **Use environment variables** - Never hardcode root keys -2. **Rotate keys regularly** - Create new keys and revoke old ones -3. **Limit permissions** - Only grant `read_analytics` permission -4. **Use separate keys** - Different keys for different services -5. **Monitor usage** - Track which keys are making queries - - - See [Getting Started](/analytics/getting-started) for step-by-step instructions on creating an analytics root key. - - -## Next Steps - - - - Create a root key and run your first query - - - Explore common query patterns - - - Browse available tables and columns - - - View all error codes and responses - - diff --git a/apps/docs/analytics/quick-reference.mdx b/apps/docs/analytics/quick-reference.mdx new file mode 100644 index 0000000000..15ec58b991 --- /dev/null +++ b/apps/docs/analytics/quick-reference.mdx @@ -0,0 +1,210 @@ +--- +title: Quick Reference +description: "Fast lookup for common analytics query patterns and table selection" +--- + +# Analytics Quick Reference + +## Essential Query Patterns + +### Usage Analytics + +**Use for**: High-level usage metrics and health monitoring + +```sql +-- Total verifications (last 7 days) +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 7 DAY + +-- Verifications by outcome (last 30 days) +SELECT outcome, SUM(count) as count +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 30 DAY +GROUP BY outcome +ORDER BY count DESC + +-- Daily usage trend (last 30 days) +SELECT time as date, SUM(count) as verifications +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 30 DAY +GROUP BY date +ORDER BY date +``` + +### User Analytics + +**Use for**: Understanding user behavior and identifying power users + +```sql +-- Top users by usage (last 30 days) +SELECT external_id, SUM(count) as total_verifications +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 30 DAY + AND external_id != '' +GROUP BY external_id +ORDER BY total_verifications DESC +LIMIT 10 + +-- Specific user activity (last 30 days) +SELECT SUM(count) as total_verifications, + SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) as successful +FROM key_verifications_per_day_v1 +WHERE external_id = 'user_123' + AND time >= now() - INTERVAL 30 DAY +``` + +### API Analytics + +**Use for**: Comparing API performance and usage + +```sql +-- Usage per API (last 30 days) +SELECT key_space_id, SUM(count) as total_verifications +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 30 DAY +GROUP BY key_space_id +ORDER BY total_verifications DESC + +-- API success rate comparison (last 7 days) +SELECT key_space_id, + SUM(count) as verifications, + round(SUM(CASE WHEN outcome = 'VALID' THEN count ELSE 0 END) / SUM(count) * 100, 2) as success_rate +FROM key_verifications_per_day_v1 +WHERE key_space_id IN ('ks_1234', 'ks_5678') + AND time >= now() - INTERVAL 7 DAY +GROUP BY key_space_id +``` + +### Billing Queries + +**Use for**: Usage-based billing and credit tracking + +```sql +-- Monthly credits per user +SELECT external_id, + toStartOfMonth(time) as month, + SUM(spent_credits) as total_credits +FROM key_verifications_per_day_v1 +WHERE external_id != '' + AND time >= toStartOfMonth(now()) +GROUP BY external_id, month +ORDER BY total_credits DESC + +-- User tier calculation (current month) +SELECT external_id, SUM(spent_credits) as total_credits, + CASE + WHEN total_credits <= 1000 THEN 'free' + WHEN total_credits <= 10000 THEN 'starter' + WHEN total_credits <= 100000 THEN 'pro' + ELSE 'enterprise' + END as tier +FROM key_verifications_per_day_v1 +WHERE time >= toStartOfMonth(now()) + AND external_id = 'user_123' +GROUP BY external_id +``` + +### Tag-Based Filtering + +**Use for**: Custom metadata filtering and endpoint analysis + +```sql +-- Filter by single tag +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 +WHERE has(tags, 'path=/api/v1/users') + AND time >= now() - INTERVAL 7 DAY + +-- Filter by multiple tags (OR) +SELECT SUM(count) as total +FROM key_verifications_per_day_v1 +WHERE hasAny(tags, ['path=/api/v1/users', 'path=/api/v1/posts']) + AND time >= now() - INTERVAL 7 DAY + +-- Group by endpoint (using path tags) +SELECT arrayJoin(arrayFilter(x -> startsWith(x, 'path='), tags)) as endpoint, + COUNT(*) as requests +FROM key_verifications_v1 +WHERE time >= now() - INTERVAL 24 HOUR +GROUP BY endpoint +ORDER BY requests DESC +``` + +## Table Selection Guide + +Choose the right table based on your time range: + +| Time Range | Recommended Table | When to Use | +| -------------- | --------------------------------- | ----------------------------------------- | +| **< 1 hour** | `key_verifications_v1` | Real-time analysis, detailed debugging | +| **< 24 hours** | `key_verifications_per_minute_v1` | Hourly/daily trends, recent activity | +| **< 30 days** | `key_verifications_per_hour_v1` | Daily/weekly analysis, user behavior | +| **< 1 year** | `key_verifications_per_day_v1` | Monthly/quarterly reports, billing cycles | +| **> 1 year** | `key_verifications_per_month_v1` | Annual trends, long-term analytics | + +**Performance Tips:** + +- Always filter by time first (uses indexes) +- Use `SUM(count)` with aggregated tables, not `COUNT(*)` +- Add `LIMIT` clauses to prevent large result sets +- Filter before grouping when possible + +## Common Filters + +### Time Ranges + +```sql +-- Relative time ranges +WHERE time >= now() - INTERVAL 7 DAY -- Last 7 days +WHERE time >= now() - INTERVAL 24 HOUR -- Last 24 hours +WHERE time >= toStartOfDay(now()) -- Today +WHERE time >= toStartOfMonth(now()) -- This month +``` + +### User & API Filters + +```sql +-- Specific user +WHERE external_id = 'user_123' + +-- Multiple users +WHERE external_id IN ('user_123', 'user_456') + +-- Specific API +WHERE key_space_id = 'ks_1234' + +-- Multiple APIs +WHERE key_space_id IN ('ks_1234', 'ks_5678') +``` + +### Tag Filters + +```sql +-- Has specific tag +WHERE has(tags, 'environment=production') + +-- Has any of multiple tags +WHERE hasAny(tags, ['team=backend', 'team=frontend']) + +-- Has all specified tags +WHERE hasAll(tags, ['environment=prod', 'tier=premium']) +``` + +### Outcome Filters + +```sql +-- Only successful verifications +WHERE outcome = 'VALID' + +-- Only errors +WHERE outcome != 'VALID' + +-- Specific error types +WHERE outcome IN ('RATE_LIMITED', 'USAGE_EXCEEDED') +``` + +## Need More Functions? + +→ [ClickHouse Function Reference](https://clickhouse.com/docs/en/sql-reference/functions) +→ [ClickHouse SQL Documentation](https://clickhouse.com/docs/en/sql-reference) diff --git a/apps/docs/analytics/schema-reference.mdx b/apps/docs/analytics/schema-reference.mdx index 35137faefc..087f98f8cd 100644 --- a/apps/docs/analytics/schema-reference.mdx +++ b/apps/docs/analytics/schema-reference.mdx @@ -6,8 +6,8 @@ description: "Tables, columns, and data types in Unkey Analytics" Unkey Analytics stores verification events across multiple time-series tables for efficient querying. This reference documents all available tables and their columns. - Use aggregated tables (`per_hour`, `per_day`, `per_month`) for queries spanning long time periods - to improve performance. + Use aggregated tables (`per_hour`, `per_day`, `per_month`) for queries + spanning long time periods to improve performance. ## Raw Events Table @@ -52,17 +52,17 @@ Pre-aggregated tables provide better query performance for long time ranges. Eac `key_verifications_per_minute_v1` - Aggregated by minute -| Column | Type | Description | -| -------------------------------- | -------- | ----------------------------------------------- | -| `time` | DateTime | Timestamp (DateTime for minute/hour, Date for day/month) | -| `workspace_id` | String | Workspace identifier | -| `key_space_id` | String | API identifier | -| `external_id` | String | Your user identifier | -| `key_id` | String | API key identifier | -| `outcome` | String | Verification outcome (VALID, RATE_LIMITED, INVALID, etc.) | -| `tags` | Array | Tags associated with verifications | -| `count` | UInt64 | Total verification count for this aggregation | -| `spent_credits` | UInt64 | Total credits spent | +| Column | Type | Description | +| --------------- | -------- | --------------------------------------------------------- | +| `time` | DateTime | Timestamp (DateTime for minute/hour, Date for day/month) | +| `workspace_id` | String | Workspace identifier | +| `key_space_id` | String | API identifier | +| `external_id` | String | Your user identifier | +| `key_id` | String | API key identifier | +| `outcome` | String | Verification outcome (VALID, RATE_LIMITED, INVALID, etc.) | +| `tags` | Array | Tags associated with verifications | +| `count` | UInt64 | Total verification count for this aggregation | +| `spent_credits` | UInt64 | Total credits spent | ### Per Hour Table @@ -259,14 +259,14 @@ WHERE time >= 1704067200000 -- Jan 1, 2024 00:00:00 UTC ### Array Functions -| Function | Description | Example | -| --------------- | ------------------ | --------------------------------------------------- | -| `has()` | Check element | `WHERE has(tags, 'environment=production')` | -| `hasAny()` | Check any element | `WHERE hasAny(tags, ['team=backend', 'team=api'])` | +| Function | Description | Example | +| --------------- | ------------------ | ---------------------------------------------------- | +| `has()` | Check element | `WHERE has(tags, 'environment=production')` | +| `hasAny()` | Check any element | `WHERE hasAny(tags, ['team=backend', 'team=api'])` | | `hasAll()` | Check all elements | `WHERE hasAll(tags, ['environment=prod', 'tier=1'])` | -| `arrayJoin()` | Expand array | `SELECT arrayJoin(tags) as tag` | -| `arrayFilter()` | Filter array | `arrayFilter(x -> startsWith(x, 'path='), tags)` | -| `length()` | Array length | `WHERE length(tags) > 0` | +| `arrayJoin()` | Expand array | `SELECT arrayJoin(tags) as tag` | +| `arrayFilter()` | Filter array | `arrayFilter(x -> startsWith(x, 'path='), tags)` | +| `length()` | Array length | `WHERE length(tags) > 0` | ### Math Functions @@ -302,21 +302,3 @@ WHERE time >= 1704067200000 -- Jan 1, 2024 00:00:00 UTC | Queries per hour | 1000 | `query_quota_exceeded` | See [Query Restrictions](/analytics/query-restrictions) for more details on query limits and restrictions. - -## Next Steps - - - - Explore common SQL patterns for analytics - - - View limits, quotas, and permissions - - - Browse ClickHouse SQL reference (external) - - diff --git a/apps/docs/analytics/troubleshooting.mdx b/apps/docs/analytics/troubleshooting.mdx new file mode 100644 index 0000000000..1dd4910adc --- /dev/null +++ b/apps/docs/analytics/troubleshooting.mdx @@ -0,0 +1,145 @@ +--- +title: Troubleshooting +description: "Common issues and solutions for analytics queries" +--- + +# Analytics Troubleshooting + +## Getting Empty Results? + +**Check these common causes:** + +### Time Range Issues + +- Your workspace might be new with no verification data yet +- Try a shorter time range: `WHERE time >= now() - INTERVAL 1 HOUR` +- Verify your time filters are working: `SELECT MAX(time) as latest FROM key_verifications_v1` + +### Filter Problems + +- Wrong `key_space_id` - Find your API ID in dashboard settings +- Empty `external_id` values - Filter them out: `WHERE external_id != ''` +- Case sensitivity - Check exact values: `SELECT DISTINCT external_id FROM key_verifications_v1 LIMIT 10` + +### Data Availability + +- Analytics may have a few minutes delay +- Check recent data: `SELECT COUNT(*) FROM key_verifications_v1 WHERE time >= now() - INTERVAL 1 HOUR` + +## Queries Timing Out? + +**Try these optimizations:** + +### Use Aggregated Tables + +```sql +-- Instead of raw table for long ranges: +-- ❌ Slow for 7+ days +SELECT COUNT(*) FROM key_verifications_v1 WHERE time >= now() - INTERVAL 7 DAY + +-- ✅ Fast for 7+ days +SELECT SUM(count) FROM key_verifications_per_day_v1 WHERE time >= now() - INTERVAL 7 DAY +``` + +### Add Time Filters First + +```sql +-- Always filter by time early (uses indexes) +WHERE time >= now() - INTERVAL 7 DAY + AND external_id = 'user_123' -- Additional filters after time +``` + +### Limit Result Size + +```sql +-- Prevent large result sets +SELECT external_id, SUM(count) as usage +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 30 DAY +GROUP BY external_id +ORDER BY usage DESC +LIMIT 100 -- Add LIMIT for large datasets +``` + +## API Errors? + +**Common fixes:** + +### Missing Property 'query' + +```json +// ❌ Wrong field name +{"sql": "SELECT COUNT(*) FROM key_verifications_v1"} + +// ✅ Correct field name +{"query": "SELECT COUNT(*) FROM key_verifications_v1"} +``` + +See [invalid_input](/errors/unkey/application/invalid_input) for API request format issues. + +### Permission Denied + +- Ensure your root key has `api.*.read_analytics` or `api..read_analytics` +- Check key is not expired or revoked +- Verify workspace ID matches your key's permissions + See [forbidden](/errors/unkey/authorization/forbidden) for permission issues. + +### Invalid Function Error + +- Check [Query Restrictions](/analytics/query-restrictions#function-allow-list) for allowed functions +- Some ClickHouse functions are blocked for security +- Use alternative approaches from [Quick Reference](/analytics/quick-reference) +- See [invalid_analytics_function](/errors/user/bad_request/invalid_analytics_function) for details + +### Invalid Table Error + +- Only analytics tables are accessible (no `system.*` or `information_schema.*`) +- Use table names from [Schema Reference](/analytics/schema-reference) +- See [invalid_analytics_table](/errors/user/bad_request/invalid_analytics_table) for details + +### Query Not Supported + +- Only SELECT queries are allowed in analytics +- INSERT, UPDATE, DELETE, etc. are blocked +- See [invalid_analytics_query_type](/errors/user/bad_request/invalid_analytics_query_type) for details + +## Performance Problems? + +**Optimization tips:** + +### Choose Right Table + +- `< 1 hour`: `key_verifications_v1` (raw) +- `< 24 hours`: `key_verifications_per_minute_v1` +- `< 30 days`: `key_verifications_per_hour_v1` +- `< 1 year`: `key_verifications_per_day_v1` +- `> 1 year`: `key_verifications_per_month_v1` + +### Filter Before Grouping + +```sql +-- ❌ Less efficient +SELECT external_id, SUM(count) as usage +FROM key_verifications_per_day_v1 +GROUP BY external_id +HAVING SUM(count) > 1000 + +-- ✅ More efficient +SELECT external_id, SUM(count) as usage +FROM key_verifications_per_day_v1 +WHERE time >= now() - INTERVAL 30 DAY -- Filter first +GROUP BY external_id +HAVING SUM(count) > 1000 +``` + +### Avoid SELECT \* + +- Only select columns you need +- Reduces memory usage and network transfer + + + For complete error reference, see the [Error Documentation](/errors/overview). + If you continue having issues, contact us at + [support@unkey.dev](mailto:support@unkey.dev) with your query and error + details. + diff --git a/apps/docs/docs.json b/apps/docs/docs.json index 0f1c806bd8..c06bf1271d 100644 --- a/apps/docs/docs.json +++ b/apps/docs/docs.json @@ -113,7 +113,10 @@ { "group": "Identities", "icon": "fingerprint", - "pages": ["concepts/identities/overview", "concepts/identities/ratelimits"] + "pages": [ + "concepts/identities/overview", + "concepts/identities/ratelimits" + ] } ] }, @@ -169,7 +172,7 @@ }, { "group": "Analytics", - "hidden": true, + "hidden": false, "icon": "chart-bar", "pages": [ "analytics/overview", @@ -269,7 +272,9 @@ }, { "group": "Too Many Requests", - "pages": ["errors/user/too_many_requests/query_quota_exceeded"] + "pages": [ + "errors/user/too_many_requests/query_quota_exceeded" + ] }, { "group": "Unprocessable Entity",