Skip to content

perf(logsdb): query optimization, shared code extraction, CI improvements #219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
strawgate merged 15 commits into from
Apr 28, 2026
Merged

perf(logsdb): query optimization, shared code extraction, CI improvements #219

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
6e08b77
perf(logsdb): raw byte scan eliminates non-matching chunks 4× faster
strawgate Apr 28, 2026
d19a19f
perf(logsdb): simplify body fast path — single decode pass
strawgate Apr 28, 2026
bd0645b
test(logsdb): expand test suite from 82 to 142 tests
strawgate Apr 28, 2026
ddf16e5
test(logsdb): expand test suite from 142 to 216 tests
strawgate Apr 28, 2026
c98eece
fix(logsdb): eliminate double-decompress, fix falsy field loss, add t…
strawgate Apr 28, 2026
c01a536
refactor: extract shared StreamRegistry and hex utils to stardb
strawgate Apr 28, 2026
bd6d0d9
refactor: extract ByteBuf, chunk wire format, FNV-1a, codec-utils to …
strawgate Apr 28, 2026
1966013
stardb: 100% coverage + extract anyValue/findAttribute/StreamKey
strawgate Apr 28, 2026
064cb0e
stardb: add timeRangeOverlaps, UUID helpers; dedup sidecar NDJSON
strawgate Apr 28, 2026
268bf2c
refactor(tracesdb): use findAttribute from stardb across all modules
strawgate Apr 28, 2026
718b180
perf(logsdb): selective sidecar decode for bodyContains queries
strawgate Apr 28, 2026
1c34d0b
refactor: extract shared code to stardb core package
strawgate Apr 28, 2026
fb7da13
test(stardb): add comprehensive tests for extracted shared modules
strawgate Apr 28, 2026
fa98631
refactor: remove re-export wrappers, import stardb directly
strawgate Apr 28, 2026
d3cfefb
fix: resolve all biome lint errors for CI
strawgate Apr 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 69 additions & 10 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,48 @@ env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"

jobs:
validate:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: 24
cache: npm
cache-dependency-path: package-lock.json
Comment on lines +17 to +24
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether workflow actions are SHA-pinned (40-hex) or tag-pinned.
# Expected: third-party `uses:` lines should use @<40-hex-sha> for hardening.

echo "All action references:"
rg -nP '^\s*uses:\s*[^@]+@[^ \t]+' .github/workflows/*.yml

echo
echo "SHA-pinned action references:"
rg -nP '^\s*uses:\s*[^@]+@[a-f0-9]{40}\b' .github/workflows/*.yml

Repository: strawgate/o11ykit

Length of output: 1913

Pin all third-party GitHub Actions to commit SHAs.

All action usages throughout the repository are pinned to mutable tags (@v6, @v5, etc.) instead of immutable commit SHAs. This weakens CI supply-chain guarantees and affects not only .github/workflows/ci.yml but also publish-octo11y.yml, release.yml, pages.yml, benchmarks.yml, and actions-dist.yml. Replace all @v<number> references with @<40-character-sha>.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 17 - 24, Replace mutable action tags
with immutable commit SHAs: locate usages like actions/checkout@v6 and
actions/setup-node@v6 in .github/workflows/ci.yml and the other workflow files
(publish-octo11y.yml, release.yml, pages.yml, benchmarks.yml, actions-dist.yml)
and update each reference to the corresponding 40-character commit SHA (e.g.,
actions/checkout@<sha>, actions/setup-node@<sha>) so all third-party actions are
pinned to a specific commit; ensure you verify the correct SHA from the action's
GitHub releases/tags page and update any other `@v`<number> occurrences
consistently.


- name: Install dependencies
run: npm ci --include=dev

- name: Lint
run: npm run lint

typecheck:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: 24
cache: npm
cache-dependency-path: package-lock.json

- name: Install dependencies
run: npm ci --include=dev

- name: Install octo11y dependencies
run: npm run octo11y:install

- name: Typecheck
run: npm run typecheck

test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
Expand All @@ -32,15 +73,8 @@ jobs:
- name: Install dependencies
run: npm ci --include=dev

- name: Install octo11y dependencies
run: npm run octo11y:install

- name: Validate workspace
run: make check

- name: Validate publish artifacts
if: matrix.node-version == 24
run: npm run check:release
- name: Test
run: npm run test

- name: Upload coverage report
if: matrix.node-version == 24
Expand All @@ -49,6 +83,31 @@ jobs:
name: coverage-report
path: coverage

build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: 24
cache: npm
cache-dependency-path: package-lock.json

- name: Install dependencies
run: npm ci --include=dev

- name: Install octo11y dependencies
run: npm run octo11y:install

- name: Build
run: npm run build

- name: Validate publish artifacts
run: npm run check:release

validate-octo11y:
runs-on: ubuntu-latest
steps:
Expand Down
1 change: 1 addition & 0 deletions .husky/pre-commit
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
npx lint-staged
11 changes: 10 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@ SHELL := /bin/bash
NPM ?= npm
BASE_PATH ?= /o11ykit/otlpkit/

.PHONY: install lint format typecheck site-typecheck test test-e2e build check check-release check-all clean clean-all
.PHONY: install lint format typecheck site-typecheck test test-fast test-e2e build check check-release check-all clean clean-all
.PHONY: dev-demo dev-chartjs dev-echarts dev-recharts dev-uplot pages-build
.PHONY: octo11y-install octo11y-lint octo11y-test octo11y-build octo11y-check
.PHONY: knip

install:
$(NPM) ci
Expand All @@ -24,12 +25,20 @@ site-typecheck:
test:
$(NPM) run test

# Fast unit tests (no coverage, no E2E)
test-fast:
npx vitest run --no-coverage

test-e2e:
$(NPM) run test:e2e

build:
$(NPM) run build

# Dead code / unused export analysis
knip:
npx knip

check:
$(NPM) run check

Expand Down
46 changes: 46 additions & 0 deletions knip.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"$schema": "https://unpkg.com/knip@latest/schema.json",
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify current schema pinning and installed knip version.
# Expected: schema URL should not use `@latest` and should align with installed knip major.

echo "knip devDependency:"
jq -r '.devDependencies.knip' package.json

echo "knip schema URL:"
jq -r '.["$schema"]' knip.json

Repository: strawgate/o11ykit

Length of output: 147

Pin Knip schema to a specific version instead of @latest.

The schema URL currently uses @latest, which can introduce silent schema drift between local development and CI/CD environments. Align the schema version with the installed Knip dependency (^6.7.0). Use a pinned major or exact version—for example, https://unpkg.com/knip@6/schema.json or https://unpkg.com/knip@6.7.0/schema.json.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@knip.json` at line 2, The "$schema" entry in knip.json currently points to
"https://unpkg.com/knip@latest/schema.json" which can cause schema drift; change
the URL to pin the Knip major or exact version that matches your dependency
(e.g., "https://unpkg.com/knip@6/schema.json" or
"https://unpkg.com/knip@6.7.0/schema.json") so the "$schema" value aligns with
the installed ^6.7.0 dependency.

"workspaces": {
".": {
"entry": ["scripts/*.{ts,js}"],
"ignore": ["coverage/**", ".site/**", "octo11y/**"]
},
"packages/otlpjson": {
"entry": ["src/index.ts"],
"ignoreDependencies": ["typescript"]
},
"packages/query": {
"entry": ["src/index.ts"],
"ignoreDependencies": ["typescript"]
},
"packages/views": {
"entry": ["src/index.ts"],
"ignoreDependencies": ["typescript"]
},
"packages/adapters": {
"entry": ["src/index.ts"],
"ignoreDependencies": ["typescript"]
},
"packages/stardb": {
"entry": ["src/index.ts"],
"ignoreDependencies": ["typescript"]
},
"packages/o11ytsdb": {
"entry": ["src/index.ts"],
"ignore": ["bench/**"],
"ignoreDependencies": ["typescript"]
},
"packages/o11ylogsdb": {
"entry": ["src/index.ts"],
"ignore": ["bench/**"],
"ignoreDependencies": ["typescript"]
},
"packages/o11ytracesdb": {
"entry": ["src/index.ts"],
"ignore": ["bench/**"],
"ignoreDependencies": ["typescript"]
}
},
"ignore": ["**/dist/**", "**/node_modules/**", "octo11y/**"],
"ignoreDependencies": ["@biomejs/biome", "husky", "lint-staged", "knip"]
}
Loading
Loading