Repository navigation
⚡ Bolt: [성능 개선] computeTaskMetrics 최적화 #508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
36
commits into
develop
Choose a base branch
from
bolt-optimize-compute-metrics-500206433809831171
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 17 commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
a6e271a
perf(computeTaskMetrics): replace map/reduce with int32array/loops
seonghobae 7d07992
test(perf): lock task metric cache semantics
seonghobae e394998
test(perf): run task metric regression in coverage
seonghobae d19d560
test(perf): add exact-head task metric benchmark evidence
seonghobae 3f3b6e6
perf(computeTaskMetrics): replace map/reduce with int32array/loops
seonghobae a118883
test(perf): benchmark metric computation against protected base
seonghobae e1ca5b7
ci(perf): run exact-base metrics benchmark
seonghobae 0094fc0
test(perf): instrument metric benchmark without production globals
seonghobae 765583d
Merge branch 'develop' into bolt-optimize-compute-metrics-50020643380…
opencode-agent[bot] cd52c06
merge(perf): reconcile protected XML import baseline
seonghobae 038b8e3
Merge protected develop into metrics performance branch
seonghobae 514b086
test(perf): require immutable benchmark base on push
seonghobae 5a17020
fix(perf): resolve benchmark base for protected pushes
seonghobae 6bb681d
fix(perf): fail closed on missing benchmark baselines
seonghobae e27f57d
test(perf): run benchmark base contract in unit CI
seonghobae 24a4e5c
test(perf): always close benchmark browser contexts
seonghobae 3a48fcf
merge(develop): reconcile metrics performance with OpenCode config
seonghobae 1699198
merge(develop): reconcile metrics performance with current protected …
seonghobae 0ae5a54
fix(stack): preserve protected orchestrator while reconciling metrics…
seonghobae 77daa1d
fix(stack): inherit protected Hono runtime baseline
seonghobae e77846d
fix(stack): inherit protected Playwright 1.62.1 in metrics benchmark
seonghobae da9b54e
test(perf): expose metrics benchmark order bias
seonghobae 9b6d98f
fix(perf): counterbalance metrics benchmark timings
seonghobae f119ac3
fix(perf): counterbalance metrics browser benchmark
seonghobae b36361e
test(perf): require immutable benchmark candidate
seonghobae 32bc31b
fix(perf): bind metrics benchmark candidate identity
seonghobae 59a8f03
fix(perf): benchmark exact contributor revision
seonghobae 1cc8b59
test(perf): expose stale protected-base benchmark
seonghobae f6fafe1
fix(perf): verify live protected benchmark base
seonghobae 388406f
fix(perf): reject stale protected-base benchmark
seonghobae 642034f
test(perf): isolate benchmark from cloud e2e
seonghobae b0108ae
perf(computeTaskMetrics): replace map/reduce with float64array/loops
seonghobae 4e62f3d
ci: re-kick required checks to bypass flake
seonghobae 745e1f1
ci: re-kick required checks to bypass flake
seonghobae f74c1f6
ci: re-kick required checks to bypass flake
seonghobae 8b7d657
ci: re-kick required checks to bypass flake
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,217 @@ | ||
| import { execFileSync } from 'node:child_process'; | ||
| import { readFileSync } from 'node:fs'; | ||
|
|
||
| import { test, expect } from '@playwright/test'; | ||
|
|
||
| import { resolveBenchmarkBaseSha } from '../helpers/benchmark-base.mjs'; | ||
|
|
||
| test.describe.configure({ retries: process.env.CI ? 2 : 0 }); | ||
|
|
||
| const TASK_COUNT = 10_000; | ||
| const SAMPLE_COUNT = 7; | ||
| const WARMUP_COUNT = 3; | ||
| const TARGET_IMPROVEMENT_PERCENT = 15; | ||
| const BASE_DATE = '2026-02-15'; | ||
| const CANDIDATE_APP_SOURCE = readFileSync(new URL('../../app.js', import.meta.url), 'utf8'); | ||
|
|
||
| const DATE_WINDOWS = Object.freeze([ | ||
| ['2026-01-01', '2026-01-02'], | ||
| ['2026-01-02', '2026-01-12'], | ||
| ['2026-02-01', '2026-03-01'], | ||
| ['2026-02-15', '2026-02-15'], | ||
| ]); | ||
|
|
||
| function protectedBaseSha() { | ||
| const eventPath = process.env.GITHUB_EVENT_PATH; | ||
| const event = eventPath ? JSON.parse(readFileSync(eventPath, 'utf8')) : {}; | ||
| return resolveBenchmarkBaseSha({ | ||
| override: process.env.SCOPEWEAVE_BENCHMARK_BASE_SHA, | ||
| event, | ||
| }); | ||
| } | ||
|
|
||
| function readGitFile(commitSha, path) { | ||
| const normalizedCommitSha = String(commitSha || ''); | ||
| if (!/^[a-f0-9]{40}$/.test(normalizedCommitSha)) { | ||
| throw new Error(`Invalid benchmark base SHA: ${normalizedCommitSha || '<missing>'}`); | ||
| } | ||
|
|
||
| const spec = `${normalizedCommitSha}:${path}`; | ||
| try { | ||
| return execFileSync('git', ['show', spec], { encoding: 'utf8' }); | ||
| } catch { | ||
| execFileSync('git', ['fetch', '--depth=1', 'origin', normalizedCommitSha], { | ||
| encoding: 'utf8', | ||
| stdio: ['ignore', 'pipe', 'pipe'], | ||
| }); | ||
| return execFileSync('git', ['show', spec], { encoding: 'utf8' }); | ||
| } | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| function instrumentMetricsSource(source) { | ||
| const bootstrapCall = '\nbootstrap();'; | ||
| const bootstrapIndex = source.lastIndexOf(bootstrapCall); | ||
| if (bootstrapIndex === -1) { | ||
| throw new Error('Benchmark app source is missing the expected bootstrap call'); | ||
| } | ||
|
|
||
| const withoutBootstrap = `${source.slice(0, bootstrapIndex)}${source.slice(bootstrapIndex + bootstrapCall.length)}`; | ||
| return `${withoutBootstrap}\n\nwindow.__scopeweaveMetricsBenchmark = Object.freeze({\n seed(tasks, baseDate) {\n state.tasks = tasks;\n state.baseDate = baseDate;\n },\n compute() {\n return computeTaskMetrics();\n },\n});\n`; | ||
| } | ||
|
|
||
| function createTask(index) { | ||
| const [plannedStartDate, plannedEndDate] = DATE_WINDOWS[index % DATE_WINDOWS.length]; | ||
| return { | ||
| id: `metrics-performance-${index}`, | ||
| parentId: null, | ||
| depth: 1, | ||
| expanded: true, | ||
| pendingDelete: false, | ||
| isSynthetic: false, | ||
| phase: `Phase ${index}`, | ||
| activity: '', | ||
| task: '', | ||
| categoryLarge: '', | ||
| categoryMedium: '', | ||
| documentName: '', | ||
| owner: `owner-${index % 17}`, | ||
| supportTeam: '', | ||
| plannedStartDate, | ||
| plannedEndDate, | ||
| actualProgressStatus: '미착수(0%)', | ||
| actualStartDate: '', | ||
| actualEndDate: '', | ||
| predecessors: '', | ||
| budget: '', | ||
| actualCost: '', | ||
| sprint: '', | ||
| storyPoints: '', | ||
| }; | ||
| } | ||
|
|
||
| function median(values) { | ||
| const sorted = [...values].sort((left, right) => left - right); | ||
| return sorted[Math.floor(sorted.length / 2)]; | ||
| } | ||
|
|
||
| async function measureMetrics(browser, { appSource, label }) { | ||
| const context = await browser.newContext(); | ||
| try { | ||
| const page = await context.newPage(); | ||
| const instrumentedSource = instrumentMetricsSource(appSource); | ||
|
|
||
| await page.route('**/app.js', async (route) => { | ||
| await route.fulfill({ | ||
| status: 200, | ||
| contentType: 'application/javascript; charset=utf-8', | ||
| body: instrumentedSource, | ||
| }); | ||
| }); | ||
|
|
||
| await page.goto('/'); | ||
| const tasks = Array.from({ length: TASK_COUNT }, (_, index) => createTask(index)); | ||
|
|
||
| const result = await page.evaluate(async ({ seededTasks, baseDate, sampleCount, warmupCount }) => { | ||
| const benchmark = window.__scopeweaveMetricsBenchmark; | ||
| if (!benchmark) throw new Error('metrics benchmark bridge did not initialize'); | ||
| benchmark.seed(seededTasks, baseDate); | ||
|
|
||
| for (let warmup = 0; warmup < warmupCount; warmup += 1) { | ||
| benchmark.compute(); | ||
| } | ||
|
|
||
| const samples = []; | ||
| for (let sample = 0; sample < sampleCount; sample += 1) { | ||
| const startedAt = performance.now(); | ||
| benchmark.compute(); | ||
| samples.push(performance.now() - startedAt); | ||
| } | ||
|
seonghobae marked this conversation as resolved.
Resolved
|
||
|
|
||
| const metrics = benchmark.compute(); | ||
| const entries = Array.from(metrics.byTask, ([taskId, taskMetrics]) => [ | ||
| taskId, | ||
| taskMetrics.durationDays, | ||
| taskMetrics.weightRatio, | ||
| taskMetrics.plannedProgressRatio, | ||
| taskMetrics.actualProgressRatio, | ||
| taskMetrics.weightedPlannedRatio, | ||
| taskMetrics.weightedActualRatio, | ||
| taskMetrics.progressState.label, | ||
| taskMetrics.progressState.className, | ||
| taskMetrics.plannedDateWarning, | ||
| taskMetrics.actualDateWarning, | ||
| ]); | ||
| const snapshot = JSON.stringify({ | ||
| totalDays: metrics.totalDays, | ||
| totalWeightedPlannedRatio: metrics.totalWeightedPlannedRatio, | ||
| totalWeightedActualRatio: metrics.totalWeightedActualRatio, | ||
| entries, | ||
| }); | ||
| const digestBytes = new Uint8Array(await crypto.subtle.digest( | ||
| 'SHA-256', | ||
| new TextEncoder().encode(snapshot), | ||
| )); | ||
| const digest = Array.from(digestBytes, (byte) => byte.toString(16).padStart(2, '0')).join(''); | ||
|
|
||
| return { | ||
| samples, | ||
| digest, | ||
| totalDays: metrics.totalDays, | ||
| byTaskSize: metrics.byTask.size, | ||
| totalWeightedPlannedRatio: metrics.totalWeightedPlannedRatio, | ||
| totalWeightedActualRatio: metrics.totalWeightedActualRatio, | ||
| }; | ||
| }, { | ||
| seededTasks: tasks, | ||
| baseDate: BASE_DATE, | ||
| sampleCount: SAMPLE_COUNT, | ||
| warmupCount: WARMUP_COUNT, | ||
| }); | ||
|
|
||
| return { | ||
| label, | ||
| ...result, | ||
| medianDurationMs: median(result.samples), | ||
| }; | ||
| } finally { | ||
| await context.close(); | ||
| } | ||
| } | ||
|
|
||
| test('10,000-task metric computation preserves exact semantics and beats the protected base', async ({ browser }) => { | ||
| test.setTimeout(120_000); | ||
|
|
||
| const baseSha = protectedBaseSha(); | ||
| const baselineSource = readGitFile(baseSha, 'app.js'); | ||
| const baseline = await measureMetrics(browser, { appSource: baselineSource, label: 'protected-base' }); | ||
| const optimized = await measureMetrics(browser, { appSource: CANDIDATE_APP_SOURCE, label: 'candidate' }); | ||
|
|
||
| expect(optimized.byTaskSize).toBe(TASK_COUNT); | ||
| expect(optimized.samples).toHaveLength(SAMPLE_COUNT); | ||
| expect(optimized.medianDurationMs).toBeGreaterThan(0); | ||
|
|
||
| expect(baseline.byTaskSize).toBe(TASK_COUNT); | ||
| expect(optimized.digest).toBe(baseline.digest); | ||
| expect(optimized.totalDays).toBe(baseline.totalDays); | ||
| expect(optimized.totalWeightedPlannedRatio).toBe(baseline.totalWeightedPlannedRatio); | ||
| expect(optimized.totalWeightedActualRatio).toBe(baseline.totalWeightedActualRatio); | ||
|
|
||
| const optimizationDeltaPercent = ((baseline.medianDurationMs - optimized.medianDurationMs) | ||
| / baseline.medianDurationMs) * 100; | ||
| expect( | ||
| optimizationDeltaPercent, | ||
| `expected >=${TARGET_IMPROVEMENT_PERCENT}% median computeTaskMetrics improvement over ${baseSha}, got ${optimizationDeltaPercent.toFixed(2)}%`, | ||
| ).toBeGreaterThanOrEqual(TARGET_IMPROVEMENT_PERCENT); | ||
|
|
||
| console.log(`SCOPEWEAVE_METRICS_BENCHMARK ${JSON.stringify({ | ||
| taskCount: TASK_COUNT, | ||
| sampleCount: SAMPLE_COUNT, | ||
| warmupCount: WARMUP_COUNT, | ||
| protectedBaseSha: baseSha, | ||
| protectedBaselineAvailable: true, | ||
| targetImprovementPercent: TARGET_IMPROVEMENT_PERCENT, | ||
| optimizationDeltaPercent, | ||
| baseline, | ||
| optimized, | ||
| })}`); | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| const COMMIT_SHA_PATTERN = /^[0-9a-f]{40}$/i; | ||
| const ZERO_COMMIT_SHA = '0'.repeat(40); | ||
|
|
||
| function canonicalCommitSha(value) { | ||
| const sha = String(value || '').trim().toLowerCase(); | ||
| if (!COMMIT_SHA_PATTERN.test(sha) || sha === ZERO_COMMIT_SHA) { | ||
| throw new Error(`Benchmark base SHA is invalid: ${sha || '<missing>'}`); | ||
| } | ||
| return sha; | ||
| } | ||
|
|
||
| /** | ||
| * Resolve the immutable revision that a performance run must compare against. | ||
| * | ||
| * Pull-request runs compare to the PR base snapshot that triggered the run. | ||
| * Protected-branch push runs compare to the immediately previous protected | ||
| * commit from the push event. Operators may provide an explicit immutable SHA | ||
| * when replaying the benchmark outside those GitHub event shapes. | ||
| * | ||
| * @param {{override?: unknown, event?: unknown}} input benchmark authority input | ||
| * @returns {string} canonical 40-character commit SHA | ||
| */ | ||
| export function resolveBenchmarkBaseSha({ override, event } = {}) { | ||
| const explicit = String(override || '').trim(); | ||
| if (explicit) return canonicalCommitSha(explicit); | ||
|
|
||
| const eventObject = event && typeof event === 'object' && !Array.isArray(event) | ||
| ? event | ||
| : {}; | ||
| const pullRequestBase = eventObject.pull_request?.base?.sha; | ||
| if (pullRequestBase) return canonicalCommitSha(pullRequestBase); | ||
|
|
||
| const pushBefore = eventObject.before; | ||
| if (pushBefore) return canonicalCommitSha(pushBefore); | ||
|
|
||
| throw new Error('Benchmark base SHA is unavailable; provide an immutable comparison revision.'); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| import assert from 'node:assert/strict'; | ||
| import test from 'node:test'; | ||
|
|
||
| import { resolveBenchmarkBaseSha } from '../helpers/benchmark-base.mjs'; | ||
|
|
||
| const PR_BASE_SHA = '1111111111111111111111111111111111111111'; | ||
| const PUSH_BEFORE_SHA = '2222222222222222222222222222222222222222'; | ||
| const OVERRIDE_SHA = '3333333333333333333333333333333333333333'; | ||
|
|
||
| test('benchmark base prefers an explicit immutable override', () => { | ||
| assert.equal(resolveBenchmarkBaseSha({ | ||
| override: OVERRIDE_SHA, | ||
| event: { | ||
| pull_request: { base: { sha: PR_BASE_SHA } }, | ||
| before: PUSH_BEFORE_SHA, | ||
| }, | ||
| }), OVERRIDE_SHA); | ||
| }); | ||
|
|
||
| test('benchmark base uses the pull-request base snapshot for pull_request runs', () => { | ||
| assert.equal(resolveBenchmarkBaseSha({ | ||
| override: '', | ||
| event: { pull_request: { base: { sha: PR_BASE_SHA } } }, | ||
| }), PR_BASE_SHA); | ||
| }); | ||
|
|
||
| test('benchmark base uses the previous protected commit for push runs', () => { | ||
| assert.equal(resolveBenchmarkBaseSha({ | ||
| override: '', | ||
| event: { before: PUSH_BEFORE_SHA }, | ||
| }), PUSH_BEFORE_SHA); | ||
| }); | ||
|
|
||
| test('benchmark base fails closed when no immutable comparison revision exists', () => { | ||
| assert.throws( | ||
| () => resolveBenchmarkBaseSha({ override: '', event: {} }), | ||
| /benchmark base SHA is unavailable/i, | ||
| ); | ||
| }); | ||
|
|
||
| test('benchmark base rejects malformed and all-zero revisions', () => { | ||
| for (const sha of ['not-a-sha', '0'.repeat(40)]) { | ||
| assert.throws( | ||
| () => resolveBenchmarkBaseSha({ override: sha, event: {} }), | ||
| /benchmark base SHA is invalid/i, | ||
| ); | ||
| } | ||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.