perf(exports): retain chart benchmark evidence

.jules/bolt.md

@@ -61,6 +61,7 @@
 ## 2026-07-13 - Array.from mapping optimization
 **Learning:** Using `Array.from({ length: N }).map(...)` creates an intermediate array of `undefined` values which requires memory allocation and garbage collection, adding O(N) unnecessary overhead in frequently re-rendered UI components.
 **Action:** Use `Array.from({ length: N }, (_, index) => ...)` to map elements directly during array creation, avoiding intermediate allocations.
-## 2026-03-06 - [파이썬 O(N^2) 리스트 룩업을 O(1) 딕셔너리로 최적화]
-**Learning:** `chart.py`의 텍스트 변환 로직에서 `not in list`로 중복을 방지하며 삽입하는 방식은 리스트 크기가 커질 때 O(N^2) 병목을 유발합니다. 파이썬 3.7+부터 딕셔너리가 삽입 순서를 유지하므로, `ordered_role_ids[role_id] = None`처럼 의미가 드러나는 키 저장소를 사용하면 순서를 보존하면서 평균 O(1) 조회가 가능합니다.
-**Action:** 순서 보존 중복 제거가 필요한 경로에서는 도메인 이름을 가진 딕셔너리 키를 사용하고, 외부 문자열은 해시·truthiness 연산 전에 안전한 built-in 문자열로 정규화합니다.
+
+## 2026-09-08 - O(N^2) list-based deduplication replaced with O(1) dict keys
+**Learning:** Checking for element existence in a list using `not in` before appending leads to O(N^2) time complexity. However, for bounded small lists ($N < 10$), standard list traversal in CPython can be marginally faster and use less memory overhead than hashing/allocating dict keys. For unbounded or large cardinalities (e.g., thousands of deduplications across a large song export payload with 1000+ sections and highly duplicated roles), dictionary O(1) insertions preserve insertion order while preventing super-linear CPU bounds.
+**Action:** Replace `if item not in lst: lst.append(item)` patterns with `dct[item] = None` and `list(dct.keys())` for efficient and order-preserving deduplication in high-throughput data exports, provided we can demonstrate concrete wall-clock wins under profiling.

CHANGELOG.md

@@ -10,7 +10,8 @@
 
 ### Changed
 
-- Changed chart-export role, cue, and priority de-duplication to semantically named insertion-ordered dictionaries, preserving first-occurrence output while replacing repeated linear membership scans with average constant-time key lookups.
+- Kept chart-export role, cue, and footer de-duplication expected-linear with insertion-ordered dictionaries, semantic internal names, and exact Unicode/order/blank-value regression coverage.
+- Made the retained chart-export benchmark reproduce its documented 96-section, 24-role, 100-warmup, 1,000-sample method and report per-sample median and p95 latency.
 - Consolidated Bandit, dependency audits, supplemental secret checks, and Trivy into one trusted-branch security backstop, delegated CodeQL to GitHub default setup, and removed duplicate local PR security and release-preflight runs.
 - Pinned npm `10.9.9` as the approved lockfile generator, activated it through Node-bundled Corepack before dependency consumption, and fail closed unless its bundled `tar` is at least `7.5.19`; primary CI still consumes the committed lock only through frozen `npm ci` validation, rejects mutable npm resolution in the lock gate, requires integrity evidence for public-registry lock entries, and preserves generator-sensitive root `@esbuild/*` peer metadata.
 

services/analysis-engine/src/bandscope_analysis/exports/chart.py

@@ -7,7 +7,7 @@
 Security Notes:
     - Pure dict-to-string transformation: no file, network, or process I/O.
     - Never reads source-path fields and never emits filesystem paths.
-    - Safe failure: ``None``, empty, or malformed input yields ``\"\"`` / ``[]``;
+    - Safe failure: ``None``, empty, or malformed input yields ``""`` / ``[]``;
       missing or malformed keys are skipped and no exceptions escape.
 """
 
@@ -73,85 +73,73 @@ def _section_roles(section: Mapping[str, object]) -> list[Mapping[str, object]]:
     return [role for role in roles if isinstance(role, Mapping)]
 
 
-def _hashable_text(raw_text_value: object) -> str | None:
-    """Return compatible string-like text as a safe built-in mapping key."""
-    if not isinstance(raw_text_value, str):
-        return None
-    try:
-        hash(raw_text_value)
-        normalized_text = str.__str__(raw_text_value)
-    except Exception:
-        return None
-    return normalized_text if normalized_text else None
-
-
-def _active_role_ids(section_payload: Mapping[str, object]) -> list[str] | None:
+def _active_role_ids(section_record: Mapping[str, object]) -> list[str] | None:
     """Return active role ids from the part graph, or ``None`` when absent."""
-    part_graph = section_payload.get("partGraph")
-    if not isinstance(part_graph, list):
+    part_graph_nodes = section_record.get("partGraph")
+    if not isinstance(part_graph_nodes, list):
         return None
-    active_role_ids_by_id: dict[str, None] = {}
-    for part_graph_node in part_graph:
+    active_role_ids_by_value: dict[str, None] = {}
+    for part_graph_node in part_graph_nodes:
         if not isinstance(part_graph_node, Mapping) or part_graph_node.get("is_active") is not True:
             continue
-        role_id = _hashable_text(part_graph_node.get("role_id"))
-        if role_id is not None:
-            active_role_ids_by_id[role_id] = None
-    return list(active_role_ids_by_id)
+        role_identifier = part_graph_node.get("role_id")
+        if isinstance(role_identifier, str) and role_identifier:
+            active_role_ids_by_value[role_identifier] = None
+    return list(active_role_ids_by_value)
 
 
-def _active_roles(section_payload: Mapping[str, object]) -> list[Mapping[str, object]]:
+def _active_roles(section: Mapping[str, object]) -> list[Mapping[str, object]]:
     """Return the section's active role payloads.
 
     Activity is derived from the part graph's ``is_active`` flags; when the
     part graph is absent the roles list itself is treated as active. Active
     graph nodes without a matching role payload keep their ``role_id`` as a
     display name.
     """
-    section_role_payloads = _section_roles(section_payload)
-    active_role_ids = _active_role_ids(section_payload)
-    if active_role_ids is None:
-        return section_role_payloads
-    role_payload_by_id: dict[str, Mapping[str, object]] = {}
-    for role_payload in section_role_payloads:
-        role_id = _hashable_text(role_payload.get("id"))
-        if role_id is not None and role_id not in role_payload_by_id:
-            role_payload_by_id[role_id] = role_payload
-    return [
-        role_payload_by_id.get(role_id, {"id": role_id, "name": role_id})
-        for role_id in active_role_ids
-    ]
-
-
-def _role_display_name(role_payload: Mapping[str, object]) -> str | None:
-    """Return a hashable display name, falling back to a hashable role id."""
-    display_name = _hashable_text(role_payload.get("name"))
-    if display_name is not None:
-        return display_name
-    return _hashable_text(role_payload.get("id"))
+    roles = _section_roles(section)
+    active_ids = _active_role_ids(section)
+    if active_ids is None:
+        return roles
+    by_id: dict[str, Mapping[str, object]] = {}
+    for role in roles:
+        role_id = role.get("id")
+        if isinstance(role_id, str) and role_id not in by_id:
+            by_id[role_id] = role
+    return [by_id.get(role_id, {"id": role_id, "name": role_id}) for role_id in active_ids]
+
+
+def _role_display_name(role: Mapping[str, object]) -> str | None:
+    """Return the role's display name, falling back to its id."""
+    name = role.get("name")
+    if isinstance(name, str) and name:
+        return name
+    role_id = role.get("id")
+    if isinstance(role_id, str) and role_id:
+        return role_id
+    return None
 
 
-def _active_role_names(section_payload: Mapping[str, object]) -> list[str]:
+def _active_role_names(section_record: Mapping[str, object]) -> list[str]:
     """Return de-duplicated display names for the section's active roles."""
-    active_role_names_by_name: dict[str, None] = {}
-    for role_payload in _active_roles(section_payload):
-        display_name = _role_display_name(role_payload)
-        if display_name is not None:
-            active_role_names_by_name[display_name] = None
-    return list(active_role_names_by_name)
+    active_role_names_by_value: dict[str, None] = {}
+    for role_record in _active_roles(section_record):
+        role_display_name = _role_display_name(role_record)
+        if role_display_name is not None:
+            active_role_names_by_value[role_display_name] = None
+    return list(active_role_names_by_value)
 
 
-def _section_cue(section_payload: Mapping[str, object]) -> str:
+def _section_cue(section_record: Mapping[str, object]) -> str:
     """Join the active roles' cue values into a single cue string."""
-    active_cue_values: dict[str, None] = {}
-    for role_payload in _active_roles(section_payload):
-        cue_payload = role_payload.get("cue")
-        if not isinstance(cue_payload, Mapping):
+    section_cues_by_value: dict[str, None] = {}
+    for role_record in _active_roles(section_record):
+        role_cue_record = role_record.get("cue")
+        if not isinstance(role_cue_record, Mapping):
             continue
-        cue_value = _hashable_text(cue_payload.get("value"))
-        if cue_value is not None:
-            active_cue_values[cue_value] = None
-    return "; ".join(active_cue_values)
+        cue_text = role_cue_record.get("value")
+        if isinstance(cue_text, str) and cue_text:
+            section_cues_by_value[cue_text] = None
+    return "; ".join(section_cues_by_value)
 
 
 def _confidence_level(section: Mapping[str, object]) -> str | None:
@@ -198,24 +186,27 @@ def _section_lines(sections: list[Mapping[str, object]]) -> list[str]:
 
 
 def _footer_lines(
-    song_payload: Mapping[str, object],
-    section_payloads: list[Mapping[str, object]],
+    song_record: Mapping[str, object], section_records: list[Mapping[str, object]]
 ) -> list[str]:
     """Build the footer: per-role rehearsal priorities and the export focus."""
     footer_lines: list[str] = []
-    rehearsal_priority_lines: dict[str, None] = {}
-    for section_payload in section_payloads:
-        for role_payload in _section_roles(section_payload):
-            display_name = _role_display_name(role_payload)
-            rehearsal_priority = _hashable_text(role_payload.get("rehearsalPriority"))
-            if display_name is None or rehearsal_priority is None:
+    rehearsal_priority_lines_by_value: dict[str, None] = {}
+    for section_record in section_records:
+        for role_record in _section_roles(section_record):
+            role_display_name = _role_display_name(role_record)
+            rehearsal_priority = role_record.get("rehearsalPriority")
+            if (
+                role_display_name is None
+                or not isinstance(rehearsal_priority, str)
+                or not rehearsal_priority
+            ):
                 continue
-            priority_line = f"  - {display_name}: {rehearsal_priority}"
-            rehearsal_priority_lines[priority_line] = None
-    if rehearsal_priority_lines:
+            priority_line = f"  - {role_display_name}: {rehearsal_priority}"
+            rehearsal_priority_lines_by_value[priority_line] = None
+    if rehearsal_priority_lines_by_value:
         footer_lines.append("Priorities:")
-        footer_lines.extend(rehearsal_priority_lines)
-    export_summary = song_payload.get("exportSummary")
+        footer_lines.extend(rehearsal_priority_lines_by_value)
+    export_summary = song_record.get("exportSummary")
     if isinstance(export_summary, Mapping):
         focus_headline = export_summary.get("headline")
         if isinstance(focus_headline, str) and focus_headline:
@@ -230,7 +221,7 @@ def build_chart_text(song: Mapping[str, object] | None) -> str:
     section (``[mm:ss-mm:ss] LABEL  (confidence)  roles: ...``), and a footer
     with rehearsal priorities and the export focus headline. Output is
     deterministic and never contains filesystem paths. Malformed input
-    yields ``\"\"``.
+    yields ``""``.
     """
     if not isinstance(song, Mapping):
         return ""

services/analysis-engine/tests/benchmark_chart_export.py

@@ -0,0 +1,103 @@
+"""Measure chart-export runtime and traced allocation on a realistic song fixture."""
+
+import statistics
+import time
+import tracemalloc
+
+from bandscope_analysis.exports.chart import build_chart_text, build_cue_sheet_rows
+
+
+def make_large_song_fixture(
+    section_count: int = 96, roles_per_section: int = 24
+) -> dict[str, object]:
+    """Build a realistic large-song export fixture for benchmarking."""
+    song_sections: list[dict[str, object]] = []
+    for section_index in range(section_count):
+        section_roles: list[dict[str, object]] = []
+        part_graph_nodes: list[dict[str, object]] = []
+        for role_index in range(roles_per_section):
+            role_identifier = f"role_{role_index % 5}"
+            section_roles.append(
+                {
+                    "id": role_identifier,
+                    "name": f"Role Name {role_identifier}",
+                    "cue": {"value": f"Cue {role_index % 4}"},
+                    "rehearsalPriority": f"Priority {role_index % 2}",
+                }
+            )
+            part_graph_nodes.append({"role_id": role_identifier, "is_active": True})
+
+        song_sections.append(
+            {
+                "label": f"Section {section_index}",
+                "timeRange": {
+                    "start": section_index * 10,
+                    "end": section_index * 10 + 5,
+                },
+                "roles": section_roles,
+                "partGraph": part_graph_nodes,
+                "confidence": {"level": "high"},
+            }
+        )
+
+    return {
+        "title": "Benchmark Large Song",
+        "bpm": 120,
+        "key": "C major",
+        "feel": "Straight",
+        "sections": song_sections,
+        "exportSummary": {"headline": "Benchmark"},
+    }
+
+
+def chart_export_benchmark() -> None:
+    """Print runtime and traced peak allocation for repeated chart exports."""
+    benchmark_song = make_large_song_fixture()
+
+    for _warmup_iteration in range(100):
+        build_chart_text(benchmark_song)
+        build_cue_sheet_rows(benchmark_song)
+
+    print("Running Latency Benchmark...")
+
+    # Phase 1: Pure Latency (no tracemalloc overhead)
+    benchmark_iteration_count = 1000
+    benchmark_sample_durations_seconds: list[float] = []
+    for _benchmark_iteration in range(benchmark_iteration_count):
+        benchmark_sample_started_at = time.perf_counter()
+        build_chart_text(benchmark_song)
+        build_cue_sheet_rows(benchmark_song)
+        benchmark_sample_finished_at = time.perf_counter()
+        benchmark_sample_durations_seconds.append(
+            benchmark_sample_finished_at - benchmark_sample_started_at
+        )
+
+    print("Running Allocation Benchmark...")
+    # Phase 2: Pure Allocation (no timing structures)
+    tracemalloc.start()
+
+    allocation_iteration_count = 10
+    for _allocation_iteration in range(allocation_iteration_count):
+        build_chart_text(benchmark_song)
+        build_cue_sheet_rows(benchmark_song)
+
+    _current_allocation_bytes, peak_allocation_bytes = tracemalloc.get_traced_memory()
+    tracemalloc.stop()
+
+    total_duration_seconds = sum(benchmark_sample_durations_seconds)
+    median_duration_seconds = statistics.median(benchmark_sample_durations_seconds)
+    p95_duration_seconds = statistics.quantiles(
+        benchmark_sample_durations_seconds, n=100, method="inclusive"
+    )[94]
+    print(f"Total time for {benchmark_iteration_count} iterations: {total_duration_seconds:.4f}s")
+    print(
+        "Average time per iteration: "
+        f"{(total_duration_seconds / benchmark_iteration_count) * 1000:.2f}ms"
+    )
+    print(f"Median time per sample: {median_duration_seconds * 1000:.2f}ms")
+    print(f"P95 time per sample: {p95_duration_seconds * 1000:.2f}ms")
+    print(f"Peak memory overhead: {peak_allocation_bytes / 1024 / 1024:.2f} MB")
+
+
+if __name__ == "__main__":
+    chart_export_benchmark()