diff --git a/openspec/changes/add-worker-original-filename-tracking/.openspec.yaml b/openspec/changes/add-worker-original-filename-tracking/.openspec.yaml new file mode 100644 index 000000000..054b8c013 --- /dev/null +++ b/openspec/changes/add-worker-original-filename-tracking/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-05-08 diff --git a/openspec/changes/add-worker-original-filename-tracking/proposal.md b/openspec/changes/add-worker-original-filename-tracking/proposal.md new file mode 100644 index 000000000..1986c0d9c --- /dev/null +++ b/openspec/changes/add-worker-original-filename-tracking/proposal.md @@ -0,0 +1,41 @@ +## Why + +`_worker` 在收到 source artifact 時,會用 `SAFE_FILENAME_RE` 把非 ASCII 字元(中文、空格、括號、連字符)全部換成 `_`,再 `.strip("._")` 把開頭/結尾的底線清掉。對於常見的中文檔名 `許良宇圖書館建築_2026.ifc`,落地檔名變成 `54d77fe1_2026.ifc`(只剩 sha256 prefix + `2026.ifc`),原始檔名完全消失。 + +更嚴重的是 `_worker` 寫入 source artifact `metadata.json` 時、寫入 `_index/source_artifacts.json` 時、以及 callback 給 `_bim-control` 的 conversion-result payload 中,**都不包含原檔名欄位**。也就是說:sanitize 之後再也找不回「這個 artifact 對應的原始 IFC 檔本來叫什麼」。 + +`_bim-control` 在 `_update_artifacts_from_conversion` 中接收 callback 時,artifact 的 `name` 欄位寫死成「原始 IFC」/「已轉換 USDC」,不會反映實際檔名。 + +實機驗證:13 個 89 MB 的中文檔名 IFC 副本(`許良宇圖書館建築_2026 - 複製 (N).ifc`)跑進 `_worker` 後,所有 disk 落地檔名都會變成形如 `_<某個 ASCII 殘片>.ifc`,從 disk 與 `_bim-control` 都無法分辨哪個 artifact 對應原檔的第幾份副本。 + +這次 change 在 metadata 層補上 `original_filename` 欄位,讓檔名 traceability 不會因為 sanitize 而斷裂;disk 檔名仍然 sanitize(保留 path 安全),但語意層保留原檔名。 + +## What Changes + +- 在 `_worker` source artifact `metadata.json` 加入 `original_filename` 欄位,保留 `ArtifactIntakeRequest.filename` 的 raw 值(未 sanitize)。 +- 在 `_worker/data/objects/_index/source_artifacts.json` 每筆 entry 中也加入 `original_filename`,方便不開啟 metadata.json 也能查。 +- `POST /api/artifacts` response 多回 `original_filename`。 +- `POST /api/dev/ifc-sources/{source_id}/conversions` response 多回 `original_filename`(從 dev source 直接帶上原 `relative_path` 對應的檔名)。 +- `_worker` 完成轉檔後產出的 `result` payload(給 `GET /api/conversions/{id}/result`,以及 callback 給 `_bim-control`)多回 `original_filename`。 +- `_bim-control._update_artifacts_from_conversion` 收到 `original_filename` 時,將 source IFC artifact 的 `name` 欄位設為原檔名(沒帶就保留原本寫死的 fallback,向前相容)。 +- 既有 metadata.json / source_artifacts.json 沒有 `original_filename` 欄位的舊資料仍可正常讀取(讀取端用 `.get(...)` 容錯)。 +- 非目標:不改 disk 檔名 sanitize 邏輯、不改 `source_id` 計算公式、不改 path traversal 防護、不改 artifact ID schema、不做 disk 既有檔案的改名/移轉。 + +## Capabilities + +### Modified Capabilities + +- `worker-artifact-pipeline`: 新增 Requirement「Worker preserves original filename in source metadata」,要求 source artifact metadata、`_index` 條目與 callback payload 都帶 `original_filename`。 +- `worker-dev-ifc-source-selection`: 修改 Requirement「Start Conversion From Selected Source」,把 selected-source conversion response 與後續 result/callback 都納入「必含 `original_filename`」的 scenario。 + +### New Capabilities + +- 無。 + +## Impact + +- `_worker`: 修改 [_worker/app/store.py](_worker/app/store.py) 的 `create_source_artifact` 與 `complete_conversion_job`:metadata dict 新增 `original_filename`、result payload 帶 `original_filename`、`_index/source_artifacts.json` 條目加欄位。修改 [_worker/app/main.py](_worker/app/main.py) 的兩個 conversion endpoint response 形狀。callback `_post_bim_control_result` 不需改(payload 已是 result dict)。新增 / 擴充 `_worker/tests/test_worker_api.py` 對應測試。 +- `_bim-control`: 修改 [_bim-control/app/main.py](_bim-control/app/main.py) `_update_artifacts_from_conversion`:當 result 包含 `original_filename` 時,設定 source IFC artifact 的 `name` 欄位為原檔名。新增 / 擴充 `_bim-control/tests/test_conversion_results_api.py` 對應測試。 +- `web-viewer-sample` / `bim-review-coordinator` / `bim-streaming-server`: 不需修改。`original_filename` 是 metadata 層的純加欄位,下游消費端可選擇是否使用。 +- 文件: `docs/contracts/worker-api.md`(如存在)需補上 `original_filename` 欄位說明。 +- 向前相容: 既有測試 fixture、archived spec、已落地的 `_worker/data/objects/...` 檔案結構不需改動。新欄位是 additive。 diff --git a/openspec/changes/add-worker-original-filename-tracking/specs/worker-artifact-pipeline/spec.md b/openspec/changes/add-worker-original-filename-tracking/specs/worker-artifact-pipeline/spec.md new file mode 100644 index 000000000..016f5dd2f --- /dev/null +++ b/openspec/changes/add-worker-original-filename-tracking/specs/worker-artifact-pipeline/spec.md @@ -0,0 +1,35 @@ +## ADDED Requirements + +### Requirement: Worker preserves original filename in source metadata + +`_worker` SHALL preserve the unsanitized client-provided filename as `original_filename` in source artifact metadata, the source artifact index, the source artifact API response, and the conversion result payload published to `_bim-control`. Disk object names MAY remain sanitized for path safety, but the metadata layer MUST keep the original filename so traceability survives across non-ASCII or special characters. + +#### Scenario: Source artifact metadata records the original filename + +- **WHEN** a client calls `POST /api/artifacts` with a `filename` containing non-ASCII characters or characters outside `[A-Za-z0-9_.-]` +- **THEN** the source artifact `metadata.json` written under the versioned object layout MUST include `original_filename` equal to the request `filename` byte-for-byte (no sanitization), while the on-disk object key MAY use a sanitized filename for path safety + +#### Scenario: Source artifact index records the original filename + +- **WHEN** `_worker` upserts an entry into `data/objects/_index/source_artifacts.json` +- **THEN** the entry MUST include `original_filename` so that consumers can recover the original filename without opening the per-artifact `metadata.json` + +#### Scenario: Source artifact API response includes the original filename + +- **WHEN** `POST /api/artifacts` succeeds +- **THEN** the response body MUST include `original_filename` equal to the request `filename` + +#### Scenario: Conversion result includes the original filename + +- **WHEN** a conversion job started from any source artifact succeeds +- **THEN** `GET /api/conversions/{conversion_job_id}/result` MUST include `original_filename` carried from the source artifact metadata + +#### Scenario: Callback to BIM control includes the original filename + +- **WHEN** `_worker` publishes a successful conversion result to `_bim-control` via the model-version conversion-result callback +- **THEN** the callback payload MUST include `original_filename`, and `_bim-control` MUST set the source IFC artifact `name` field to that value when present, falling back to the existing default name when the field is absent + +#### Scenario: Backward-compatible reads of legacy metadata + +- **WHEN** `_worker` reads an existing `metadata.json` or `_index/source_artifacts.json` entry that was written before this requirement existed +- **THEN** the read path MUST treat `original_filename` as optional and MUST NOT fail or refuse to serve the artifact when the field is missing diff --git a/openspec/changes/add-worker-original-filename-tracking/specs/worker-dev-ifc-source-selection/spec.md b/openspec/changes/add-worker-original-filename-tracking/specs/worker-dev-ifc-source-selection/spec.md new file mode 100644 index 000000000..69b205f88 --- /dev/null +++ b/openspec/changes/add-worker-original-filename-tracking/specs/worker-dev-ifc-source-selection/spec.md @@ -0,0 +1,25 @@ +## MODIFIED Requirements + +### Requirement: Start Conversion From Selected Source + +`_worker` SHALL provide `POST /api/dev/ifc-sources/{source_id}/conversions` to create a source artifact from a selected dev IFC and start a conversion job through the existing worker artifact pipeline. The response, the resulting source artifact metadata, and the conversion result MUST preserve the original dev IFC filename (including non-ASCII characters) as `original_filename`, even when the on-disk object name is sanitized for path safety. + +#### Scenario: Selected source starts worker job + +- **WHEN** a valid `source_id` is posted with tenant, project, model version, source system, and uploaded-by metadata +- **THEN** `_worker` SHALL store the selected IFC in the versioned worker object layout, create a source artifact, create a conversion job, and return `source_artifact_id`, `artifact_group_id`, `conversion_job_id`, `status`, `original_filename`, and result lookup URLs + +#### Scenario: Conversion uses existing worker result contract + +- **WHEN** the selected-source conversion job succeeds +- **THEN** `GET /api/conversions/{conversion_job_id}/result` SHALL return the same derived artifact, object URL, mapping URL, readiness, and lineage shape as conversions started from `POST /api/conversions`, plus `original_filename` carried from the source artifact metadata + +#### Scenario: Unknown source is rejected + +- **WHEN** `POST /api/dev/ifc-sources/{source_id}/conversions` is called with an unknown, stale, non-IFC, or out-of-root `source_id` +- **THEN** `_worker` MUST return a 4xx response and MUST NOT create a source artifact or conversion job + +#### Scenario: Non-ASCII source filename is preserved end-to-end + +- **WHEN** a dev IFC under `WORKER_DEV_STORAGE_ROOT` has a filename containing non-ASCII characters (for example, traditional Chinese characters, spaces, or parentheses) +- **THEN** the selected-source conversion response, the worker source `metadata.json`, the conversion result, and the `_bim-control` callback payload MUST all include `original_filename` equal to the original `relative_path` filename, even though the on-disk object key uses a sanitized form for path safety diff --git a/openspec/changes/add-worker-original-filename-tracking/tasks.md b/openspec/changes/add-worker-original-filename-tracking/tasks.md new file mode 100644 index 000000000..4fe46da4e --- /dev/null +++ b/openspec/changes/add-worker-original-filename-tracking/tasks.md @@ -0,0 +1,54 @@ +## 1. Preparation And Impact Review + +- [ ] 1.1 Re-read `_worker/app/store.py`、`_worker/app/main.py`、`_worker/app/models.py`、`_bim-control/app/main.py` 中跟 source artifact metadata、conversion result、callback 有關的程式碼,確認加欄位的最小改動面。 +- [ ] 1.2 Run GitNexus impact analysis for `create_source_artifact`、`complete_conversion_job`、`_post_bim_control_result`、`_update_artifacts_from_conversion`;確認 HIGH/CRITICAL 風險前回報。 +- [ ] 1.3 列出既有 `_worker/data/objects/_index/source_artifacts.json`、metadata.json fixture 是否含舊資料;確認向前相容讀取策略(`.get("original_filename", None)`)。 +- [ ] 1.4 確認 `ArtifactIntakeRequest.filename` 欄位本身已是 raw(未 sanitize),可以直接保存到 metadata 不需重做 decode。 + +## 2. Spec Deltas + +- [ ] 2.1 在 `openspec/changes/add-worker-original-filename-tracking/specs/worker-artifact-pipeline/spec.md` 新增 Requirement「Worker preserves original filename in source metadata」與其 scenarios。 +- [ ] 2.2 在 `openspec/changes/add-worker-original-filename-tracking/specs/worker-dev-ifc-source-selection/spec.md` 修改 Requirement「Start Conversion From Selected Source」,確保 response shape 必含 `original_filename`。 +- [ ] 2.3 Run `openspec validate add-worker-original-filename-tracking`。 + +## 3. `_worker` Source Artifact Metadata + +- [ ] 3.1 修改 `_worker/app/store.py` `create_source_artifact()` 的 metadata dict,加入 `original_filename: str = request.filename`(保留 raw,不經 `safe_filename`)。 +- [ ] 3.2 修改 `_worker/app/store.py` `_upsert_source_index()` 寫入 `_index/source_artifacts.json` 的 entry,加入 `original_filename`。 +- [ ] 3.3 修改 `_worker/app/store.py` `create_source_artifact()` 的 return dict,加入 `original_filename`。 +- [ ] 3.4 確認 `_worker/app/store.py` `complete_conversion_job()` 從 `source["metadata"]` 取得 `original_filename` 並寫進 result payload(給 callback 與 GET result 用)。 + +## 4. `_worker` API Response Shape + +- [ ] 4.1 修改 `_worker/app/main.py` `POST /api/artifacts` 的 response,確認 `original_filename` 在 response body(透過 `create_source_artifact` 的 return dict 自動帶出)。 +- [ ] 4.2 修改 `_worker/app/main.py` `POST /api/dev/ifc-sources/{source_id}/conversions` 的 response,加入 `original_filename`(從 `source_item["filename"]` 取,這是原 `relative_path` 對應的純檔名)。 +- [ ] 4.3 修改 `_worker/app/main.py` `GET /api/conversions/{id}/result` 透過 `complete_conversion_job` 自然帶出 `original_filename`,無需個別處理。 + +## 5. `_bim-control` Artifact Name Mapping + +- [ ] 5.1 修改 `_bim-control/app/main.py` `_update_artifacts_from_conversion()`:source IFC artifact upsert 時,`name` 欄位優先用 `result.get("original_filename")`,fallback 保留現有「原始 IFC」字串。 +- [ ] 5.2 USDC artifact 的 `name` 欄位保持現有「已轉換 USDC」(USDC 是 worker 自己生的衍生檔,不需要展示原檔名)。 +- [ ] 5.3 確認 `_bim-control` 沒有任何 schema validation 會擋掉 result payload 多帶的欄位(目前是 `dict[str, Any]` 接收,OK)。 + +## 6. Tests + +- [ ] 6.1 `_worker/tests/test_worker_api.py` 新增測試:上傳含中文檔名的 IFC,確認 metadata.json、`_index/source_artifacts.json`、API response 都含 `original_filename` 等於原檔名;disk filename 仍 sanitize。 +- [ ] 6.2 `_worker/tests/test_worker_api.py` 新增測試:selected-source conversion (dev IFC) 完成後,`GET /api/conversions/{id}/result` 包含 `original_filename`。 +- [ ] 6.3 `_worker/tests/test_worker_store.py` 補測 `create_source_artifact` 直接呼叫時 metadata 含 `original_filename`。 +- [ ] 6.4 `_bim-control/tests/test_conversion_results_api.py` 新增測試:POST conversion-result 含 `original_filename` 時,artifacts.json 的 source IFC entry `name` 欄位 = 原檔名;不含 `original_filename` 時 fallback 保持原行為。 +- [ ] 6.5 確認既有測試(無 `original_filename` 的舊 fixture)仍綠燈,驗證向前相容。 + +## 7. Documentation + +- [ ] 7.1 若 `docs/contracts/worker-api.md` 存在,補上 `original_filename` 欄位說明(位置:source artifact response、conversion result、callback payload)。 +- [ ] 7.2 `_worker/README.md` API 段落如有列 response 範例,補欄位(可選,不阻擋)。 +- [ ] 7.3 `_bim-control/README.md` 不需改(callback 接收端形狀不變,只是多接受一個欄位)。 + +## 8. Validation And Review + +- [ ] 8.1 `cd _worker && python -m pytest tests -q` 全綠。 +- [ ] 8.2 `cd _bim-control && python -m pytest tests -q` 全綠。 +- [ ] 8.3 Run `openspec validate add-worker-original-filename-tracking` 通過。 +- [ ] 8.4 GitNexus detect_changes 確認 affected scope 只在 `_worker` 與 `_bim-control` 範圍內。 +- [ ] 8.5 PR 描述列出向前相容性說明(既有資料 / 既有 callback 沒帶 `original_filename` 仍能運作)。 +- [ ] 8.6 Smoke:手動跑一次中文檔名 IFC 從 storage 進 worker、callback 到 `_bim-control` 的完整流程,確認 `_bim-control` artifact `name` 顯示原檔名。