Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
source =
audio_library
chapters
credential_registry
diarize
job_store
mcp_driver
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
python -m pip install --disable-pip-version-check --no-index --no-deps --no-build-isolation -e .

- name: Compile check
run: python -m py_compile media_shrinker.py config_file.py presets.py saas_web.py mcp_driver.py job_store.py
run: python -m py_compile media_shrinker.py config_file.py presets.py saas_web.py mcp_driver.py job_store.py credential_registry.py

- name: Run tests
run: python -m unittest discover -s tests -v
Expand Down
9 changes: 5 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,11 @@ repo.
job store, and open PRs adding API-key auth and usage metering), so it *will*
read runtime secrets/config (API keys, DB creds, endpoints). When you add them,
source them from the KV, not `os.getenv`.
- **Known deviation to migrate:** the in-flight API-key auth work reads keys from
a `CODEC_CARVER_API_KEYS` environment variable — that is exactly the anti-pattern
above. Move it to read from the credential registry (env may still be the
bootstrap transport that *populates* the KV, never the runtime source).
- **API keys:** `saas_web.py` request-time auth reads
`credential_registry.CredentialRegistry` only. `CODEC_CARVER_API_KEYS` is
bootstrap transport into `bootstrap_from_mapping` at process start (and in
tests). Do not add request-time `os.getenv("CODEC_CARVER_API_KEYS")` back.
Decision record: `docs/doctoring/api-credential-registry.md`.

### Code exploration
- There is no `.codegraph/` index in this repo today, so use normal search
Expand Down
70 changes: 70 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Architecture

Codec Carver is a standalone conversion service that also works as a git
submodule. The Python CLI, FastAPI upload UI, MCP tool, and Rust
`codec-carver-core` binary can run alone. When naruon or another
ContextualWisdomLab service imports the module, `convert_file` is the
stable in-process port.

```text
+------------------+
browser / API -->| saas_web.py | GET / GET /health
| require_api_key |----> credential_registry
| /shrink /jobs | (api_credentials)
+--------+---------+
|
v
media_shrinker.py ----> ffmpeg / ffprobe
|
+--------------+--------------+
| |
job_store.py rust-core/
(jobs table; codec-carver-core
rename to (library CLI)
conversion_jobs is
follow-up debt)
```

## Credential port

`credential_registry.CredentialRegistry` is the provider-neutral
credential port. Other CWL services can depend on this module without
importing FastAPI. Bootstrap transport may be an environment snapshot;
request-time verification may not. See
`docs/doctoring/api-credential-registry.md`.

## Core ERD (auth + jobs)

```text
api_credentials
credential_id PK
key_digest UK
lifecycle_status
created_at
updated_at
rotated_at
revoked_at
expires_at

jobs -- existing; one-word name is known debt
id PK
status
created_at
updated_at
output_path
output_name
error
temp_dir
```

Both tables are in 3NF: non-key attributes depend only on the primary
key. Usage metering still keys rows by plaintext `api_key` (`usage`);
rebinding that table to `credential_id` is a later migration.

## Next actions

1. Land this registry. Close HMAC-only sentinels (#376, #421) as
superseded once compare_digest no longer sees raw Unicode headers.
2. Keep Cloud Agent environment work on #427; do not mix it here.
3. Rename `jobs` → `conversion_jobs` in a dedicated migration.
4. Add production fail-closed bind policy without request-time env reads.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@
- 다중 파일 업로드 선택 시 즉각적인 파일 개수 피드백 및 제한 초과 경고 메시지 추가
- 일괄 업로드 폼에 대상 바이트 프리셋 버튼과 총 파일 크기 미리보기를 추가하여 사용성을 개선했습니다.
- 클라이언트 측 폼 검증 시 하드코딩된 '5 GiB' 텍스트를 동적으로 변환되도록 수정하고 일괄 업로드 폼에 최대 크기(MAX_UPLOAD_BYTES) 검증 피드백을 추가했습니다.
- `GET /health`는 인증 없이 `{"status":"ok","service":"codec-carver"}`를 반환합니다. 로드 밸런서와 Cloud Agent `start`는 이 URL을 프로브하면 됩니다.

### Changed
- 순수 영숫자 토큰은 정규식 호출을 건너뛰되 다국어·문장부호 토큰화 결과는 기존 의미와 동일하게 유지합니다. 근거, 한계, APA 7 참고문헌은 [`docs/doctoring/token-fast-path-equivalence.md`](docs/doctoring/token-fast-path-equivalence.md)에 기록했습니다.
- SaaS API 키 인증은 요청 시점에 환경 변수를 읽지 않습니다. 시작 시 `CODEC_CARVER_API_KEYS`를 부트스트랩 수송으로만 사용해 `api_credentials` 레지스트리에 SHA-256 다이제스트를 넣고, 요청은 UTF-8 다이제스트를 `hmac.compare_digest`로 전량 비교합니다. 운영자는 키를 회전·폐기한 뒤 해당 시크릿을 헤더에서 제거하면 됩니다. 근거는 [`docs/doctoring/api-credential-registry.md`](docs/doctoring/api-credential-registry.md)입니다.

### Fixed
- 단일·일괄 대상 크기 입력을 비웠을 때 이전 custom validity와 `aria-invalid` 상태를 즉시 초기화해 현재 필수 입력 상태를 정확히 전달합니다.
Expand Down
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ python3 -m unittest tests.test_media_shrinker -v
python3 -m unittest tests.test_job_store.TestCreateAndGet.test_create_get_roundtrip

# Compile check (CI runs this on all four modules)
python -m py_compile media_shrinker.py saas_web.py mcp_driver.py job_store.py
python -m py_compile media_shrinker.py saas_web.py mcp_driver.py job_store.py credential_registry.py

# CLI (omit --execute for a dry run that only lists candidates)
codec-carver /path/to/recordings --execute --output-dir under_2gb
Expand All @@ -49,7 +49,8 @@ Four flat top-level modules (declared as `py-modules` in `pyproject.toml`; there
- **`media_shrinker.py`** — the core engine and CLI, deliberately stdlib-only (external work happens in `ffmpeg`/`ffprobe` subprocesses). The console script `codec-carver` maps to `media_shrinker:main`. Pipeline for a batch run: `find_candidates` scans the root (pruned `os.walk`, excludes the output dir and `--exclude-dir-prefix` dirs) → per file, `convert_file` probes with ffprobe (`probe_media` / `_parse_probe_payload`), detects silence and builds a split plan for long sources (`detect_silence_intervals`, `parse_silencedetect_intervals`, `build_segments`) → each segment gets a `ConversionPlan` (`build_audio_plan` prefers FLAC; `build_opus_plan` is the fallback when a FLAC output exceeds the target size) → `_execute_plan` runs ffmpeg and `preserve_file_attributes` restores permissions/timestamps/xattrs best-effort → `write_report` emits a JSON report. `convert_file(source, root=..., output_dir=..., target_bytes=...)` is the programmatic API that the web and MCP layers call.
- **`saas_web.py`** — single-file FastAPI upload UI (the `[web]` extra; what the Docker image serves). Streams one upload into a temp workspace, calls `media_shrinker.convert_file`, and returns the first generated output as a download. Middleware enforces a 5 GiB upload cap and security headers. Processing is synchronous per request.
- **`mcp_driver.py`** — FastMCP server (the `[mcp]` extra) exposing a single `shrink_media` tool that wraps `convert_file`.
- **`job_store.py`** — stdlib-only SQLite (WAL) durable job store intended for async/worker job tracking. It is tested but not yet wired into `saas_web.py`. Callers pass `now` explicitly; the store never calls `datetime.now()` itself.
- **`job_store.py`** — stdlib-only SQLite (WAL) durable job store intended for async/worker job tracking. It is tested but not yet wired into `saas_web.py`. Callers pass ``now`` explicitly; the store never calls ``datetime.now()`` itself.
- **`credential_registry.py`** — stdlib-only SQLite (WAL) hashed API-key registry (`api_credentials`). Request-time auth in `saas_web.py` reads only this store. `CODEC_CARVER_API_KEYS` is bootstrap transport into `bootstrap_from_mapping`, never a request-time `os.getenv`. Callers pass ``now`` for expiry and rotation.

Supporting directories: `fuzz/` holds Atheris harnesses plus seed corpora for the three untrusted-input parsing surfaces (`parse_silencedetect_intervals`, `_parse_probe_payload`, `build_segments`); the same invariants run as Hypothesis property tests in `tests/test_fuzz_properties.py` so they execute in the normal suite. `docs/papers/` holds the fuzzing survey the harness design references.

Expand Down
Loading
Loading