Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,12 @@ COPY --from=uv-bin /uv /uvx /bin/
COPY pyproject.toml uv.lock README.md ./
COPY src/ src/

RUN uv sync --frozen --no-dev --extra mineru
RUN uv sync --frozen --no-dev

FROM ${PYTHON_BASE} AS runtime

ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
NEWSDOM_MINERU_BIN=mineru \
PATH="/app/.venv/bin:${PATH}"

WORKDIR /app
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile.test
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ COPY src/newsdom_api/__init__.py ./src/newsdom_api/__init__.py

# Install dependencies using uv
# Install to system Python so that it's accessible without activating a venv
RUN uv pip install --system -e ".[dev,mineru]"
RUN uv pip install --system -e ".[dev]"

ENV PYTHONPATH=/app/src

CMD ["pytest"]
CMD ["pytest"]
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ To enable real parsing with MinerU, install the MinerU CLI separately in the
same `.venv` that `uv sync` created:

```bash
uv pip install --python .venv/bin/python "mineru[pipeline]==3.0.9"
uv pip install --python .venv/bin/python "mineru[pipeline]==3.4.0"
```

On Windows, replace `.venv/bin/python` with `.venv\Scripts\python.exe`.
Expand All @@ -48,9 +48,8 @@ The default image exposes the REST API on port `8000` as a multi-arch service
image. It is suitable for `linux/amd64` and `linux/arm64`, including Apple
Silicon hosts running the API service inside Docker.

The default image already includes the MinerU runtime and sets
`NEWSDOM_MINERU_BIN=mineru`, so `/parse` is available without layering an extra
OCR package into the container at runtime.
The default image ships the API service only and does not bundle the MinerU runtime.
`/parse` requires a compatible MinerU runtime to be available inside the container image or exposed through `NEWSDOM_MINERU_BIN`.

For heavier parsing deployments, build the optional NVIDIA-oriented variant:

Expand Down
8 changes: 5 additions & 3 deletions docs/operations/deploy-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,12 @@ delivery paths.
3. Verify:
- `curl -fsS http://127.0.0.1:18080/health`
- `curl -fsS -F "file=@sample.pdf" http://127.0.0.1:18080/parse`
only after a compatible MinerU runtime is installed or exposed through
`NEWSDOM_MINERU_BIN`

The default image already includes the MinerU runtime and sets
`NEWSDOM_MINERU_BIN=mineru`, so container smoke should treat `/parse`
as part of the supported contract instead of a docs-only shell.
The default image ships the API service only and does not bundle the MinerU runtime,
so container smoke should treat `/parse` as an external-runtime check rather
than a default-image contract.

`/health` proves the API process is serving but does not validate a full `/parse` round-trip, MinerU execution, or OCR artifact production.

Expand Down
4 changes: 2 additions & 2 deletions manual/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ pip install -e ".[dev]"

```bash
# MinerU 파이프라인 CLI 설치
pip install "mineru[pipeline]==3.0.9"
pip install "mineru[pipeline]==3.4.0"
```

이 명령어를 통해 **`mineru[pipeline]==3.0.9`** 버전이 설치되며 딥러닝 기반 모델을 위한 준비가 완료됩니다. 설치 후 처음 API 서버를 구동하고 PDF를 파싱할 때 모델(Weight) 파일을 백그라운드에서 다운로드할 수 있으므로, 첫 실행에는 다운로드 대기 시간이 발생할 수 있습니다.
이 명령어를 통해 **`mineru[pipeline]==3.4.0`** 버전이 설치되며 딥러닝 기반 모델을 위한 준비가 완료됩니다. 설치 후 처음 API 서버를 구동하고 PDF를 파싱할 때 모델(Weight) 파일을 백그라운드에서 다운로드할 수 있으므로, 첫 실행에는 다운로드 대기 시간이 발생할 수 있습니다.

### 커스텀 MinerU 실행 경로 (고급)
만약 `mineru` CLI 바이너리가 시스템 PATH에 잡혀있지 않거나, 특정 가상환경의 실행 파일을 수동으로 지정하고 싶다면 환경변수를 설정하세요:
Expand Down
7 changes: 0 additions & 7 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,6 @@ dev = [
"pyyaml>=6.0,<7.0",
"httpx2>=2.4.0",
]
mineru = [
"mineru[pipeline]==3.4.0",
"transformers>=4.48.0",
]
docs = [
"mkdocs>=1.6,<2.0",
"mkdocs-material>=9.6,<9.7",
Expand All @@ -42,9 +38,6 @@ fuzz = [
"pyinstaller==6.21.0",
]

[project.scripts]
mineru = "mineru.cli.app:app"

[tool.setuptools]
package-dir = { "" = "src", "tools" = "tools"}

Expand Down
13 changes: 7 additions & 6 deletions tests/test_docker_delivery.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,12 @@ def test_dockerfile_uses_project_metadata_and_src_layout():
assert _contains_pinned_uv_image(text)


def test_dockerfile_runs_uvicorn_with_external_mineru_path():
def test_dockerfile_runs_uvicorn_without_bundled_mineru_runtime():
text = Path("Dockerfile").read_text(encoding="utf-8")
assert "uvicorn" in text
assert "newsdom_api.main:app" in text
assert "NEWSDOM_MINERU_BIN" in text
assert "--extra mineru" not in text
assert "NEWSDOM_MINERU_BIN" not in text
assert "--host" in text
assert "0.0.0.0" in text
assert "8000" in text
Expand Down Expand Up @@ -137,15 +138,15 @@ def test_readme_documents_docker_build_and_run():
assert "NVIDIA" in text


def test_readme_describes_default_image_as_shipping_mineru_runtime() -> None:
def test_readme_describes_default_image_as_api_only_runtime() -> None:
text = Path("README.md").read_text(encoding="utf-8")

assert "NEWSDOM_MINERU_BIN=mineru" in text
assert "includes the MinerU runtime" in text
assert "ships the API service only" in text
assert "does not bundle the MinerU runtime" in text
assert "real `/parse` execution" not in text
assert (
"requires a compatible MinerU runtime to be available inside the container image"
not in text
in text
)


Expand Down
5 changes: 3 additions & 2 deletions tests/test_engineering_canonical_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,8 +143,9 @@ def test_deploy_runbook_describes_current_runtime_and_probe_contract() -> None:
text = Path("docs/operations/deploy-runbook.md").read_text(encoding="utf-8")

for expected in (
"default image already includes the MinerU runtime",
"`NEWSDOM_MINERU_BIN=mineru`",
"default image ships the API service only",
"does not bundle the MinerU runtime",
"`NEWSDOM_MINERU_BIN`",
"`/health` proves the API process is serving but does not validate a full `/parse` round-trip",
"no in-tree Kubernetes manifests",
):
Expand Down
2 changes: 1 addition & 1 deletion tests/test_manual_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def test_installation_doc_uses_quoted_extras_and_clear_python_wording():
assert token in text
assert "python3.10 -m venv .venv" in text
assert 'pip install -e ".[dev]"' in text
assert 'pip install "mineru[pipeline]==3.0.9"' in text
assert 'pip install "mineru[pipeline]==3.4.0"' in text


def test_installation_doc_includes_manual_api_healthcheck_commands():
Expand Down
17 changes: 17 additions & 0 deletions tests/test_project_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,5 +150,22 @@ def test_project_declares_locked_fuzz_extra_without_bundling_nvidia_stack():
assert "nvidia = [" not in text


def test_project_keeps_mineru_pipeline_stack_out_of_pyproject_metadata():
text = Path("pyproject.toml").read_text(encoding="utf-8")
dependencies_section = _dependencies_section(text)

assert '"mineru[pipeline]==3.4.0"' not in dependencies_section
assert "\nmineru = [" not in text
assert 'mineru = "mineru.cli.app:app"' not in text


def test_uv_lock_does_not_track_external_mineru_pipeline_runtime_stack():
text = Path("uv.lock").read_text(encoding="utf-8")

assert '[[package]]\nname = "mineru"' not in text
assert '[[package]]\nname = "transformers"' not in text
assert '[[package]]\nname = "torch"' not in text


def test_uv_lock_pins_pypdf_at_patched_release():
assert _locked_package_version("pypdf") >= (6, 10, 0)
Loading
Loading