diff --git a/.jules/bolt.md b/.jules/bolt.md index aa33c37f..2389b4f5 100644 --- a/.jules/bolt.md +++ b/.jules/bolt.md @@ -40,6 +40,3 @@ ## 2026-06-30 - Regex over Generator `any` string loops **Learning:** Using `any(...)` with a generator comprehension in string evaluation paths allocates a new generator and adds Python-level loop overhead for every character. **Action:** Replace `any()` generators with a pre-compiled regex (`re.compile().search()`) to evaluate string patterns in C, achieving a ~7x speedup for text-heavy operations. -## 2026-07-01 - Precompile MinerU Argument Allowlist -**Learning:** Recompiling the same allowlist pattern through repeated `re.match(...)` calls adds avoidable overhead in MinerU command argument validation. -**Action:** Keep the strict allowlist semantics, but compile the regex once at module load and validate each argument with `Pattern.fullmatch()`. diff --git a/.jules/sentinel.md b/.jules/sentinel.md index 529fe8d7..18667ffe 100644 --- a/.jules/sentinel.md +++ b/.jules/sentinel.md @@ -36,8 +36,3 @@ **Vulnerability:** Unhandled FastAPI exceptions can produce sanitized 500 responses without the same defense-in-depth headers applied by normal middleware responses. **Learning:** Error response paths need explicit coverage because exception handlers can bypass or duplicate header logic differently from successful request paths. **Prevention:** Route both middleware responses and global 500 exception responses through a shared security-header helper. - -## 2025-02-14 - Fix Command Injection Risk in CLI arguments -**Vulnerability:** Command arguments filtering relied on a denylist of shell control characters (`_SHELL_CONTROL_CHARS = frozenset("&;|`$<>")`), which can be bypassed. Furthermore, certain valid file path characters could still trigger issues or be blocked incorrectly. -**Learning:** Checking for bad characters (denylist) is brittle and frequently bypassable. CLI tools and scripts should strictly validate user inputs (such as file paths) using regex allowlists rather than mutating paths or using weak character blocklists. -**Prevention:** Apply a strict regex allowlist (e.g., `^[\w\s/\\.:~()-]+$`) to validate user-supplied file paths and arguments. Ensure the allowlist permits valid cross-OS path characters like colons, tildes, and parentheses to avoid functional regressions while maintaining security. diff --git a/CHANGELOG.md b/CHANGELOG.md index 8eae9aa2..ddd92dce 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,14 +15,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Performance - `newsdom_api.dom_builder._html_safe_text` 함수에 early return과 타입 체크를 도입하여 불필요한 `str()` 캐스팅을 제거함으로써 처리 속도를 개선했습니다. -- `mineru_runner.py`의 MinerU 명령어 인자 allowlist 검증 정규식을 사전 컴파일하여 반복 호출 비용을 줄였습니다. ### Added -- Manual now includes a dedicated security reporting page so vulnerability disclosure, supported branches, and safe evidence handling are visible from the documentation navigation. -- Package metadata now publishes Homepage, Documentation, Repository, Issues, and Security URLs so package indexes and developer tools show the same live project surfaces as the manual and OpenAPI docs. -- Public documentation metadata now points to the current `ContextualWisdomLab/newsdom-api` repository, keeping the MkDocs header, OpenAPI contact link, Scorecard badges, and security reporting links aligned with the live project. -- API reference manual now embeds the checked-in Swagger UI and ReDoc screenshots so the interactive documentation section renders with visible product context instead of text-only placeholders. -- OpenAPI/ReDoc response samples now include concrete `HealthResponse`, `ParseResponse`, and error examples so visual API documentation no longer renders placeholder property names. +- [CLI] 파싱된 DOM JSON 파일이 `ParseResponse` 스키마 형식을 만족하는지 검사하는 `tools/validate_dom.py` 도구를 추가했습니다. +- [CLI] 파싱된 DOM JSON 파일에서 headline과 body_blocks 등 순수 텍스트만 추출하는 `tools/extract_text.py` 도구를 추가했습니다. - [CLI] 파싱된 NewsDOM JSON을 Markdown 포맷으로 변환하는 `tools/export_markdown.py` 도구를 추가했습니다. - [CLI] `tools/batch_parse_pdf.py`에 하위 디렉터리의 PDF를 일괄 처리하고 상대 경로로 JSON을 저장하는 `--recursive` 옵션을 추가했습니다. - OpenAPI 문서에 contact 및 MIT license metadata를 추가하여 API 소비자가 maintainer와 라이선스 정보를 더 쉽게 확인할 수 있도록 개선 @@ -86,7 +82,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Synthetic newspaper fixture generation and structural equivalence checks - Protected-branch CI, security gates, release provenance workflow, and Git Flow documentation -[Unreleased]: https://github.com/ContextualWisdomLab/newsdom-api/compare/v0.2.0...HEAD -[0.2.0]: https://github.com/ContextualWisdomLab/newsdom-api/compare/v0.1.1...v0.2.0 -[0.1.1]: https://github.com/ContextualWisdomLab/newsdom-api/compare/v0.1.0...v0.1.1 -[0.1.0]: https://github.com/ContextualWisdomLab/newsdom-api/releases/tag/v0.1.0 +[Unreleased]: https://github.com/Seongho-Bae/newsdom-api/compare/v0.2.0...HEAD +[0.2.0]: https://github.com/Seongho-Bae/newsdom-api/compare/v0.1.1...v0.2.0 +[0.1.1]: https://github.com/Seongho-Bae/newsdom-api/compare/v0.1.0...v0.1.1 +[0.1.0]: https://github.com/Seongho-Bae/newsdom-api/releases/tag/v0.1.0 diff --git a/README.md b/README.md index 70e51a1f..4aa8d56e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # NewsDOM API -[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/ContextualWisdomLab/newsdom-api/badge)](https://securityscorecards.dev/viewer/?uri=github.com/ContextualWisdomLab/newsdom-api) +[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/Seongho-Bae/newsdom-api/badge)](https://securityscorecards.dev/viewer/?uri=github.com/Seongho-Bae/newsdom-api) NewsDOM API parses scanned Japanese newspaper PDFs into DOM-like article trees. diff --git a/SECURITY.md b/SECURITY.md index 17701726..e80f8ecf 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -5,7 +5,7 @@ Please report a vulnerability privately by opening a GitHub Security Advisory draft for this repository: -- [GitHub Security Advisory draft](https://github.com/ContextualWisdomLab/newsdom-api/security/advisories/new) +- [GitHub Security Advisory draft](https://github.com/Seongho-Bae/newsdom-api/security/advisories/new) - [Maintainer profile](https://github.com/seonghobae) Do not open a public issue for an unpatched vulnerability. diff --git a/manual/api-reference.md b/manual/api-reference.md index 9d485d48..7ceec967 100644 --- a/manual/api-reference.md +++ b/manual/api-reference.md @@ -21,14 +21,6 @@ FastAPI는 OpenAPI 기반의 대화형 API 문서를 자동으로 생성합니 해당 페이지에서 **`/parse`** 버튼을 클릭한 후, `Try it out` 기능을 통해 파일을 직접 첨부하고 결과물을 테스트해 볼 수 있습니다. -### 문서 UI 미리보기 - -아래 캡처는 `/docs`와 `/redoc`의 기본 배치를 빠르게 확인하기 위한 정적 레이아웃 예시입니다. 실제 응답 예시는 실행 중인 서버의 `/openapi.json` 스키마에서 생성됩니다. - -![Swagger UI에서 health와 parse 엔드포인트가 보이는 화면](assets/swagger-ui.png) - -![ReDoc에서 NewsDOM API 엔드포인트 문서가 보이는 화면](assets/redoc.png) - --- ## 3. API 엔드포인트 세부 설명 diff --git a/manual/index.md b/manual/index.md index 97c627ad..b808afa1 100644 --- a/manual/index.md +++ b/manual/index.md @@ -1,6 +1,6 @@ # NewsDOM API 개요 및 아키텍처 -[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/ContextualWisdomLab/newsdom-api/badge)](https://securityscorecards.dev/viewer/?uri=github.com/ContextualWisdomLab/newsdom-api) +[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/Seongho-Bae/newsdom-api/badge)](https://securityscorecards.dev/viewer/?uri=github.com/Seongho-Bae/newsdom-api) **NewsDOM API**는 스캔된 일본어 신문 PDF 문서를 분석하여, 웹 브라우저의 DOM(Document Object Model)과 유사한 기사(Article) 단위의 트리 구조로 파싱해주는 API 서비스입니다. @@ -37,4 +37,4 @@ MinerU가 뱉어낸 선형적인(Linear) 블록 리스트(`content_list.json`) --- -👉 **[설치 가이드](installation.md)**를 읽고 직접 환경을 구성해 보세요. +👉 **[설치 가이드](installation.md)**를 읽고 직접 환경을 구성해 보세요. \ No newline at end of file diff --git a/manual/security.md b/manual/security.md deleted file mode 100644 index 91bf017b..00000000 --- a/manual/security.md +++ /dev/null @@ -1,32 +0,0 @@ -# 보안 및 취약점 제보 - -NewsDOM API에서 아직 공개되지 않은 취약점을 발견했다면 공개 이슈를 열지 말고 GitHub Security Advisory 초안으로 비공개 제보해 주세요. - -- 비공개 제보: [GitHub Security Advisory draft](https://github.com/ContextualWisdomLab/newsdom-api/security/advisories/new) -- Maintainer: [seonghobae](https://github.com/seonghobae) - -## 제보에 포함할 내용 - -- 영향을 받는 브랜치 또는 커밋 -- 재현 단계와 기대 동작 -- 영향 범위와 악용 가능성에 대한 설명 -- 안전하게 재현할 수 있는 proof-of-concept 입력 또는 정제된 로그 - -## 안전한 증거 처리 - -가능하면 합성 fixture를 사용해 재현 자료를 구성하세요. 실제 서비스 secret, production credential, 비공개 참조 입력, 저작권이 있는 제3자 원문 문서는 제보에 포함하지 않습니다. - -로그나 proof-of-concept 입력이 필요한 경우 민감한 식별자와 토큰을 제거하되, maintainer가 동일한 문제를 재현할 수 있을 만큼의 구조와 경로는 남겨 주세요. - -## 지원 브랜치 - -- `develop`: 적극적으로 유지보수되는 통합 브랜치 -- `main`: 안정 릴리스 브랜치 - -보안 수정은 Git Flow에 맞는 브랜치에서 진행하고 필요하면 `docs/workflow/git-flow.md` 절차에 따라 역병합합니다. - -## 응답 기대치 - -- 접수 확인 목표: 7일 이내 -- 수정 가능성이 있는 경우 triage 또는 상태 업데이트 목표: 30일 이내 -- 수정 또는 완화책이 준비된 뒤 조율된 공개를 선호합니다. diff --git a/manual/workflow/ocr-accuracy-evidence.md b/manual/workflow/ocr-accuracy-evidence.md index 3cfdb8c4..596f2e02 100644 --- a/manual/workflow/ocr-accuracy-evidence.md +++ b/manual/workflow/ocr-accuracy-evidence.md @@ -34,6 +34,6 @@ This will overwrite the existing placeholder fixture with a new baseline derived As of 2026-04-23, the execution of this harness is **BLOCKED**. The underlying `mineru` OCR process hangs indefinitely when processing the private PDF files. -This issue is tracked in **[#70](https://github.com/ContextualWisdomLab/newsdom-api/issues/70)**. +This issue is tracked in **[#70](https://github.com/Seongho-Bae/newsdom-api/issues/70)**. Until this blocker is resolved, generating a new `private_page_baseline.json` from the local corpus is not possible. The harness script and tests have been merged so that the tooling is in place once the blocker is fixed. diff --git a/mkdocs.yml b/mkdocs.yml index ae6d8cae..e2489723 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,8 +1,8 @@ site_name: NewsDOM API 매뉴얼 -site_url: https://contextwisdomlab.github.io/newsdom-api/ +site_url: https://seongho-bae.github.io/newsdom-api/ site_description: 일본어 신문 스캔 PDF를 DOM 형태의 트리로 파싱하는 API -repo_url: https://github.com/ContextualWisdomLab/newsdom-api -repo_name: ContextualWisdomLab/newsdom-api +repo_url: https://github.com/Seongho-Bae/newsdom-api +repo_name: Seongho-Bae/newsdom-api docs_dir: manual @@ -33,7 +33,6 @@ nav: - 시작하기: index.md - 설치 가이드: installation.md - 사용 방법 및 API: api-reference.md - - 보안 제보: security.md - 개발 및 기여: - development.md - 워크플로우: diff --git a/pyproject.toml b/pyproject.toml index 5f178630..daca4ece 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -20,13 +20,6 @@ dependencies = [ "pypdf>=6.13.3,<7.0", ] -[project.urls] -Homepage = "https://contextwisdomlab.github.io/newsdom-api/" -Documentation = "https://contextwisdomlab.github.io/newsdom-api/" -Repository = "https://github.com/ContextualWisdomLab/newsdom-api" -Issues = "https://github.com/ContextualWisdomLab/newsdom-api/issues" -Security = "https://github.com/ContextualWisdomLab/newsdom-api/security/advisories/new" - [project.optional-dependencies] dev = [ "pytest>=8.3,<10.0", diff --git a/src/newsdom_api/main.py b/src/newsdom_api/main.py index 8acba361..41acaacf 100644 --- a/src/newsdom_api/main.py +++ b/src/newsdom_api/main.py @@ -13,7 +13,7 @@ from pypdf.errors import PdfReadError from .errors import MineruIncompleteOutputError, MineruRuntimeUnavailableError -from .schemas import ErrorResponse, HealthResponse, ParseResponse +from .schemas import HealthResponse, ParseResponse from .service import parse_pdf_bytes MAX_PARSE_UPLOAD_BYTES = 20 * 1024 * 1024 @@ -38,7 +38,7 @@ version="0.2.0", contact={ "name": "Seongho Bae", - "url": "https://github.com/ContextualWisdomLab/newsdom-api", + "url": "https://github.com/Seongho-Bae/newsdom-api", }, license_info={ "name": "MIT License", @@ -132,10 +132,10 @@ def _validate_pdf_structure(pdf_bytes: bytes) -> None: "document using MinerU." ), responses={ - 413: {"model": ErrorResponse, "description": "Payload Too Large"}, - 415: {"model": ErrorResponse, "description": "Unsupported Media Type"}, - 502: {"model": ErrorResponse, "description": "Bad Gateway"}, - 503: {"model": ErrorResponse, "description": "Service Unavailable"}, + 413: {"description": "Payload Too Large"}, + 415: {"description": "Unsupported Media Type"}, + 502: {"description": "Bad Gateway"}, + 503: {"description": "Service Unavailable"}, }, tags=["Parser"], ) diff --git a/src/newsdom_api/mineru_runner.py b/src/newsdom_api/mineru_runner.py index 552e952c..a9ac1b6d 100644 --- a/src/newsdom_api/mineru_runner.py +++ b/src/newsdom_api/mineru_runner.py @@ -4,7 +4,6 @@ import json import os -import re import shutil import subprocess import tempfile @@ -14,14 +13,14 @@ from .errors import MineruIncompleteOutputError, MineruRuntimeUnavailableError -_MINERU_COMMAND_ARG_PATTERN = re.compile(r"[\w /\\.:~()+\-@=,\[\]!']+") +_SHELL_CONTROL_CHARS = frozenset("&;|`$<>") def _mineru_command_arg(value: str | Path, *, label: str) -> str: """Validate a path or executable string before passing it to MinerU argv.""" value_str = str(value) - if not _MINERU_COMMAND_ARG_PATTERN.fullmatch(value_str): + if "\0" in value_str or any(char in value_str for char in _SHELL_CONTROL_CHARS): raise ValueError(f"Unsafe {label} for MinerU command") if value_str.startswith("-"): raise ValueError(f"Unsafe {label} for MinerU command") @@ -114,9 +113,13 @@ def _read_mineru_json(path: Path, *, artifact: str) -> Any: try: return json.loads(path.read_text(encoding="utf-8")) except json.JSONDecodeError as exc: - raise MineruIncompleteOutputError(f"{artifact} JSON was malformed") from exc + raise MineruIncompleteOutputError( + f"{artifact} JSON was malformed" + ) from exc except (OSError, UnicodeDecodeError) as exc: - raise MineruIncompleteOutputError(f"{artifact} JSON could not be read") from exc + raise MineruIncompleteOutputError( + f"{artifact} JSON could not be read" + ) from exc def _parse_mineru_output( diff --git a/src/newsdom_api/schemas.py b/src/newsdom_api/schemas.py index d1e97f51..9c5a88bb 100644 --- a/src/newsdom_api/schemas.py +++ b/src/newsdom_api/schemas.py @@ -6,53 +6,6 @@ from pydantic import BaseModel, Field -BOUNDING_BOX_EXAMPLE = {"x0": 120.0, "y0": 320.0, "x1": 420.0, "y1": 560.0} -CAPTION_EXAMPLE = { - "text": "Photo caption text extracted from the newspaper page.", - "bbox": {"x0": 120.0, "y0": 565.0, "x1": 420.0, "y1": 620.0}, -} -IMAGE_EXAMPLE = { - "path": "extracted_images/page1_img1.jpg", - "media_type": "image", - "bbox": BOUNDING_BOX_EXAMPLE, - "captions": [CAPTION_EXAMPLE], - "footnotes": [], -} -ARTICLE_EXAMPLE = { - "article_id": "page-1-article-1", - "headline": "Morning edition headline", - "bbox": {"x0": 10.0, "y0": 20.0, "x1": 760.0, "y1": 900.0}, - "body_blocks": [ - "First paragraph of the article body.", - "Second paragraph continues the story.", - ], - "images": [IMAGE_EXAMPLE], - "captions": [], - "footnotes": [], -} -PAGE_EXAMPLE = { - "page_number": 1, - "width": 800.5, - "height": 1200.0, - "articles": [ARTICLE_EXAMPLE], - "ads": ["Advertisement block text"], - "headers": ["2026-04-09 morning edition"], - "footers": ["Published by NewsDOM sample publisher"], - "page_numbers": ["1"], -} -PARSE_QUALITY_EXAMPLE = { - "status": "success", - "parser": "mineru", - "warnings": ["Page 1: overlapping layout blocks were normalized."], -} -PARSE_RESPONSE_EXAMPLE = { - "document_id": "sample-newspaper", - "pages": [PAGE_EXAMPLE], - "quality": PARSE_QUALITY_EXAMPLE, -} -HEALTH_RESPONSE_EXAMPLE = {"status": "ok"} -ERROR_RESPONSE_EXAMPLE = {"detail": "Unsupported Media Type"} - class BoundingBox(BaseModel): """Axis-aligned bounding box expressed in page coordinates.""" @@ -62,8 +15,6 @@ class BoundingBox(BaseModel): x1: float = Field(..., description="Rightmost X coordinate of the bounding box.") y1: float = Field(..., description="Bottommost Y coordinate of the bounding box.") - model_config = {"json_schema_extra": {"example": BOUNDING_BOX_EXAMPLE}} - class CaptionNode(BaseModel): """Caption text associated with an image or figure.""" @@ -74,8 +25,6 @@ class CaptionNode(BaseModel): description="Bounding box of the caption when parser coordinates are available.", ) - model_config = {"json_schema_extra": {"example": CAPTION_EXAMPLE}} - class ImageNode(BaseModel): """Image metadata preserved in the canonical page structure.""" @@ -98,8 +47,6 @@ class ImageNode(BaseModel): description="Footnotes associated with this image.", ) - model_config = {"json_schema_extra": {"example": IMAGE_EXAMPLE}} - class ArticleNode(BaseModel): """Article-level grouping of headline, body blocks, and related media.""" @@ -129,8 +76,6 @@ class ArticleNode(BaseModel): description="Footnotes associated with the article.", ) - model_config = {"json_schema_extra": {"example": ARTICLE_EXAMPLE}} - class PageNode(BaseModel): """Single parsed page including article, ad, and header groupings.""" @@ -167,8 +112,6 @@ class PageNode(BaseModel): description="Visible page-number text blocks extracted from this page.", ) - model_config = {"json_schema_extra": {"example": PAGE_EXAMPLE}} - class ParseQuality(BaseModel): """Quality metadata describing parser provenance and warnings.""" @@ -184,8 +127,6 @@ class ParseQuality(BaseModel): description="Non-fatal warnings encountered during the parsing process.", ) - model_config = {"json_schema_extra": {"example": PARSE_QUALITY_EXAMPLE}} - class ParseResponse(BaseModel): """Top-level API response for a parsed document.""" @@ -202,8 +143,6 @@ class ParseResponse(BaseModel): description="Metadata detailing parsing provenance and encountered warnings.", ) - model_config = {"json_schema_extra": {"example": PARSE_RESPONSE_EXAMPLE}} - class HealthResponse(BaseModel): """Liveness response model for deployment health checks.""" @@ -212,16 +151,3 @@ class HealthResponse(BaseModel): default="ok", description="Current operational status of the service.", ) - - model_config = {"json_schema_extra": {"example": HEALTH_RESPONSE_EXAMPLE}} - - -class ErrorResponse(BaseModel): - """Standardized error response model for HTTP exceptions.""" - - detail: str = Field( - ..., - description="A clear and specific error message describing what went wrong.", - ) - - model_config = {"json_schema_extra": {"example": ERROR_RESPONSE_EXAMPLE}} diff --git a/tests/test_benchmark_ocr.py b/tests/test_benchmark_ocr.py index 2163b941..20db0554 100644 --- a/tests/test_benchmark_ocr.py +++ b/tests/test_benchmark_ocr.py @@ -73,9 +73,7 @@ def test_benchmark_ocr_harness_json(mock_pdf_dir: Path, tmp_path: Path) -> None: assert results["summary"]["total_runs"] == 6 -def test_benchmark_ocr_recursive_and_csv( - mock_pdf_dir: Path, tmp_path: Path -) -> None: +def test_benchmark_ocr_recursive_and_csv(mock_pdf_dir: Path, tmp_path: Path) -> None: output_path = tmp_path / "results.csv" mock_engine = MagicMock(return_value={"status": "success", "page_count": 2}) @@ -118,9 +116,7 @@ def test_benchmark_ocr_no_pdfs(tmp_path: Path) -> None: ) -def test_benchmark_ocr_unknown_engine( - mock_pdf_dir: Path, tmp_path: Path -) -> None: +def test_benchmark_ocr_unknown_engine(mock_pdf_dir: Path, tmp_path: Path) -> None: """If an unknown engine is specified, ValueError is raised.""" with pytest.raises(ValueError, match="Unknown engine: fake_engine"): benchmark_ocr.main( diff --git a/tests/test_changelog.py b/tests/test_changelog.py index 24d18095..64eb9985 100644 --- a/tests/test_changelog.py +++ b/tests/test_changelog.py @@ -27,11 +27,11 @@ def test_initial_release_entry_keeps_added_section_and_links(): in text ) assert ( - "[Unreleased]: https://github.com/ContextualWisdomLab/newsdom-api/compare/v0.2.0...HEAD" + "[Unreleased]: https://github.com/Seongho-Bae/newsdom-api/compare/v0.2.0...HEAD" in text ) assert ( - "[0.1.0]: https://github.com/ContextualWisdomLab/newsdom-api/releases/tag/v0.1.0" + "[0.1.0]: https://github.com/Seongho-Bae/newsdom-api/releases/tag/v0.1.0" in text ) @@ -42,10 +42,10 @@ def test_changelog_prepares_the_0_1_1_release_entry(): assert "### Added" in text assert "*.intoto.jsonl" in text assert ( - "[Unreleased]: https://github.com/ContextualWisdomLab/newsdom-api/compare/v0.2.0...HEAD" + "[Unreleased]: https://github.com/Seongho-Bae/newsdom-api/compare/v0.2.0...HEAD" in text ) assert ( - "[0.1.1]: https://github.com/ContextualWisdomLab/newsdom-api/compare/v0.1.0...v0.1.1" + "[0.1.1]: https://github.com/Seongho-Bae/newsdom-api/compare/v0.1.0...v0.1.1" in text ) diff --git a/tests/test_derive_private_baseline.py b/tests/test_derive_private_baseline.py index af166ae7..0288bf6c 100644 --- a/tests/test_derive_private_baseline.py +++ b/tests/test_derive_private_baseline.py @@ -124,9 +124,7 @@ def test_derive_baseline_no_pdfs(tmp_path: Path) -> None: @patch("tools.derive_private_baseline.parse_pdf_bytes") -def test_derive_baseline_http_exception( - mock_parse_pdf_bytes, tmp_path: Path -) -> None: +def test_derive_baseline_http_exception(mock_parse_pdf_bytes, tmp_path: Path) -> None: """HTTPException from parse_pdf_bytes should be wrapped in RuntimeError.""" fixtures_dir = tmp_path / "fixtures" fixtures_dir.mkdir() @@ -153,9 +151,7 @@ def test_main_success(mock_derive_baseline, tmp_path: Path) -> None: ["--private-fixtures-dir", str(fixtures_dir), str(output_path)] ) - mock_derive_baseline.assert_called_once_with( - fixtures_dir, output_path, False, True - ) + mock_derive_baseline.assert_called_once_with(fixtures_dir, output_path, False, True) @patch("tools.derive_private_baseline.derive_baseline") @@ -173,9 +169,7 @@ def test_main_success_with_options(mock_derive_baseline, tmp_path: Path) -> None ] ) - mock_derive_baseline.assert_called_once_with( - fixtures_dir, output_path, True, False - ) + mock_derive_baseline.assert_called_once_with(fixtures_dir, output_path, True, False) @patch("tools.derive_private_baseline.derive_baseline") @@ -199,9 +193,7 @@ def test_main_runtime_error(mock_derive_baseline, tmp_path: Path, capsys) -> Non @patch("tools.derive_private_baseline.derive_baseline") -def test_main_unexpected_error( - mock_derive_baseline, tmp_path: Path, capsys -) -> None: +def test_main_unexpected_error(mock_derive_baseline, tmp_path: Path, capsys) -> None: """main() should exit(1) and print an unexpected error.""" fixtures_dir = tmp_path / "fixtures" output_path = tmp_path / "baseline.json" diff --git a/tests/test_errors.py b/tests/test_errors.py index 8f7dd5d9..86d16f86 100644 --- a/tests/test_errors.py +++ b/tests/test_errors.py @@ -1,4 +1,7 @@ -from newsdom_api.errors import MineruIncompleteOutputError, MineruRuntimeUnavailableError +from newsdom_api.errors import ( + MineruIncompleteOutputError, + MineruRuntimeUnavailableError, +) def test_mineru_runtime_unavailable_error_initialization(): diff --git a/tests/test_health.py b/tests/test_health.py index eaea135e..0667b90e 100644 --- a/tests/test_health.py +++ b/tests/test_health.py @@ -51,24 +51,9 @@ def test_openapi_metadata_includes_contact_and_license(): info = response.json()["info"] assert info["contact"] == { "name": "Seongho Bae", - "url": "https://github.com/ContextualWisdomLab/newsdom-api", + "url": "https://github.com/Seongho-Bae/newsdom-api", } assert info["license"] == { "name": "MIT License", "identifier": "MIT", } - - -def test_openapi_response_schemas_use_real_examples(): - client = TestClient(app) - response = client.get("/openapi.json") - - assert response.status_code == 200 - schemas = response.json()["components"]["schemas"] - - assert schemas["HealthResponse"]["example"] == {"status": "ok"} - assert schemas["ParseResponse"]["example"]["document_id"] == "sample-newspaper" - assert schemas["ParseResponse"]["example"]["quality"]["parser"] == "mineru" - assert schemas["ErrorResponse"]["example"] == {"detail": "Unsupported Media Type"} - assert "property1" not in str(schemas["HealthResponse"]) - assert "property2" not in str(schemas["HealthResponse"]) diff --git a/tests/test_manual_docs.py b/tests/test_manual_docs.py index 0b73bca1..a0f499d2 100644 --- a/tests/test_manual_docs.py +++ b/tests/test_manual_docs.py @@ -40,46 +40,6 @@ def test_api_reference_documents_sanitized_parse_failure_semantics() -> None: assert expected in text -def test_api_reference_embeds_checked_in_visual_previews() -> None: - text = Path("manual/api-reference.md").read_text(encoding="utf-8") - - for expected in [ - "### 문서 UI 미리보기", - "assets/swagger-ui.png", - "assets/redoc.png", - "정적 레이아웃 예시", - "/openapi.json", - ]: - assert expected in text - - for image_path in [ - Path("manual/assets/swagger-ui.png"), - Path("manual/assets/redoc.png"), - ]: - assert image_path.is_file() - image_bytes = image_path.read_bytes() - assert image_bytes.startswith(b"\x89PNG\r\n\x1a\n") - assert image_bytes - - -def test_manual_security_page_documents_private_reporting_flow() -> None: - text = Path("manual/security.md").read_text(encoding="utf-8") - - for expected in [ - "# 보안 및 취약점 제보", - "https://github.com/ContextualWisdomLab/newsdom-api/security/advisories/new", - "공개 이슈를 열지 말고", - "합성 fixture", - "secret", - "production credential", - "`develop`", - "`main`", - "7일 이내", - "30일 이내", - ]: - assert expected in text - - def test_development_doc_uses_tree_wording(): text = Path("manual/development.md").read_text(encoding="utf-8") assert "트리 구조의 DOM" in text diff --git a/tests/test_mineru_runner_paths.py b/tests/test_mineru_runner_paths.py index f4591747..15de431d 100644 --- a/tests/test_mineru_runner_paths.py +++ b/tests/test_mineru_runner_paths.py @@ -5,7 +5,10 @@ import pytest from newsdom_api import mineru_runner -from newsdom_api.errors import MineruIncompleteOutputError, MineruRuntimeUnavailableError +from newsdom_api.errors import ( + MineruIncompleteOutputError, + MineruRuntimeUnavailableError, +) class _FakeTempDir: @@ -121,9 +124,7 @@ def test_build_mineru_command_rejects_option_like_arguments( input_pdf: Path, output_dir: Path, mineru_bin: str, match: str ): with pytest.raises(ValueError, match=match): - mineru_runner.build_mineru_command( - input_pdf, output_dir, mineru_bin=mineru_bin - ) + mineru_runner.build_mineru_command(input_pdf, output_dir, mineru_bin=mineru_bin) def test_execute_mineru_uses_safe_subprocess_options(monkeypatch): diff --git a/tests/test_project_metadata.py b/tests/test_project_metadata.py index f24eaeff..bf096e78 100644 --- a/tests/test_project_metadata.py +++ b/tests/test_project_metadata.py @@ -2,26 +2,6 @@ from pathlib import Path import re -import sys - - -def _project_urls(text: str) -> dict[str, str]: - if sys.version_info >= (3, 11): - import tomllib - - return tomllib.loads(text)["project"]["urls"] - - match = re.search( - r"^\[project\.urls\]\n(?P(?:^[A-Za-z].*\n?)+)", - text, - re.MULTILINE, - ) - assert match is not None, "pyproject.toml is missing [project.urls]" - urls: dict[str, str] = {} - for line in match.group("body").splitlines(): - key, value = line.split("=", 1) - urls[key.strip()] = value.strip().strip('"') - return urls def _locked_package_version(name: str) -> tuple[int, ...]: @@ -162,20 +142,6 @@ def test_project_uses_spdx_license_string_not_deprecated_table(): assert 'license = {text = "MIT"}' not in text -def test_project_urls_fill_public_package_metadata(): - text = Path("pyproject.toml").read_text(encoding="utf-8") - - assert _project_urls(text) == { - "Homepage": "https://contextwisdomlab.github.io/newsdom-api/", - "Documentation": "https://contextwisdomlab.github.io/newsdom-api/", - "Repository": "https://github.com/ContextualWisdomLab/newsdom-api", - "Issues": "https://github.com/ContextualWisdomLab/newsdom-api/issues", - "Security": ( - "https://github.com/ContextualWisdomLab/newsdom-api/security/advisories/new" - ), - } - - def test_project_declares_locked_fuzz_extra_without_bundling_nvidia_stack(): text = Path("pyproject.toml").read_text(encoding="utf-8") assert "fuzz = [" in text diff --git a/tests/test_readme.py b/tests/test_readme.py index de5aa710..72b79bca 100644 --- a/tests/test_readme.py +++ b/tests/test_readme.py @@ -1,7 +1,5 @@ from pathlib import Path -import yaml - def test_readme_points_to_user_and_maintainer_docs(): text = Path("README.md").read_text(encoding="utf-8") @@ -15,31 +13,6 @@ def test_readme_includes_scorecard_badge(): assert "securityscorecards.dev" in text -def test_public_docs_point_to_current_org_repository(): - expected_repo = "github.com/ContextualWisdomLab/newsdom-api" - old_repo = "github.com/Seongho-Bae/newsdom-api" - - for path in [ - Path("README.md"), - Path("manual/index.md"), - Path("manual/workflow/ocr-accuracy-evidence.md"), - Path("mkdocs.yml"), - ]: - text = path.read_text(encoding="utf-8") - assert expected_repo in text, path - assert old_repo not in text, path - - mkdocs_text = Path("mkdocs.yml").read_text(encoding="utf-8") - assert "https://contextwisdomlab.github.io/newsdom-api/" in mkdocs_text - assert "https://seongho-bae.github.io/newsdom-api/" not in mkdocs_text - - -def test_mkdocs_nav_exposes_security_reporting_page(): - mkdocs_config = yaml.safe_load(Path("mkdocs.yml").read_text(encoding="utf-8")) - assert {"보안 제보": "security.md"} in mkdocs_config["nav"] - assert Path("manual/security.md").is_file() - - def test_contributing_mentions_develop_branch(): text = Path("CONTRIBUTING.md").read_text(encoding="utf-8") assert "develop" in text diff --git a/tests/test_schemas.py b/tests/test_schemas.py index 6d816c19..848f8faa 100644 --- a/tests/test_schemas.py +++ b/tests/test_schemas.py @@ -21,16 +21,8 @@ def test_page_node_openapi_schema_descriptions(): schema = PageNode.model_json_schema() properties = schema["properties"] - assert properties["page_number"]["description"] == "One-based page number from the parsed PDF." + assert ( + properties["page_number"]["description"] + == "One-based page number from the parsed PDF." + ) assert properties["articles"]["description"] == "Articles extracted from this page." - - -def test_parse_response_schema_has_visual_documentation_example(): - schema = ParseResponse.model_json_schema() - example = schema["example"] - - assert example["document_id"] == "sample-newspaper" - assert example["pages"][0]["articles"][0]["headline"] == "Morning edition headline" - assert example["quality"]["warnings"] == [ - "Page 1: overlapping layout blocks were normalized." - ] diff --git a/tests/test_security_policy.py b/tests/test_security_policy.py index 9cf32d66..401b8b95 100644 --- a/tests/test_security_policy.py +++ b/tests/test_security_policy.py @@ -14,10 +14,7 @@ def test_security_policy_covers_reporting_and_supported_branches(): def test_security_policy_includes_explicit_reporting_links(): text = Path("SECURITY.md").read_text(encoding="utf-8") - assert ( - "https://github.com/ContextualWisdomLab/newsdom-api/security/advisories/new" - in text - ) + assert "https://github.com/Seongho-Bae/newsdom-api/security/advisories/new" in text assert "https://github.com/seonghobae" in text diff --git a/tests/test_tools_batch_parse.py b/tests/test_tools_batch_parse.py index 123b175d..d3eaaaec 100644 --- a/tests/test_tools_batch_parse.py +++ b/tests/test_tools_batch_parse.py @@ -33,7 +33,9 @@ def test_batch_parse_success(mock_parse, mock_pdf_dir, tmp_path, capsys): @patch("tools.batch_parse_pdf.parse_pdf_bytes") -def test_batch_parse_recursive_preserves_relative_paths(mock_parse, mock_pdf_dir, tmp_path): +def test_batch_parse_recursive_preserves_relative_paths( + mock_parse, mock_pdf_dir, tmp_path +): nested_dir = mock_pdf_dir / "section" nested_dir.mkdir() (nested_dir / "doc3.pdf").write_bytes(b"content3") diff --git a/tests/test_tools_extract_text.py b/tests/test_tools_extract_text.py new file mode 100644 index 00000000..7b2271e2 --- /dev/null +++ b/tests/test_tools_extract_text.py @@ -0,0 +1,118 @@ +import json +import sys +import warnings +from pathlib import Path +from unittest.mock import patch +import runpy + +import pytest + +from tools.extract_text import extract_text, main + + +def get_sample_data(): + return { + "pages": [ + { + "articles": [ + { + "headline": "First Headline", + "body_blocks": ["First paragraph.", "Second paragraph."], + }, + {"headline": "", "body_blocks": ["Only body block."]}, + ] + } + ] + } + + +def test_extract_text_stdout(tmp_path: Path, capsys): + """Test extract_text without output path (prints to stdout).""" + valid_json = tmp_path / "valid.json" + valid_json.write_text(json.dumps(get_sample_data()), encoding="utf-8") + + extract_text(valid_json) + captured = capsys.readouterr() + assert "# First Headline" in captured.out + assert "First paragraph." in captured.out + assert "Second paragraph." in captured.out + assert "Only body block." in captured.out + assert "---" in captured.out + + +def test_extract_text_file(tmp_path: Path, capsys): + """Test extract_text with output path.""" + valid_json = tmp_path / "valid.json" + valid_json.write_text(json.dumps(get_sample_data()), encoding="utf-8") + output_txt = tmp_path / "output.txt" + + extract_text(valid_json, output_path=output_txt) + captured = capsys.readouterr() + assert f"Extracted text saved to {output_txt}" in captured.out + + content = output_txt.read_text(encoding="utf-8") + assert "# First Headline" in content + assert "First paragraph." in content + assert "Second paragraph." in content + assert "Only body block." in content + assert "---" in content + + +def test_extract_text_file_not_found(tmp_path: Path): + """Test extract_text with non-existent file.""" + non_existent = tmp_path / "non_existent.json" + with pytest.raises(FileNotFoundError, match="File not found or is not a file"): + extract_text(non_existent) + + +def test_extract_text_invalid_suffix(tmp_path: Path): + """Test extract_text with non-json file.""" + invalid_suffix = tmp_path / "test.txt" + invalid_suffix.write_text("test") + with pytest.raises(ValueError, match="File must be a .json file."): + extract_text(invalid_suffix) + + +def test_main_success_stdout(tmp_path: Path, capsys): + """Test main function writing to stdout.""" + valid_json = tmp_path / "valid.json" + valid_json.write_text(json.dumps(get_sample_data()), encoding="utf-8") + + main([str(valid_json)]) + captured = capsys.readouterr() + assert "# First Headline" in captured.out + + +def test_main_success_file(tmp_path: Path, capsys): + """Test main function writing to file.""" + valid_json = tmp_path / "valid.json" + valid_json.write_text(json.dumps(get_sample_data()), encoding="utf-8") + output_txt = tmp_path / "output.txt" + + main([str(valid_json), "--output", str(output_txt)]) + captured = capsys.readouterr() + assert f"Extracted text saved to {output_txt}" in captured.out + + +def test_main_exception(tmp_path: Path, capsys): + """Test main function exception handling.""" + invalid_suffix = tmp_path / "test.txt" + invalid_suffix.write_text("test") + + with pytest.raises(SystemExit) as e: + main([str(invalid_suffix)]) + assert e.value.code == 1 + captured = capsys.readouterr() + assert "Error: File must be a .json file." in captured.err + + +def test_module_execution(tmp_path: Path): + """Test __main__ block execution with a valid json.""" + valid_json = tmp_path / "valid.json" + valid_json.write_text(json.dumps(get_sample_data()), encoding="utf-8") + + # Patch sys.argv to simulate running from command line + with patch.object(sys, "argv", ["extract_text.py", str(valid_json)]): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", RuntimeWarning) + runpy.run_module("tools.extract_text", run_name="__main__") diff --git a/tests/test_tools_validate_dom.py b/tests/test_tools_validate_dom.py new file mode 100644 index 00000000..9ed4bc44 --- /dev/null +++ b/tests/test_tools_validate_dom.py @@ -0,0 +1,121 @@ +import json +import sys +import warnings +from pathlib import Path +from unittest.mock import patch +import runpy + +import pytest + +from tools.validate_dom import validate_dom, main + + +def test_validate_dom_success(tmp_path: Path): + """Test successful validation of a valid DOM JSON.""" + valid_json = tmp_path / "valid.json" + valid_data = { + "document_id": "test_doc", + "pages": [ + { + "page_number": 1, + "articles": [ + { + "article_id": "art_1", + "headline": "Test Headline", + "body_blocks": ["Test body."], + } + ], + } + ], + "quality": {"status": "success", "parser": "mineru", "warnings": []}, + } + valid_json.write_text(json.dumps(valid_data), encoding="utf-8") + + assert validate_dom(valid_json) is True + + +def test_validate_dom_json_decode_error(tmp_path: Path, capsys): + """Test validation failure with malformed JSON.""" + malformed_json = tmp_path / "malformed.json" + malformed_json.write_text("{invalid json", encoding="utf-8") + + assert validate_dom(malformed_json) is False + captured = capsys.readouterr() + assert "JSON Decode Error" in captured.err + + +def test_validate_dom_validation_error(tmp_path: Path, capsys): + """Test validation failure with invalid DOM JSON.""" + invalid_json = tmp_path / "invalid.json" + # Missing required field 'document_id' + invalid_data = {"pages": []} + invalid_json.write_text(json.dumps(invalid_data), encoding="utf-8") + + assert validate_dom(invalid_json) is False + captured = capsys.readouterr() + assert "Validation Error" in captured.err + + +def test_validate_dom_file_not_found(tmp_path: Path): + """Test validate_dom with a non-existent file.""" + non_existent = tmp_path / "non_existent.json" + with pytest.raises(FileNotFoundError, match="File not found or is not a file"): + validate_dom(non_existent) + + +def test_validate_dom_invalid_suffix(tmp_path: Path): + """Test validate_dom with a non-json file.""" + invalid_suffix = tmp_path / "test.txt" + invalid_suffix.write_text("test") + with pytest.raises(ValueError, match="File must be a .json file."): + validate_dom(invalid_suffix) + + +def test_main_success(tmp_path: Path, capsys): + """Test main function with successful validation.""" + valid_json = tmp_path / "valid.json" + valid_data = {"document_id": "test_doc", "pages": []} + valid_json.write_text(json.dumps(valid_data), encoding="utf-8") + + main([str(valid_json)]) + captured = capsys.readouterr() + assert "Successfully validated valid.json" in captured.out + + +def test_main_validation_error(tmp_path: Path): + """Test main function with validation error (exits with 1).""" + invalid_json = tmp_path / "invalid.json" + invalid_data = {} + invalid_json.write_text(json.dumps(invalid_data), encoding="utf-8") + + with pytest.raises(SystemExit) as e: + main([str(invalid_json)]) + assert e.value.code == 1 + + +def test_main_exception(tmp_path: Path, capsys): + """Test main function with other exceptions (exits with 1).""" + invalid_suffix = tmp_path / "test.txt" + invalid_suffix.write_text("test") + + with pytest.raises(SystemExit) as e: + main([str(invalid_suffix)]) + assert e.value.code == 1 + captured = capsys.readouterr() + assert "Error: File must be a .json file." in captured.err + + +def test_module_execution(tmp_path: Path): + """Test __main__ block execution with a valid json.""" + valid_json = tmp_path / "valid.json" + valid_data = {"document_id": "test_doc", "pages": []} + valid_json.write_text(json.dumps(valid_data), encoding="utf-8") + + # Patch sys.argv to simulate running from command line + with patch.object(sys, "argv", ["validate_dom.py", str(valid_json)]): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", RuntimeWarning) + # Remove src dir from sys.path to hit the pragma: no cover path logic if needed, + # but since we are testing tools/validate_dom.py directly we just run it. + # actually we don't strictly need to remove it, run_module will work + runpy.run_module("tools.validate_dom", run_name="__main__") diff --git a/tools/batch_parse_pdf.py b/tools/batch_parse_pdf.py index 2be64376..1c1d6257 100644 --- a/tools/batch_parse_pdf.py +++ b/tools/batch_parse_pdf.py @@ -22,7 +22,9 @@ def batch_parse( output_dir.mkdir(parents=True, exist_ok=True) - pdf_files = sorted(input_dir.rglob("*.pdf") if recursive else input_dir.glob("*.pdf")) + pdf_files = sorted( + input_dir.rglob("*.pdf") if recursive else input_dir.glob("*.pdf") + ) if not pdf_files: print(f"No PDF files found in {input_dir}") return @@ -39,7 +41,9 @@ def batch_parse( json_output = json.dumps(output_dict, ensure_ascii=False, indent=indent) if recursive: - out_path = output_dir / pdf_path.relative_to(input_dir).with_suffix(".json") + out_path = output_dir / pdf_path.relative_to(input_dir).with_suffix( + ".json" + ) out_path.parent.mkdir(parents=True, exist_ok=True) else: out_path = output_dir / f"{pdf_path.stem}.json" diff --git a/tools/export_markdown.py b/tools/export_markdown.py index 4a751aae..26965272 100644 --- a/tools/export_markdown.py +++ b/tools/export_markdown.py @@ -54,12 +54,16 @@ def generate_markdown(data: dict[str, Any]) -> str: captions = article.get("captions", []) if captions: - lines.extend(f"- Caption: {_caption_text(caption)}" for caption in captions) + lines.extend( + f"- Caption: {_caption_text(caption)}" for caption in captions + ) lines.append("") footnotes = article.get("footnotes", []) if footnotes: - lines.extend(f"- Footnote: {_caption_text(footnote)}" for footnote in footnotes) + lines.extend( + f"- Footnote: {_caption_text(footnote)}" for footnote in footnotes + ) lines.append("") ads = page.get("ads", []) diff --git a/tools/extract_text.py b/tools/extract_text.py new file mode 100644 index 00000000..c856b28d --- /dev/null +++ b/tools/extract_text.py @@ -0,0 +1,65 @@ +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + + +def extract_text(json_path: Path, output_path: Path | None = None) -> None: + """Extract headline and body blocks text from parsed DOM JSON.""" + if not json_path.is_file(): + raise FileNotFoundError(f"File not found or is not a file: {json_path}") + if json_path.suffix.lower() != ".json": + raise ValueError("File must be a .json file.") + + data = json.loads(json_path.read_text(encoding="utf-8")) + extracted_lines = [] + + pages = data.get("pages", []) + for page in pages: + articles = page.get("articles", []) + for article in articles: + headline = article.get("headline", "") + if headline: + extracted_lines.append(f"# {headline}") + extracted_lines.append("") + + body_blocks = article.get("body_blocks", []) + for block in body_blocks: + extracted_lines.append(block) + extracted_lines.append("") + + extracted_lines.append("---") + extracted_lines.append("") + + extracted_text = "\n".join(extracted_lines) + + if output_path: + output_path.write_text(extracted_text, encoding="utf-8") + print(f"Extracted text saved to {output_path}") + else: + print(extracted_text) + + +def main(argv: list[str] | None = None) -> None: + """Run extract_text main entry point.""" + parser = argparse.ArgumentParser(description="Extract text from NewsDOM JSON.") + parser.add_argument("input", type=Path, help="Path to the JSON DOM file.") + parser.add_argument( + "--output", + type=Path, + help="Path to save the extracted text. If omitted, prints to stdout.", + ) + + args = parser.parse_args(argv) + + try: + extract_text(args.input, args.output) + except Exception as e: + print(f"Error: {e}", file=sys.stderr) + sys.exit(1) + + +if __name__ == "__main__": # pragma: no cover + main() diff --git a/tools/parse_pdf.py b/tools/parse_pdf.py index 07465937..ef22fac2 100644 --- a/tools/parse_pdf.py +++ b/tools/parse_pdf.py @@ -21,7 +21,9 @@ def _resolve_pdf_input(input_path: Path) -> Path: if input_path.suffix.lower() != ".pdf": raise ValueError("The input file must use a .pdf extension.") if not input_path.is_file(): - raise ValueError(f"The input file {input_path} does not exist or is not a file.") + raise ValueError( + f"The input file {input_path} does not exist or is not a file." + ) return input_path.resolve(strict=True) diff --git a/tools/validate_dom.py b/tools/validate_dom.py new file mode 100644 index 00000000..03933009 --- /dev/null +++ b/tools/validate_dom.py @@ -0,0 +1,58 @@ +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +_REPO_ROOT = Path(__file__).resolve().parents[1] +_SRC_ROOT = _REPO_ROOT / "src" +if str(_SRC_ROOT) not in sys.path: # pragma: no cover + sys.path.insert(0, str(_SRC_ROOT)) + +from pydantic import ValidationError # noqa: E402 + +from newsdom_api.schemas import ParseResponse # noqa: E402 + + +def validate_dom(json_path: Path) -> bool: + """Validate DOM JSON against ParseResponse schema.""" + if not json_path.is_file(): + raise FileNotFoundError(f"File not found or is not a file: {json_path}") + if json_path.suffix.lower() != ".json": + raise ValueError("File must be a .json file.") + + try: + data = json.loads(json_path.read_text(encoding="utf-8")) + ParseResponse.model_validate(data) + return True + except json.JSONDecodeError as e: + print(f"JSON Decode Error in {json_path.name}:", file=sys.stderr) + print(e, file=sys.stderr) + return False + except ValidationError as e: + print(f"Validation Error in {json_path.name}:", file=sys.stderr) + print(e, file=sys.stderr) + return False + + +def main(argv: list[str] | None = None) -> None: + """Run validate_dom main entry point.""" + parser = argparse.ArgumentParser(description="Validate NewsDOM JSON output.") + parser.add_argument("input", type=Path, help="Path to the JSON DOM file.") + + args = parser.parse_args(argv) + + try: + is_valid = validate_dom(args.input) + if is_valid: + print(f"Successfully validated {args.input.name}") + else: + sys.exit(1) + except Exception as e: + print(f"Error: {e}", file=sys.stderr) + sys.exit(1) + + +if __name__ == "__main__": # pragma: no cover + main()