feat(tools): DOM 검증 및 텍스트 추출 CLI 도구 추가 - #253
Conversation
- `tools/validate_dom.py`: 파싱된 DOM JSON 파일이 `ParseResponse` Pydantic 스키마 형식을 만족하는지 검사하는 도구 추가 - `tools/extract_text.py`: 파싱된 DOM JSON 파일에서 본문(headline, body_blocks) 등 텍스트만 추출하는 도구 추가 - 새로운 도구들의 테스트 파일(`tests/test_tools_validate_dom.py`, `tests/test_tools_extract_text.py`) 추가 (커버리지 100%) - `CHANGELOG.md` 업데이트
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
OpenCode Review Overview
Pull request overviewOpenCode reviewed the current-head bounded evidence and found no blocking issues. FindingsNo blocking findings. SummaryApproval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (19 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (19 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (14 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (14 files)"]
R2 --> V2["targeted test run"]
|
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including CHANGELOG.md, tests/test_benchmark_ocr.py, tests/test_derive_private_baseline.py, tests/test_errors.py, tests/test_mineru_runner_paths.py, and 9 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects CHANGELOG.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: New CLI tools with comprehensive tests and documentation
- Head SHA:
028b1f17e7ce0f725721b5930056828c92d69833 - Workflow run: 28478117547
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (6 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (6 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (8 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (8 files)"]
R2 --> V2["targeted test run"]
…ation-and-text-extraction-tools-14108974707487559791 # Conflicts: # CHANGELOG.md
There was a problem hiding this comment.
Pull request overview
This PR expands the tools/ CLI suite for newsdom-api by adding two developer-facing utilities: one to validate parsed NewsDOM JSON outputs against the canonical ParseResponse schema, and another to extract readable text (headline/body) from DOM JSON for downstream analysis and pipelines.
Changes:
- Added
tools/validate_dom.pyto validate a DOM JSON file usingnewsdom_api.schemas.ParseResponse. - Added
tools/extract_text.pyto extractheadlineandbody_blockstext to stdout or a file. - Added focused unit tests for both tools and updated
CHANGELOG.md(plus minor formatting-only updates across existing tools/tests).
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/validate_dom.py | New CLI to validate DOM JSON against the ParseResponse schema. |
| tools/extract_text.py | New CLI to extract human-readable text from DOM JSON. |
| tools/parse_pdf.py | Formatting-only change to error raising style. |
| tools/export_markdown.py | Formatting-only change for long lines.extend(...) expressions. |
| tools/batch_parse_pdf.py | Formatting-only change for long expressions / path building. |
| tests/test_tools_validate_dom.py | New unit tests covering validate_dom behavior and CLI execution. |
| tests/test_tools_extract_text.py | New unit tests covering extract_text behavior and CLI execution. |
| tests/test_tools_batch_parse.py | Formatting-only change to a long test function signature. |
| tests/test_schemas.py | Formatting-only change to a long assertion. |
| tests/test_mineru_runner_paths.py | Formatting-only change to imports and a wrapped call expression. |
| tests/test_errors.py | Formatting-only change to imports. |
| tests/test_derive_private_baseline.py | Formatting-only change to signatures and assertions. |
| tests/test_benchmark_ocr.py | Formatting-only change to long test signatures. |
| CHANGELOG.md | Added Korean changelog entries documenting the new CLI tools. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head evidence but found unresolved reviewer or review-agent threads before approval.
Findings
1. HIGH .github/workflows/opencode-review.yml:1 - Unresolved reviewer thread blocks automated approval
- Problem: OpenCode reached an APPROVE control result, but the approval step found unresolved, non-outdated human or review-agent thread evidence on the current pull request.
- Root cause: Reviewer and review-agent feedback can arrive after bounded model evidence is prepared, so the approval step must re-query GitHub immediately before publishing an approval.
- Fix: Address or resolve the listed reviewer thread(s), then re-run OpenCode on the current head.
- Regression test: Keep the approval gate querying reviewThreads(first: 100) after model output and before create_pull_review APPROVE, including bot review agents other than OpenCode itself.
Review thread evidence
Latest unresolved reviewer thread evidence
tools/validate_dom.py line 32
-
Latest reviewer comment: @copilot-pull-request-reviewer at 2026-07-01T05:15:35Z
-
Comment URL: #253 (comment)
-
Comment excerpt: 'validate_dom()' only treats Pydantic 'ValidationError' as a non-fatal validation result (returns 'False'). If the JSON is syntactically invalid, 'json.loads(...)' raises 'json.JSONDecodeError', which currently bubbles up to 'main()' and is printed as a generic 'Error: ...' message without the filename context. Handling 'JSONDecodeError' in 'validate_dom()' will make malformed JSON failures consistent and more actionable.
-
Result: REQUEST_CHANGES
-
Reason: unresolved reviewer or review-agent thread(s) were present before approval.
-
Head SHA:
afe137c0d54222f09a8c67c8ac722387ce4f3181 -
Workflow run: 28495144470
-
Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (6 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (6 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (8 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (8 files)"]
R2 --> V2["targeted test run"]
- `validate_dom()`에서 `json.JSONDecodeError`를 포착하여 유효성 검사 실패(`False`)로 일관되게 처리 - `tests/test_tools_validate_dom.py`에 malformed JSON에 대한 regression test 추가
The previous check run failed to lookup statusCheckRollup (CHECKS_LOOKUP_FAILED). Triggering another check run.
Trigger another check run. The previous run failed due to CHECKS_LOOKUP_FAILED, an infrastructure issue.
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head bounded evidence and found no blocking issues.
Findings
No blocking findings.
Summary
Approval sufficiency: bounded evidence supplied affirmative approval evidence for changed files, coverage/docstring posture, risk surfaces, and current-head verification; approval is not based merely on the absence of known blockers.
Verification posture: CodeGraph evidence was initialized and bounded current-head evidence reviewed for changed-file evidence including .jules/bolt.md, .jules/sentinel.md, CHANGELOG.md, README.md, SECURITY.md, and 28 more.
Linter/static: workflow/static review evidence is bounded by the current-head GitHub Checks gate and changed-file evidence.
TDD/regression: coverage execution evidence and focused changed hunks were reviewed from bounded-review-evidence.md.
Coverage: coverage execution evidence reports supported repository test suites passed.
Docstring coverage: coverage execution evidence reports configured repository docstring gates passed or docstring coverage was advisory.
DAG: CodeGraph/source-backed behavior map connects .jules/bolt.md to the affected review, runtime, or workflow path and required checks.
PoC/execution: coverage-evidence job executed on the current head and reported PASS.
DDD/domain: workflow and repository-governance invariants were reviewed against changed files in bounded evidence.
CDD/context: CodeGraph evidence, changed-file history, and focused hunks were reviewed from bounded-review-evidence.md.
Similar issues: changed-file history evidence was reviewed for comparable local precedents.
Claim/concept check: bounded evidence, repository source, current-head workflow evidence, and, where numeric, scientific, statistical, or literature-backed claims are affected, original-paper/formula evidence and parameter-recovery expectations were used for claims.
Standards search: standards and external-source checks are delegated to configured OpenCode web_search/Context7/DeepWiki sources when applicable; no evidence-backed standards blocker is present in bounded evidence.
Compatibility/convention: changed workflow/script conventions, object naming, and reserved-word safety for schema/API/config/code surfaces were checked in bounded evidence.
Breaking-change/backcompat: deployment evidence and changed-file history were checked for backward-compatibility risk.
Performance: changed surfaces were checked for performance risk in bounded evidence.
Developer experience: changed automation, review, test, setup, and maintenance surfaces were checked for helpful or obstructive DX impact in bounded evidence.
User experience: connected user, operator, API, CLI, documentation, review-comment, status-check, rendering, and workflow-reader behavior was checked for contradictions against code, docs, and tests in bounded evidence.
Visual/DOM: Playwright visual, DOM locator, ARIA snapshot, console, and responsive evidence were checked when a web UI surface was present; for non-web surfaces, API/CLI/log/docs/workflow interaction evidence was reviewed instead.
Accessibility/i18n: accessibility, localization, and human-readable text surfaces were checked where UI, CLI, API message, docs, logs, or review text changed.
Supply-chain/license: dependency, package, model, container, and external-tool changes were checked in bounded evidence.
Packaging: package, build, test, lint, and security contracts were checked in bounded evidence.
Security/privacy: workflow-token, review-gate, and repository-automation security/privacy boundaries were checked in bounded evidence.
- Result: APPROVE
- Reason: New CLI utilities with comprehensive tests and documentation
- Head SHA:
dab717813172a1daae7851abebe7b63b5c8007e2 - Workflow run: 28497195790
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (19 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (19 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Test (14 files)"]
S2 --> I2["regression suite"]
I2 --> R2["Review risk: Test (14 files)"]
R2 --> V2["targeted test run"]
What (구현 내용)
tools/validate_dom.py추가: 파싱된 NewsDOM JSON 출력 파일이 핵심 API 스키마인ParseResponse규칙을 올바르게 준수하고 있는지 검증하는 CLI 도구를 새롭게 기획하고 구현했습니다.tools/extract_text.py추가: 복잡한 DOM JSON 구조에서headline및body_blocks등 사람이 읽을 수 있는 텍스트만 순수하게 추출하여 화면에 출력하거나 파일로 저장할 수 있는 CLI 도구를 추가했습니다.tests/test_tools_validate_dom.py,tests/test_tools_extract_text.py)를 추가하여tools디렉터리의 테스트 커버리지 100%를 달성했습니다.CHANGELOG.md의[Unreleased]섹션에 한국어로 기록했습니다.Why (해결하는 문제)
Impact (성능/영향)
validate_dom도구를 통해 합성 데이터나 실제 파싱 파이프라인의 결과물이 항상 유효한 스키마를 유지하고 있음을 빠르게 확인할 수 있습니다.extract_text도구를 통해 분석가나 후속 NLP 파이프라인이 DOM 구조에 얽매이지 않고 순수 기사 텍스트에 쉽게 접근할 수 있습니다.Measurement (측정/검증 방법)
uv run ruff format tools/ tests/및uv run ruff check tools/ tests/를 실행하여 코드 스타일이 정책에 부합하는지 확인했습니다.uv run pytest --cov=tools --cov-branch --cov-report=term-missing --cov-fail-under=100를 실행하여 새롭게 추가된 코드를 포함한 모든tools하위 코드의 테스트 커버리지가 100%임을 확인했습니다.PR created automatically by Jules for task 14108974707487559791 started by @seonghobae