Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a6c2db5
ci: expand CodeQL coverage and tighten repo guardrails
seonghobae Apr 10, 2026
7945aec
fix: unblock fuzz CI and harden governance tests
seonghobae Apr 10, 2026
bb1cb40
fix: forward libFuzzer flags so ClusterFuzzLite fuzz jobs run
seonghobae Apr 10, 2026
551b374
docs: align repository truth sources with current workflow state
seonghobae Apr 10, 2026
a9d0ebb
Merge pull request #37 from Seongho-Bae/chore/truth-source-alignment
seonghobae Apr 10, 2026
1d263a8
docs: add canonical engineering truth sources
seonghobae Apr 10, 2026
3e7ae79
fix: harden workflow attestation and fuzz builder paths
seonghobae Apr 10, 2026
70383e4
Merge pull request #38 from Seongho-Bae/chore/canonical-engineering-docs
seonghobae Apr 10, 2026
737696b
Merge pull request #39 from Seongho-Bae/chore/workflow-robustness-har…
seonghobae Apr 10, 2026
7fa1b7f
docs: scope markdownlint around active repository docs
seonghobae Apr 10, 2026
1af74b0
Merge pull request #42 from Seongho-Bae/chore/markdownlint-scope-policy
seonghobae Apr 10, 2026
a0e3ac9
docs: pin the supported MkDocs toolchain stance
seonghobae Apr 10, 2026
83c84a1
Merge pull request #43 from Seongho-Bae/chore/docs-toolchain-hold
seonghobae Apr 10, 2026
b3d84ae
docs: align public setup guidance with uv defaults
seonghobae Apr 10, 2026
ae12473
Merge pull request #44 from Seongho-Bae/chore/uv-first-public-setup
seonghobae Apr 10, 2026
06ba722
fix: lock pypdf to patched release
seonghobae Apr 11, 2026
fbb13ef
Merge pull request #46 from Seongho-Bae/fix/pypdf-6-10-lock
seonghobae Apr 11, 2026
bbf0dbc
docs: record reviewer-capacity ruleset alignment plan
seonghobae Apr 11, 2026
c042621
docs: align governance truth with single-maintainer exception
seonghobae Apr 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
* @Seongho-Bae

# Security and workflow ownership
.github/ @Seongho-Bae
docs/ @Seongho-Bae
manual/ @Seongho-Bae
1 change: 1 addition & 0 deletions .github/workflows/clusterfuzzlite.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
language: python
sanitizer: address
github-token: ${{ github.token }}
bad-build-check: false

- name: Run fuzzers
uses: google/clusterfuzzlite/actions/run_fuzzers@52ecc61cb587ee99c26825a112a21abf19c7448c
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ permissions:

jobs:
analyze:
name: codeql (python)
name: codeql (python, actions)
runs-on: ubuntu-latest
permissions:
contents: read
Expand All @@ -27,7 +27,7 @@ jobs:
- name: Initialize CodeQL
uses: github/codeql-action/init@c10b8064de6f491fea524254123dbe5e09572f13
with:
languages: python
languages: python, actions

- name: Autobuild
uses: github/codeql-action/autobuild@c10b8064de6f491fea524254123dbe5e09572f13
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ on:
push:
branches:
- main
- master
- develop
paths:
- 'manual/**'
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,8 @@ build/
*.pyc
private/
tmp/
site/
registered_agents.json
task_agent_mapping.json
content_file_3982.pdf
page1_300dpi-1.png
39 changes: 28 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ NewsDOM API parses scanned Japanese newspaper PDFs into DOM-like article trees.

- Primary engine: `MinerU` pipeline backend
- Service wrapper: FastAPI
- Output: canonical JSON with pages, articles, headlines, body blocks, images, captions, and quality metadata
- Output: canonical JSON with pages, articles, headlines, body
blocks, images, captions, and quality metadata

## Quickstart

Expand All @@ -20,7 +21,8 @@ source .venv/bin/activate
pip install -e ".[dev]"
```

To enable real parsing with MinerU, install the MinerU CLI separately in the environment that will execute parsing:
To enable real parsing with MinerU, install the MinerU CLI separately in the
environment that will execute parsing:

```bash
pip install "mineru[pipeline]==3.0.9"
Expand All @@ -29,7 +31,7 @@ pip install "mineru[pipeline]==3.0.9"
### Run

```bash
uvicorn newsdom_api.main:app --reload
uvicorn --app-dir src newsdom_api.main:app --reload
```

### Docker
Expand All @@ -39,9 +41,14 @@ docker build -t newsdom-api .
docker run -p 8000:8000 newsdom-api
```

The default image exposes the REST API on port `8000` as a lean multi-arch service image. It is suitable for `linux/amd64` and `linux/arm64`, including Apple Silicon hosts running the API service inside Docker.
The default image exposes the REST API on port `8000` as a lean multi-arch
service image. It is suitable for `linux/amd64` and `linux/arm64`, including
Apple Silicon hosts running the API service inside Docker.

The lean image is intentionally a REST API shell: `/health`, `/docs`, and OpenAPI endpoints are available immediately, while real `/parse` execution still requires a compatible MinerU runtime to be available inside the container image.
The lean image is intentionally a REST API shell: `/health`, `/docs`, and
OpenAPI endpoints are available immediately, while real `/parse` execution
still requires a compatible MinerU runtime to be available inside the
container image.

For heavier parsing deployments, build the optional NVIDIA-oriented variant:

Expand All @@ -50,9 +57,14 @@ docker build -f Dockerfile.nvidia -t newsdom-api:nvidia .
docker run --gpus all -p 8000:8000 newsdom-api:nvidia
```

`Dockerfile.nvidia` is intended for Linux/NVIDIA environments and is `linux/amd64`-only. Apple Silicon can run the lean API image, but Docker Desktop does not expose Apple GPU acceleration to Linux containers, so real GPU-accelerated parsing should stay on a native Apple Silicon path instead of the containerized runtime.
`Dockerfile.nvidia` is intended for Linux/NVIDIA environments and is
`linux/amd64`-only. Apple Silicon can run the lean API image, but Docker
Desktop does not expose Apple GPU acceleration to Linux containers, so real
GPU-accelerated parsing should stay on a native Apple Silicon path instead of
the containerized runtime.

The NVIDIA variant is `linux/amd64`-only and is meant for hosts that can provide the CUDA user-space/runtime stack required by MinerU.
The NVIDIA variant is `linux/amd64`-only and is meant for hosts that can
provide the CUDA user-space/runtime stack required by MinerU.

### Parse a PDF

Expand All @@ -72,18 +84,23 @@ pytest
python fuzzers/dom_builder_fuzzer.py --smoke tests/fixtures/mineru_sample.json
```

The repository also enforces a `quality-gate` workflow with 100% source coverage and docstring audit coverage.
The repository also enforces a `quality-gate` workflow with 100% source
coverage and docstring audit coverage.

## Fixtures and provenance

This repository ships only synthetic test fixtures and derived structural baselines. For fixture provenance and regeneration notes, see `tests/fixtures/README.md`.
This repository ships only synthetic test fixtures and derived structural
baselines. For fixture provenance and regeneration notes, see
`tests/fixtures/README.md`.

## Development

Development setup, fixture handling rules, and local-only baseline maintenance are documented in `CONTRIBUTING.md`.
Development setup, fixture handling rules, and local-only baseline
maintenance are documented in `CONTRIBUTING.md`.

Security reporting guidance is documented in `SECURITY.md`.
Version tags trigger a GitHub-native release workflow that builds distribution artifacts, checksums, and provenance attestations.
Version tags trigger a GitHub-native release workflow that builds
distribution artifacts, checksums, and provenance attestations.

Project history is tracked in `CHANGELOG.md`.

Expand Down
27 changes: 20 additions & 7 deletions docs/adr/0001-openssf-best-practices-badge.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,44 @@ Accepted

## Context

The repository already has branch protection, CI checks, CodeQL, OpenSSF Scorecard, Dependabot, a security policy, locked workflow dependencies, and a planned release pipeline. Scorecard still reports a best-practices gap because the OpenSSF Best Practices badge program has not been started.
The repository already has branch protection, CI checks, CodeQL, OpenSSF
Scorecard, Dependabot, a security policy, locked workflow dependencies, and a
planned release pipeline. Scorecard still reports a best-practices gap because
the OpenSSF Best Practices badge program has not been started.

The current repository also has only one organization member and one repository collaborator, so external reviewer capacity is not yet in place. The first tagged release is not available yet because the current PR stack still needs external review before it can merge into protected branches.
The current repository also has only one organization member and one repository
collaborator, so external reviewer capacity is not yet in place. The first
tagged release is not available yet because the current PR stack still needs
external review before it can merge into protected branches.

## Decision

We will **defer** OpenSSF Best Practices badge enrollment until after:

1. the current protected-branch PR stack is merged,
2. the first tagged release has been produced with release provenance, and
3. at least one external reviewer is available for normal protected-branch review flow.
3. at least one external reviewer is available for normal protected-branch
review flow.

## Consequences

### Positive

- Keeps focus on finishing concrete repository hardening already underway.
- Avoids starting a badge questionnaire before the release and review processes are stable.
- Avoids starting a badge questionnaire before the release and review processes
are stable.
- Preserves a clear, auditable decision in the repository.

### Negative

- Scorecard will continue to report the best-practices gap until enrollment is revisited.
- Scorecard will continue to report the best-practices gap until enrollment is
revisited.

## Follow-up

- Revisit enrollment after issue #8 and issue #10 are resolved.
- If the repository still intends to pursue the badge at that time, assign an owner and complete the OpenSSF questionnaire.
- Revisit enrollment after the blocked protected-branch PR stack has merged,
the first provenance-backed stable release has shipped, and reviewer
capacity exists beyond the sole author/admin account.
- If the repository still intends to pursue the badge at that time, assign an
owner, link the active release and reviewer-capacity issues, and complete
the OpenSSF questionnaire.
17 changes: 12 additions & 5 deletions docs/plans/2026-04-08-security-gates-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,34 @@

## Goal

Add GitHub-native security gates centered on OpenSSF Scorecard and make them required for protected branches.
Add GitHub-native security gates centered on OpenSSF Scorecard and make them
required for protected branches.

## Constraints

- The repository already uses GitHub Actions for tests.
- `main` and `develop` are branch-protected and should treat security workflows as required gates.
- `main` and `develop` are branch-protected and should treat security
workflows as required gates.
- Changes should remain lightweight for a small Python FastAPI repository.
- Security workflows should be least-privilege and avoid unnecessary secrets.

## Approaches considered

### Approach A — Scorecard only

- Add OpenSSF Scorecard workflow and badge.
- Low effort, but misses code and dependency risk coverage.

### Approach B — Scorecard + CodeQL + Dependency Review (recommended)

- Scorecard for repository posture.
- CodeQL for Python SAST.
- Dependency Review for pull-request dependency deltas.

### Approach C — Broad security suite (Semgrep, Trivy, etc.)
- Stronger coverage but too heavy for the current repository size and request scope.

- Stronger coverage but too heavy for the current repository size and request
scope.

## Decision

Expand All @@ -35,11 +41,12 @@ Choose **Approach B**.

- `pytest`
- `scorecard`
- `codeql (python)`
- `codeql (python, actions)`
- `dependency-review`

## Operational notes

- Add minimal README signal via a Scorecard badge.
- Keep workflow permissions explicit.
- Trigger CodeQL and Scorecard on both `develop` and `main` so both protected branches can require them.
- Trigger CodeQL and Scorecard on both `develop` and `main` so both protected
branches can require them.
64 changes: 38 additions & 26 deletions docs/plans/2026-04-08-security-gates.md
Original file line number Diff line number Diff line change
@@ -1,107 +1,119 @@
# Security Gates Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
> **For Claude:** REQUIRED SUB-SKILL: Use
> superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Add OpenSSF Scorecard and adjacent security workflows, then make them required checks on `main` and `develop`.
**Goal:** Add OpenSSF Scorecard and adjacent security workflows, then make them
required checks on `main` and `develop`.

**Architecture:** Extend GitHub Actions with three lightweight security workflows—Scorecard, CodeQL, and Dependency Review—using explicit job names so branch protection can require them deterministically.
**Architecture:** Extend GitHub Actions with three lightweight security
workflows—Scorecard, CodeQL, and Dependency Review—using explicit job names so
branch protection can require them deterministically.

**Tech Stack:** GitHub Actions, OpenSSF Scorecard Action, GitHub CodeQL, Dependency Review Action.
**Tech Stack:** GitHub Actions, OpenSSF Scorecard Action, GitHub CodeQL,
Dependency Review Action.

---

### Task 1: Add security workflows
## Task 1: Add security workflows

**Files:**

- Create: `.github/workflows/scorecards.yml`
- Create: `.github/workflows/codeql.yml`
- Create: `.github/workflows/dependency-review.yml`

**Step 1: Write the failing test**
### Task 1 / Step 1: Write the failing test

Add a repo test that asserts these workflow files exist.

**Step 2: Run test to verify it fails**
### Task 1 / Step 2: Run test to verify it fails

Run: `pytest tests/test_readme.py -v`
Expected: FAIL until the files are added and referenced.

**Step 3: Write minimal implementation**
### Task 1 / Step 3: Write minimal implementation

Add the workflows with explicit job names:

- `scorecard`
- `codeql (python)`
- `codeql (python, actions)`
- `dependency-review`

**Step 4: Run test to verify it passes**
### Task 1 / Step 4: Run test to verify it passes

Run: `pytest`
Expected: PASS.

**Step 5: Commit**
### Task 1 / Step 5: Commit

```bash
git add .github/workflows/*.yml tests/
git commit -m "ci: add security workflows"
```

### Task 2: Add minimal documentation signal
## Task 2: Add minimal documentation signal

**Files:**

- Modify: `README.md`

**Step 1: Write the failing test**
### Task 2 / Step 1: Write the failing test

Add a doc test that checks for the Scorecard badge URL or text.

**Step 2: Run test to verify it fails**
### Task 2 / Step 2: Run test to verify it fails

Run: `pytest tests/test_readme.py -v`
Expected: FAIL until README is updated.

**Step 3: Write minimal implementation**
### Task 2 / Step 3: Write minimal implementation

Add a single Scorecard badge near the title.

**Step 4: Run test to verify it passes**
### Task 2 / Step 4: Run test to verify it passes

Run: `pytest`
Expected: PASS.

### Task 3: Verify workflow syntax and tests
## Task 3: Verify workflow syntax and tests

**Files:**

- No new files.

**Step 1: Run unit tests**
### Task 3 / Step 1: Run unit tests

Run: `pytest`

**Step 2: Run warnings-as-errors tests**
### Task 3 / Step 2: Run warnings-as-errors tests

Run: `PYTHONWARNINGS=error pytest`

**Step 3: Validate workflow files structurally**
### Task 3 / Step 3: Validate workflow files structurally

Run a YAML parser or equivalent check against `.github/workflows/*.yml`.

### Task 4: Update branch protections
## Task 4: Update branch protections

**Files:**

- No new files.

**Step 1: Update `main` protection**
### Task 4 / Step 1: Update `main` protection

Require:

- `pytest`
- `scorecard`
- `codeql (python)`
- `codeql (python, actions)`
- `dependency-review`

**Step 2: Update `develop` protection**
### Task 4 / Step 2: Update `develop` protection

Require the same checks.

**Step 3: Verify protections**
### Task 4 / Step 3: Verify protections

Run GitHub API reads for both branches and confirm the contexts list matches exactly.
Run GitHub API reads for both branches and confirm the contexts list matches
exactly.
Loading
Loading