ci: solo-dev tooling bundle (zizmor, codeql, lychee, typos, deptry+vulture, mutmut, ADRs, OSSInsight) - #325
Conversation
Reviewer's GuideAdds a CI "solo-dev tooling" bundle: new GitHub Actions workflows for security/static analysis (zizmor, CodeQL), dependency/dead-code checks (deptry, vulture), link checking (lychee), spell checking (typos), and mutation testing (mutmut), plus an ADR system and an OSSInsight analytics badge in the README. Sequence diagram for the new lychee link-check workflowsequenceDiagram
actor Dev as Developer
participant Repo as GitHub_repo
participant Scheduler as Cron_scheduler
participant WF as link-check_workflow
participant Lychee as lychee_action
participant Issues as GitHub_issues
Scheduler->>WF: Trigger on cron (Mon 14:00 UTC)
Dev->>WF: Optional workflow_dispatch
WF->>Repo: actions/checkout@v4
WF->>Lychee: Run lycheeverse/lychee-action@v2 with args
Lychee-->>WF: exit_code, report_file(lychee/out.md)
alt exit_code != 0
WF->>Issues: create-issue-from-file@v5
Issues-->>WF: Issue created (title link-check: broken links detected)
else exit_code == 0
WF-->>WF: No issue created
end
Flow diagram for ADR authoring and evolutionflowchart TD
Start[Identify_non_obvious_architecture_decision] --> CheckNeed{Needs_long_term_rationale?}
CheckNeed -- No --> Skip[Document_only_in_PR_or_commit]
CheckNeed -- Yes --> Template[Copy_docs/adr/template.md_to_new_file]
Template --> Number[Assign_next_monotonic_number_NNNN]
Number --> Edit[Fill_in_Context_Decision_Alternatives_Consequences]
Edit --> Commit[Commit_ADR_under_docs/adr]
Commit --> Merge[Merge_to_main_branch]
Merge --> Use[Refer_to_ADR_in_future_changes]
Use --> Supersede{Decision_superseded?}
Supersede -- No --> End[Keep_ADR_append_only]
Supersede -- Yes --> NewADR[Create_new_ADR_linking_to_old]
NewADR --> End
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (13)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 22 minutes and 22 seconds.Comment |
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
zizmor found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
| matrix: | ||
| language: [python, actions] | ||
| steps: | ||
| - uses: step-security/harden-runner@v2 |
| deptry: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: step-security/harden-runner@v2 |
| with: | ||
| egress-policy: audit | ||
| - uses: actions/checkout@v4 | ||
| - uses: astral-sh/setup-uv@v5 |
| vulture: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: step-security/harden-runner@v2 |
| with: | ||
| egress-policy: audit | ||
| - uses: actions/checkout@v4 | ||
| - uses: astral-sh/setup-uv@v5 |
| with: | ||
| egress-policy: audit | ||
| - uses: actions/checkout@v4 | ||
| - uses: astral-sh/setup-uv@v5 |
| typos: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: step-security/harden-runner@v2 |
| with: | ||
| egress-policy: audit | ||
| - uses: actions/checkout@v4 | ||
| - uses: crate-ci/typos@master |
| zizmor: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: step-security/harden-runner@v2 |
| with: | ||
| egress-policy: audit | ||
| - uses: actions/checkout@v4 | ||
| - uses: astral-sh/setup-uv@v5 |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 3 issues, and left some high level feedback:
- The
typosworkflow usescrate-ci/typos@masterwhile the rest of the CI favors pinned or versioned actions; consider switching this to a tagged release or SHA so it aligns with your digest-pinning policy and avoids unexpected breaking changes. - In
link-check.yml,create-issue-from-fileassumes./lychee/out.mdexists; double-check that the lychee action is configured to actually write that file (and in the expected format) or add an explicit output path to avoid silent failures when creating issues.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `typos` workflow uses `crate-ci/typos@master` while the rest of the CI favors pinned or versioned actions; consider switching this to a tagged release or SHA so it aligns with your digest-pinning policy and avoids unexpected breaking changes.
- In `link-check.yml`, `create-issue-from-file` assumes `./lychee/out.md` exists; double-check that the lychee action is configured to actually write that file (and in the expected format) or add an explicit output path to avoid silent failures when creating issues.
## Individual Comments
### Comment 1
<location path=".github/workflows/mutation.yml" line_range="32-37" />
<code_context>
+ - run: uv sync --frozen --extra dev --extra archive
+ - name: Install mutmut
+ run: uv pip install mutmut
+ - name: Run mutation tests
+ id: mutmut
+ run: |
+ uv run mutmut run --paths-to-mutate=src/aelfrice --tests-dir=tests || true
+ uv run mutmut results > mutation-report.txt
+ cat mutation-report.txt
+ - uses: actions/upload-artifact@v4
+ with:
</code_context>
<issue_to_address>
**issue (bug_risk):** The `mutmut results` command can fail and cause the job to fail even though `mutmut run` is guarded with `|| true`.
Only `mutmut run` is guarded with `|| true`; `uv run mutmut results > mutation-report.txt` can still exit non‑zero if no results are produced, causing the step to fail. If this job is meant to be non‑blocking, either guard the `mutmut results` call as well, wrap both commands in a script that always exits 0, or mark the step with `continue-on-error: true` so failures here don’t break the workflow.
</issue_to_address>
### Comment 2
<location path="docs/adr/0002-two-repo-physical-separation.md" line_range="11" />
<code_context>
+
+aelfrice is developed alongside private artifacts: research notes, planning documents, LLC/business records, experiment data, draft posts. The product code is open-source and ships to PyPI; the surrounding workspace is not.
+
+Earlier the project tried a single-repo "filter on publish" pipeline: a quarantine staging area, allowlist (`.github-include`), gitleaks rules, and a publish script that filtered private content out before pushing to GitHub. This pipeline was complex, easy to mis-configure, and a single allowlist typo could leak private content irreversibly — git history is hard to scrub once pushed, and `refs/pull/N/head` pins commits permanently.
+
+## Decision
</code_context>
<issue_to_address>
**nitpick (typo):** Consider changing "mis-configure" to the more standard "misconfigure".
The hyphenated form is rarely used in technical writing; the standard spelling is "misconfigure," which also improves flow.
```suggestion
Earlier the project tried a single-repo "filter on publish" pipeline: a quarantine staging area, allowlist (`.github-include`), gitleaks rules, and a publish script that filtered private content out before pushing to GitHub. This pipeline was complex, easy to misconfigure, and a single allowlist typo could leak private content irreversibly — git history is hard to scrub once pushed, and `refs/pull/N/head` pins commits permanently.
```
</issue_to_address>
### Comment 3
<location path="docs/adr/0002-two-repo-physical-separation.md" line_range="18" />
<code_context>
+Use **two physically separate repositories** with no path between them:
+
+- `~/projects/aelfrice` — public product code. Origin is GitHub only.
+- `~/projects/aelfrice-lab` — private workspace. Origin is a self-hosted gitea only. A pre-push hook rejects any remote URL containing `github.com`.
+
+The boundary is the directory of origin, not a transformation. Test fixtures, paraphrased examples, and "abstracted" content all count as derived from their source directory.
</code_context>
<issue_to_address>
**suggestion (typo):** Capitalize "Gitea" as a proper noun.
As a product name, this should be capitalized as “self-hosted Gitea.”
```suggestion
- `~/projects/aelfrice-lab` — private workspace. Origin is a self-hosted Gitea only. A pre-push hook rejects any remote URL containing `github.com`.
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| - name: Run mutation tests | ||
| id: mutmut | ||
| run: | | ||
| uv run mutmut run --paths-to-mutate=src/aelfrice --tests-dir=tests || true | ||
| uv run mutmut results > mutation-report.txt | ||
| cat mutation-report.txt |
There was a problem hiding this comment.
issue (bug_risk): The mutmut results command can fail and cause the job to fail even though mutmut run is guarded with || true.
Only mutmut run is guarded with || true; uv run mutmut results > mutation-report.txt can still exit non‑zero if no results are produced, causing the step to fail. If this job is meant to be non‑blocking, either guard the mutmut results call as well, wrap both commands in a script that always exits 0, or mark the step with continue-on-error: true so failures here don’t break the workflow.
|
|
||
| aelfrice is developed alongside private artifacts: research notes, planning documents, LLC/business records, experiment data, draft posts. The product code is open-source and ships to PyPI; the surrounding workspace is not. | ||
|
|
||
| Earlier the project tried a single-repo "filter on publish" pipeline: a quarantine staging area, allowlist (`.github-include`), gitleaks rules, and a publish script that filtered private content out before pushing to GitHub. This pipeline was complex, easy to mis-configure, and a single allowlist typo could leak private content irreversibly — git history is hard to scrub once pushed, and `refs/pull/N/head` pins commits permanently. |
There was a problem hiding this comment.
nitpick (typo): Consider changing "mis-configure" to the more standard "misconfigure".
The hyphenated form is rarely used in technical writing; the standard spelling is "misconfigure," which also improves flow.
| Earlier the project tried a single-repo "filter on publish" pipeline: a quarantine staging area, allowlist (`.github-include`), gitleaks rules, and a publish script that filtered private content out before pushing to GitHub. This pipeline was complex, easy to mis-configure, and a single allowlist typo could leak private content irreversibly — git history is hard to scrub once pushed, and `refs/pull/N/head` pins commits permanently. | |
| Earlier the project tried a single-repo "filter on publish" pipeline: a quarantine staging area, allowlist (`.github-include`), gitleaks rules, and a publish script that filtered private content out before pushing to GitHub. This pipeline was complex, easy to misconfigure, and a single allowlist typo could leak private content irreversibly — git history is hard to scrub once pushed, and `refs/pull/N/head` pins commits permanently. |
| Use **two physically separate repositories** with no path between them: | ||
|
|
||
| - `~/projects/aelfrice` — public product code. Origin is GitHub only. | ||
| - `~/projects/aelfrice-lab` — private workspace. Origin is a self-hosted gitea only. A pre-push hook rejects any remote URL containing `github.com`. |
There was a problem hiding this comment.
suggestion (typo): Capitalize "Gitea" as a proper noun.
As a product name, this should be capitalized as “self-hosted Gitea.”
| - `~/projects/aelfrice-lab` — private workspace. Origin is a self-hosted gitea only. A pre-push hook rejects any remote URL containing `github.com`. | |
| - `~/projects/aelfrice-lab` — private workspace. Origin is a self-hosted Gitea only. A pre-push hook rejects any remote URL containing `github.com`. |
62c3519 to
cdb97ad
Compare
|
[claim:review:Toug:2026-04-30T22:18:13Z] |
|
Review: blocking on action-pinning regressions and a likely-broken issue-creation step. Test-plan items 1 and 2 also can't be evaluated until the required CI surface actually runs on this PR. Blockers
Non-blocking but worth fixing before merge
Observation, not a request The branch-protection-required checks ( Will not merge as-is. Please address blockers 1 and 2, then re-flag for review. |
|
[release:review:Toug:2026-04-30T22:19:35Z] |
c15f2ac to
cb4b23d
Compare
Static analysis for GitHub Actions workflow files. Catches common misconfigurations (broad token scopes, script-injection sinks, unpinned third-party actions) and uploads SARIF results to the Security tab. Triggered on workflow file changes, push to main, and weekly cron. Harden-Runner is in audit mode so the egress baseline can be reviewed before promoting to block.
GitHub-native semantic code analysis. The python language pack covers the src/ tree; the actions pack flags vulnerable workflow patterns (complementary to zizmor's static lint). security-extended + security-and-quality query suites enabled. Results land in the Security tab. Free for public repos.
Crawls all .md and .rst files for broken links once a week (Mondays 14:00 UTC). On failure, opens an issue with the broken-link report attached so it shows up in the issue list rather than a silent CI red.
crate-ci/typos catches typos in identifiers, comments, and docs without flagging legitimate jargon. Config in .typos.toml excludes lockfiles, telemetry, and test fixtures, and ignores hex-like identifiers (commit SHAs in CHANGELOG entries, etc).
deptry detects undeclared/unused/transitive imports relative to pyproject.toml — catches the 'imported but never declared' failure mode common when prototypes graduate to library code. vulture finds unreachable functions, classes, and imports. Runs at min-confidence 80 with continue-on-error: it surfaces signal in logs without blocking PRs on false positives.
Runs Sunday 08:00 UTC. mutmut applies small mutations to src/aelfrice and re-runs the test suite — surviving mutants reveal tests that pass without actually asserting the behavior. Report is uploaded as a 30-day artifact rather than a blocking gate; mutation testing on a 200+ test suite is too noisy and slow for PR-time enforcement, but a weekly snapshot is the solo-dev substitute for a second human reviewing test quality.
Hosted repo-activity dashboard at ossinsight.io. Zero install — links out from the README to a public analytics view of contributors, PR velocity, and dependency graph.
Lightweight Nygard-style Architecture Decision Records under docs/adr/. Plain markdown, no published site, no tooling. ADR 0001 records the decision to keep ADRs at all; 0002 captures the two-repo physical separation rationale that was previously only in commit history and CLAUDE.md fragments.
Eight commits adding solo-dev tooling: zizmor (workflow auditor), CodeQL (python+actions SAST), lychee (weekly link-rot), typos (PR spell-check), deptry+vulture (dep/dead-code hygiene), mutmut (weekly mutation tests), OSSInsight badge, ADR directory with first two records. Verification: - new workflow files: 6 (zizmor, codeql, link-check, typos, deadcode, mutation) - existing workflows: untouched - atomic commits: yes (one tool per commit) - secrets required: none (Codecov skipped per user) - README badge: OSSInsight added; build/CI badges unchanged - ADRs: 0001 (record-decisions), 0002 (two-repo-separation) Blockers (require user decision before merge): - [user] Verify CodeQL `actions` language pack is enabled for the repo (default-on for public repos created after 2024; check Settings → Code security) - [user] First mutmut run will be long (~1-4h on full suite). Consider manual workflow_dispatch on a quiet day before relying on the cron - [user] Most actions reference floating major tags (@v4, @v5). Renovate's pinDigests rule should rewrite these to SHAs on next run; if it doesn't, diagnose Renovate before merging more workflows Open questions: - Harden-Runner is in audit mode on new workflows only. Existing workflows (ci.yml, staging-gate.yml, publish.yml, etc.) were not modified — adding Harden-Runner there is a separate follow-up to keep this PR focused Rollback: - git revert 258abda..HEAD (or delete the branch before merge)
Author names (Mattern), acronyms (FPR, PN), code identifiers (gard_NN, c_thr, ba SQL alias), accepted variants (unparseable), and intentional test-fixture misspellings (lisence, fragmints in test_noise_config.py) are not typos. Allowlist them so the typos workflow goes green on the existing tree.
Declare aelfrice and benchmarks as first-party. Ignore DEP002 for pytest/pytest-timeout/pyright/nltk/tiktoken/datasets/huggingface_hub (declared but not imported by src/ — dev/test/bench-only; reorganize into PEP 735 groups as a follow-up). Ignore DEP001 for benchmarks (in-repo top-level package via pytest pythonpath shim, not a PyPI dep). Ignore DEP003 for tomli (Py 3.10 fallback for stdlib tomllib). Lets the deadcode workflow go green on the existing tree.
The bundle already shipped .typos.toml with baseline excludes and aelfrice/aelf identifiers. Move the new extend-words allowlist into that file and drop the duplicate _typos.toml. Single source of truth for typos config.
cb4b23d to
1531a8e
Compare
3 participants
Summary
Adds eight tools to fill the solo-dev tooling gaps surfaced in audit. One commit per tool plus a
gate:summary commit..github/workflows/(broad token scopes, script-injection, unpinned actions) → SARIF.md/.rst; opens issue on failure.typos.toml)pyproject.tomldocs/adr/with template + 0001 (record-decisions) + 0002 (two-repo-separation)Codecov skipped per
[user]decision (no token configured).Trivy skipped — March 2026 supply-chain compromise.
All new workflows include
step-security/harden-runner@v2in audit mode.Test plan
actionslanguage pack initializes (matrix entry succeeds on first run; if it fails the fix is dropping it from the matrix in a follow-up)@v4,@v5,@master) get rewritten to SHAs in a digest-pinning PRFollow-ups
ci/harden-runner-expansionbranch)workflow_dispatchbefore relying on the cronSummary by Sourcery
Add additional CI workflows for code quality, security, and mutation testing, introduce ADR documentation, and expose repository analytics via an OSSInsight badge.
New Features:
Enhancements: