Skip to content

feat(essay): add accessible standalone score report HTML - #521

Merged
seonghobae merged 28 commits into
mainfrom
feat/essay-score-report-html
Aug 4, 2026
Merged

seonghobae merged 28 commits into
mainfrom
feat/essay-score-report-html

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Buyer-visible gap

Governed essay score reports are currently JSON-only. Buyers need a portable, exact-value, source-text-free audit artifact that can be reviewed without a Python environment while preserving the report's conservative validity boundary.

Reviewed target

  • add render_essay_score_report_html over the existing EssayScoreReport contract;
  • replay the complete governed report before serialization and fail closed on canonical differences;
  • expose exact report, assessment, rubric, task-revision, engine, request, result, observation, criterion, trigger, and evidence identities;
  • provide semantic landmarks, keyboard-accessible exact-value tables, and canonical JSON without scripts or external resources;
  • apply output encoding and a restrictive meta-delivered Content Security Policy as defense in depth;
  • state explicitly that routing status is not validity, fairness, reliability, accessibility conformance, security certification, or deployment authorization;
  • add deterministic 100% statement/branch coverage, WCAG 2.2/CSP3 standards traceability, documentation, and authoritative changelog material.

Architecture constraints

No new scoring schema, scoring arithmetic, provider SDK, raw prompt/response/source text, workflow dependency, or release bump.

Advances #397

Summary by CodeRabbit

  • New Features

    • Added standalone HTML rendering for governed essay score reports.
    • Reports include accessible metadata, criterion outcomes, evidence identities, review routing, provenance, and canonical JSON.
    • Generated files are script-free, source-text-free, UTF-8 encoded, and protected by a restrictive content security policy.
  • Documentation

    • Added usage guidance, privacy and security considerations, accessibility details, and changelog coverage.
  • Bug Fixes

    • Added validation to reject invalid, altered, or incorrectly formatted reports before rendering.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6c1fd08e-5389-40c4-a2ad-31ea67c91464

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds render_essay_score_report_html for replay-validated, accessible standalone essay score report artifacts. The change includes public exports, comprehensive tests, documentation, a changelog entry, and a one-time synchronization workflow.

Changes

Essay report HTML artifact

Layer / File(s) Summary
HTML report rendering
python/fast_mlsirm/scoring/essay/report_html.py
Adds deterministic HTML generation, replay validation, escaping, accessibility markup, CSP metadata, canonical JSON, and UTF-8 file output.
Public API and renderer validation
python/fast_mlsirm/scoring/essay/__init__.py, tests/test_scoring_essay_report_html.py, tests/test_scoring_essay_reporting_exports.py
Exports the renderer and tests its output, metadata, review states, escaping, invalid inputs, and mutation detection.
Documentation and release synchronization
docs/automated_essay_score_reports.md, docs/changelog.d/essay-score-report-html.md, .github/workflows/one-time-essay-report-html-sync.yml
Documents the artifact contract and adds a self-removing workflow to synchronize the contract test and changelog.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant render_essay_score_report_html
  participant EssayScoreReport
  participant HTMLFile
  Caller->>render_essay_score_report_html: Provide report and .html output path
  render_essay_score_report_html->>EssayScoreReport: Replay-validate report
  render_essay_score_report_html->>HTMLFile: Write standalone HTML
  HTMLFile-->>Caller: Return output Path
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the added accessible standalone HTML score report renderer.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/essay-score-report-html

Comment @coderabbitai help to get the list of available commands.

Comment thread .github/workflows/one-time-essay-report-html-sync.yml Fixed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/changelog.d/essay-score-report-html.md`:
- Around line 1-3: Replace the leading “### Added” heading in the changelog
fragment with a repository-compliant level-one title, while preserving the essay
score report entry beneath it; then verify the change with
render_changelog_fragments.py --check.

In `@python/fast_mlsirm/scoring/essay/report_html.py`:
- Around line 293-311: Update render_essay_score_report_html to validate
non-None custom titles before rendering: reject empty or whitespace-only values
with ValueError, while continuing to use _DEFAULT_TITLE when title is None and
preserving nonblank titles unchanged.

In `@tests/test_scoring_essay_report_html.py`:
- Around line 154-155: Update the pytest.raises match pattern in the
render_essay_score_report_html test to use a raw string and escape the literal
period before “html” with \., ensuring the assertion matches the exact “.html”
suffix.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a07495e-7f55-4d02-91d1-508e7c02f1b3

📥 Commits

Reviewing files that changed from the base of the PR and between 567bd6d and 95e4e91.

📒 Files selected for processing (7)
  • .github/workflows/one-time-essay-report-html-sync.yml
  • docs/automated_essay_score_reports.md
  • docs/changelog.d/essay-score-report-html.md
  • python/fast_mlsirm/scoring/essay/__init__.py
  • python/fast_mlsirm/scoring/essay/report_html.py
  • tests/test_scoring_essay_report_html.py
  • tests/test_scoring_essay_reporting_exports.py

Comment thread docs/changelog.d/essay-score-report-html.md Outdated
Comment thread python/fast_mlsirm/scoring/essay/report_html.py
Comment thread tests/test_scoring_essay_report_html.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants