Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
385cd74
feat(essay): add accessible standalone score report renderer
seonghobae Aug 4, 2026
0ca0e54
feat(essay): export standalone report renderer
seonghobae Aug 4, 2026
725741b
test(essay): cover standalone score report renderer
seonghobae Aug 4, 2026
15b80c4
test(essay): verify standalone report export
seonghobae Aug 4, 2026
e432180
docs(essay): document accessible standalone report artifact
seonghobae Aug 4, 2026
bde53fe
docs(changelog): add standalone essay report artifact
seonghobae Aug 4, 2026
95e4e91
chore: apply one-time report HTML synchronization
seonghobae Aug 4, 2026
74d09ee
chore: trigger one-time report HTML synchronization
seonghobae Aug 4, 2026
3ffe92b
chore: enable PR-triggered report HTML synchronization
seonghobae Aug 4, 2026
0f98f45
docs(changelog): fix essay report fragment structure
seonghobae Aug 4, 2026
bc9e2cf
test(essay): synchronize report HTML public surface
github-actions[bot] Aug 4, 2026
d1bf739
chore: retry report HTML synchronization
seonghobae Aug 4, 2026
1702e09
chore: remove completed report HTML synchronizer
seonghobae Aug 4, 2026
0681b49
style(essay): remove unused report renderer import
seonghobae Aug 4, 2026
e527ba8
style(essay): normalize report renderer imports
seonghobae Aug 4, 2026
8a01edd
chore: apply essay report HTML review fixes
seonghobae Aug 4, 2026
83fa206
fix(essay): validate standalone report titles
github-actions[bot] Aug 4, 2026
6ee89ba
chore: harden essay report review fixer
seonghobae Aug 4, 2026
399765c
chore: trigger essay report review fixes
seonghobae Aug 4, 2026
58a2e2d
fix(essay): validate standalone report titles
github-actions[bot] Aug 4, 2026
be6cda1
chore: deduplicate essay report title validation
seonghobae Aug 4, 2026
4f64705
fix(essay): deduplicate report title validation
seonghobae Aug 4, 2026
506242d
chore: remove completed title deduplication workflow
seonghobae Aug 4, 2026
0aa2e39
chore: synchronize essay HTML branch with main once
seonghobae Aug 4, 2026
e35a496
merge: synchronize essay HTML report with main
github-actions[bot] Aug 4, 2026
fa78e88
docs(essay): distinguish score and calibration reports
seonghobae Aug 4, 2026
5e97acf
test(essay): keep report validation cases reviewable
seonghobae Aug 4, 2026
0156bbd
test(essay): normalize report renderer validation formatting
seonghobae Aug 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .github/workflows/one-time-essay-report-html-sync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: One-time essay report HTML synchronization

on:
push:
branches:
- feat/essay-score-report-html
paths:
- .github/workflows/one-time-essay-report-html-sync.yml

permissions:
contents: write
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed

concurrency:
group: one-time-essay-report-html-sync-${{ github.ref }}
cancel-in-progress: true

jobs:
synchronize-contracts:
if: github.actor != 'github-actions[bot]'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out exact feature branch
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
ref: ${{ github.ref_name }}
fetch-depth: 0
- name: Synchronize the established public-surface contract
run: |
python - <<'PY'
from pathlib import Path

path = Path("tests/test_scoring_essay_contracts.py")
content = path.read_text(encoding="utf-8")
old = ''' "build_essay_submission",
"score_essay_request",'''
new = ''' "build_essay_submission",
"render_essay_score_report_html",
"score_essay_request",'''
if content.count(old) != 1:
raise RuntimeError("expected exactly one essay public-surface block")
path.write_text(content.replace(old, new), encoding="utf-8")
PY
- name: Render and verify authoritative changelog
run: |
python scripts/render_changelog_fragments.py --update CHANGELOG.md
python scripts/render_changelog_fragments.py --check CHANGELOG.md
- name: Commit synchronization and remove this workflow
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git rm .github/workflows/one-time-essay-report-html-sync.yml
git add tests/test_scoring_essay_contracts.py CHANGELOG.md
git commit -m "test(essay): synchronize report HTML public surface"
git push origin "HEAD:${GITHUB_REF_NAME}"
40 changes: 40 additions & 0 deletions docs/automated_essay_score_reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,40 @@ The report embeds the canonical serialized forms and fingerprints of:
- the complete shared scoring result and criterion observations; and
- transparent review triggers and audit metadata.

## Standalone HTML audit artifact

The same governed report can be rendered as a self-contained, source-text-free
HTML artifact:

```python
from fast_mlsirm.scoring.essay import render_essay_score_report_html

render_essay_score_report_html(
report,
"artifacts/essay_score_report.html",
title="Pilot Essay Score Audit",
)
```

The renderer first rebuilds the report through the governed request, observation,
result, and report factories and rejects canonical replay differences. The HTML
contains exact report, assessment, rubric, task-revision, engine, request, result,
observation, criterion, trigger, and evidence-reference identities. It does not
contain prompt text, response text, or source text.

The artifact uses semantic landmarks, a keyboard-accessible skip link, labelled
exact-value tables, focusable overflow regions, and a focusable canonical JSON
section. It has no script or external resource dependency. A restrictive
meta-delivered Content Security Policy is included as defense in depth; output
encoding and governed content validation remain required because CSP is not a
replacement for either control.

These implementation choices are informed by WCAG 2.2 and the current Content
Security Policy Level 3 working draft. They support audit usability and reduce
content-injection impact, but they do not constitute a blanket conformance or
security certification. Accessibility conformance still requires full-page,
assistive-technology, and human evaluation in the buyer's deployment context.

## Non-suppressible review triggers

The builder derives structural triggers that callers cannot remove:
Expand Down Expand Up @@ -84,3 +118,9 @@ handbook of automated essay evaluation*. Routledge.
Williamson, D. M., Xi, X., & Breyer, F. J. (2012). A framework for evaluation
and use of automated scoring. *Educational Measurement: Issues and Practice,
31*(1), 2–13. https://doi.org/10.1111/j.1745-3992.2011.00223.x

World Wide Web Consortium. (2023, October 5). *Web Content Accessibility
Guidelines (WCAG) 2.2*. https://www.w3.org/TR/WCAG22/

World Wide Web Consortium. (2026, May 5). *Content Security Policy Level 3*
(W3C Working Draft). https://www.w3.org/TR/CSP3/
7 changes: 7 additions & 0 deletions docs/changelog.d/essay-score-report-html.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
### Added

#### Accessible standalone essay score report artifacts
Comment thread
github-actions[bot] marked this conversation as resolved.
Outdated

- Added `render_essay_score_report_html`, which replay-verifies one governed `EssayScoreReport` and emits a deterministic, source-text-free, script-free standalone HTML audit artifact.
- The artifact exposes exact report, assessment, rubric, task-revision, engine, request, result, observation, criterion, trigger, and evidence-reference identities through semantic landmarks, keyboard-accessible exact-value tables, and canonical JSON.
- A restrictive meta-delivered Content Security Policy and output encoding reduce content-injection impact. Review routing remains an audit signal only and does not establish scoring validity, fairness, reliability, interchangeability, accessibility conformance, security certification, or authorization for consequential deployment.
4 changes: 4 additions & 0 deletions python/fast_mlsirm/scoring/essay/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@
from .contracts import build_essay_scoring_request as build_essay_scoring_request
from .contracts import build_essay_submission as build_essay_submission
from .contracts import score_essay_request as score_essay_request
from .report_html import (
render_essay_score_report_html as render_essay_score_report_html,
)
from .reporting import EssayScoreReport as EssayScoreReport
from .reporting import (
MAX_ESSAY_REPORT_REVIEW_TRIGGERS as MAX_ESSAY_REPORT_REVIEW_TRIGGERS,
Expand All @@ -45,5 +48,6 @@
"build_essay_score_report",
"build_essay_scoring_request",
"build_essay_submission",
"render_essay_score_report_html",
"score_essay_request",
]
Loading
Loading