Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,15 @@

### Changed

#### Explicit criterion row headers in governed HTML reports

- Replaced positional first-column row-header inference with an explicit
zero-based `row_header_column` contract in the governed essay score renderer.
- Emit `<th scope="row">` only for criterion identifiers; evidence-reference
values remain data cells, and malformed header/row widths fail closed.
- Added complete-artifact parsing tests that verify table semantics and exact
canonical JSON reconstruction, plus APA 7th WCAG 2.2 doctoring.

#### Release cut 0.7.0

- Project version is bumped to 0.7.0 in `pyproject.toml`,
Expand Down
10 changes: 10 additions & 0 deletions docs/changelog.d/556-explicit-row-headers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Explicit criterion row headers in governed HTML reports

## Changed

- Replaced positional first-column row-header inference with an explicit
zero-based `row_header_column` contract in the governed essay score renderer.
- Emit `<th scope="row">` only for criterion identifiers; evidence-reference
values remain data cells, and malformed header/row widths fail closed.
- Added complete-artifact parsing tests that verify table semantics and exact
canonical JSON reconstruction, plus APA 7th WCAG 2.2 doctoring.
76 changes: 76 additions & 0 deletions docs/doctoring/accessible-html-table-row-headers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Explicit row-header semantics for governed HTML reports

## Decision

The governed essay score renderer identifies a table-body header only when the
caller supplies an explicit zero-based `row_header_column`. The helper does not
infer that the first column is a row header from position alone.

The criterion-outcomes table supplies `row_header_column=0` because each
`criterion_id` identifies the observation row. The source-text-free evidence
table does not declare a row-header column: its first value is repeated
criterion provenance inside a many-row evidence relation rather than a unique
row identity.

Every non-empty row must have exactly the same width as the declared header
axis. Invalid, Boolean, negative, or out-of-range row-header indices fail before
HTML is emitted. This prevents a malformed row from silently shifting a
`scope="row"` association onto the wrong exact value.

## Accessibility contract

The renderer emits:

- `<th scope="col">` for every declared column header;
- `<th scope="row">` only for the explicitly identified criterion field;
- `<td>` for all other body values;
- a semantic `<caption>` for each table;
- a keyboard-focusable overflow region around each table; and
- tabular numeric styling without using presentation as the semantic signal.

This contract implements the programmatic header relationships required by
WCAG 2.2 Success Criterion 1.3.1. W3C guidance recommends `<th>` plus `scope`
when a data table has both row and column headers; it also warns that applying
header semantics where a cell is not actually a header is itself a structural
failure. Therefore row-header markup is a domain decision, not a blanket first-
column transform.

## Verification

Focused tests parse a complete rendered report with Python's standard-library
`HTMLParser` and verify that:

1. every criterion outcome starts with `<th scope="row">`;
2. the remaining criterion cells are `<td>`;
3. every evidence-reference body cell remains `<td>`;
4. the canonical JSON in the same artifact reconstructs the exact report; and
5. invalid row-header indices and header/row width drift fail closed.

The tests operate on the final standalone artifact rather than asserting only a
helper substring, so they cover the production composition path and the exact
criterion/evidence distinction.

## Interpretation boundary

Correct table markup improves programmatic navigation but does not by itself
establish WCAG conformance for the full product, assistive-technology
interoperability across every user agent, psychometric validity, fairness,
reliability, or authorization for consequential scoring. Those claims require
separate evidence.

## References

World Wide Web Consortium. (2023). *Web Content Accessibility Guidelines
(WCAG) 2.2*. https://www.w3.org/TR/WCAG22/

World Wide Web Consortium, Web Accessibility Initiative. (2019, July 27).
*Tables with two headers*. https://www.w3.org/WAI/tutorials/tables/two-headers/

World Wide Web Consortium, Web Accessibility Initiative. (2026, January 26).
*F91: Failure of Success Criterion 1.3.1 for not correctly marking up table
headers*. https://www.w3.org/WAI/WCAG22/Techniques/failures/F91

World Wide Web Consortium, Web Accessibility Initiative. (2026, January 26).
*F46: Failure of Success Criterion 1.3.1 due to using th elements, caption
elements, or non-empty summary attributes in layout tables*.
https://www.w3.org/WAI/WCAG22/Techniques/failures/F46
34 changes: 30 additions & 4 deletions python/fast_mlsirm/scoring/essay/report_html.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,15 +81,39 @@ def _table(
headers: tuple[str, ...],
rows: tuple[tuple[object | None, ...], ...],
empty_message: str,
row_header_column: int | None = None,
) -> str:
"""Render an accessible exact-value table or one explicit empty state."""
"""Render one exact-value table with an explicitly identified row-header column.

``row_header_column`` is zero-based and must name a real header. Leaving it
as ``None`` emits only data cells in ``tbody``. Every non-empty row must
match the declared header width so semantic associations cannot drift from
the serialized values.
"""
if row_header_column is not None and (
isinstance(row_header_column, bool)
or not isinstance(row_header_column, int)
or row_header_column < 0
or row_header_column >= len(headers)
):
raise ValueError("row_header_column must identify an existing table header")
if not rows:
return f'<div class="empty-state" role="status">{escape(empty_message)}</div>'
if not headers:
raise ValueError("table headers must not be empty when rows are present")
if any(len(row) != len(headers) for row in rows):
raise ValueError("every table row must match the declared header width")

heading = "".join(f'<th scope="col">{escape(header)}</th>' for header in headers)
body = []
for row in rows:
cells = "".join(f"<td>{_display(value)}</td>" for value in row)
body.append(f"<tr>{cells}</tr>")
cells = []
for column_index, value in enumerate(row):
if column_index == row_header_column:
cells.append(f'<th scope="row">{_display(value)}</th>')
else:
cells.append(f"<td>{_display(value)}</td>")
body.append(f"<tr>{''.join(cells)}</tr>")
return "\n".join(
(
'<div class="table-scroll" tabindex="0" role="region" '
Expand Down Expand Up @@ -183,7 +207,8 @@ def _css() -> str:
.table-scroll:focus-visible, pre:focus-visible { outline: 3px solid Highlight; outline-offset: 3px; }
table { width: 100%; border-collapse: collapse; }
caption { text-align: left; font-weight: 700; margin-bottom: 8px; }
th, td { padding: 10px; border: 1px solid GrayText; text-align: left; vertical-align: top; overflow-wrap: anywhere; font-variant-numeric: tabular-nums; }
thead th, tbody th, td { padding: 10px; border: 1px solid GrayText; text-align: left; vertical-align: top; overflow-wrap: anywhere; font-variant-numeric: tabular-nums; }
tbody th { font-weight: normal; }
tbody tr { transition: background-color 0.15s ease-in-out; }
tbody tr:hover { background-color: rgba(128, 128, 128, 0.15); }
code, pre { font-family: ui-monospace, monospace; }
Expand Down Expand Up @@ -240,6 +265,7 @@ def _render_html(report: EssayScoreReport, title: str) -> str:
),
rows=_criterion_rows(report),
empty_message="No criterion observations are available.",
row_header_column=0,
)
evidence = _table(
caption="Source-text-free evidence references",
Expand Down
173 changes: 173 additions & 0 deletions tests/test_scoring_essay_report_html_row_headers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
"""Regression tests for explicit HTML table row-header semantics."""

from __future__ import annotations

import json
import runpy
from html.parser import HTMLParser
from pathlib import Path

import pytest

import fast_mlsirm.scoring.essay.report_html as report_html
from fast_mlsirm.scoring.essay import (
build_essay_score_report,
render_essay_score_report_html,
)

_REPORT_FIXTURES = runpy.run_path(
str(Path(__file__).with_name("test_scoring_essay_reporting.py"))
)
essay_request = _REPORT_FIXTURES["essay_request"]
result_bundle = _REPORT_FIXTURES["result_bundle"]


class _SemanticReportParser(HTMLParser):
"""Collect table cell semantics and canonical JSON from a complete report."""

def __init__(self) -> None:
super().__init__(convert_charrefs=True)
self.tables: dict[str, list[list[tuple[str, str | None, str]]]] = {}
self.canonical_json = ""
self._caption_parts: list[str] | None = None
self._table_caption: str | None = None
self._in_body = False
self._row: list[tuple[str, str | None, str]] | None = None
self._cell: tuple[str, str | None, list[str]] | None = None
self._in_canonical_json = False

def handle_starttag(
self,
tag: str,
attrs: list[tuple[str, str | None]],
) -> None:
attributes = dict(attrs)
if tag == "caption":
self._caption_parts = []
elif tag == "tbody":
self._in_body = True
elif tag == "tr" and self._in_body:
self._row = []
elif tag in {"th", "td"} and self._row is not None:
self._cell = (tag, attributes.get("scope"), [])
elif tag == "pre" and attributes.get("aria-label") == (
"Canonical essay score report JSON"
):
self._in_canonical_json = True

def handle_data(self, data: str) -> None:
if self._caption_parts is not None:
self._caption_parts.append(data)
if self._cell is not None:
self._cell[2].append(data)
if self._in_canonical_json:
self.canonical_json += data

def handle_endtag(self, tag: str) -> None:
if tag == "caption" and self._caption_parts is not None:
self._table_caption = "".join(self._caption_parts)
self.tables[self._table_caption] = []
self._caption_parts = None
elif tag in {"th", "td"} and self._cell is not None:
cell_tag, scope, parts = self._cell
assert self._row is not None
self._row.append((cell_tag, scope, "".join(parts)))
self._cell = None
elif tag == "tr" and self._row is not None:
assert self._table_caption is not None
self.tables[self._table_caption].append(self._row)
self._row = None
elif tag == "tbody":
self._in_body = False
elif tag == "table":
self._table_caption = None
elif tag == "pre" and self._in_canonical_json:
self._in_canonical_json = False


def _complete_report():
"""Return a realistic deterministic report with criterion and evidence rows."""
request = essay_request()
_engine, descriptor, result = result_bundle(request)
return build_essay_score_report(
report_id="semantic_table_report",
request=request,
result=result,
engine=descriptor,
metadata={"workflow_stage": "pilot_review"},
)


def test_complete_report_marks_only_criterion_identifiers_as_row_headers(
tmp_path: Path,
) -> None:
"""Full rendered output scopes criterion IDs without relabeling evidence data."""
report = _complete_report()
output = tmp_path / "semantic-table-report.html"
render_essay_score_report_html(report, output)

parser = _SemanticReportParser()
parser.feed(output.read_text(encoding="utf-8"))
parser.close()

criterion_rows = parser.tables["Criterion-level scoring outcomes"]
evidence_rows = parser.tables["Source-text-free evidence references"]
assert criterion_rows
assert evidence_rows
assert all(row[0][:2] == ("th", "row") for row in criterion_rows)
assert all(
cell[:2] == ("td", None)
for row in criterion_rows
for cell in row[1:]
)
assert all(
cell[:2] == ("td", None)
for row in evidence_rows
for cell in row
)
assert json.loads(parser.canonical_json) == report.to_dict()


def test_table_requires_explicit_valid_row_header_column() -> None:
"""The helper scopes exactly the declared column and rejects ambiguous indices."""
rendered = report_html._table(
caption="Explicit row identity",
headers=("Value", "Identifier", "Status"),
rows=((1, "criterion_a", "observed"),),
empty_message="No values.",
row_header_column=1,
)
assert (
"<tbody><tr><td>1</td><th scope=\"row\">criterion_a</th>"
"<td>observed</td></tr></tbody>"
) in rendered

for invalid_column in (True, -1, 3, "0"):
with pytest.raises(ValueError, match="existing table header"):
report_html._table(
caption="Invalid row identity",
headers=("Identifier",),
rows=(("criterion_a",),),
empty_message="No values.",
row_header_column=invalid_column, # type: ignore[arg-type]
)


def test_table_rejects_header_row_width_drift() -> None:
"""Malformed row widths cannot produce misleading header associations."""
with pytest.raises(ValueError, match="match the declared header width"):
report_html._table(
caption="Mismatched evidence",
headers=("Identifier", "Status"),
rows=(("criterion_a",),),
empty_message="No values.",
row_header_column=0,
)

with pytest.raises(ValueError, match="headers must not be empty"):
report_html._table(
caption="Headerless evidence",
headers=(),
rows=(("criterion_a",),),
empty_message="No values.",
)
Loading