feat(wal): add Wal::append + frame format §6.2.2 (PR-M5)

[jensholdgaard]

What

Second WAL implementation slice — Wal::append + the §6.2.2 frame format. Builds on PR-M4 (Wal::open + segment header).

• New crates/ourios-wal/src/frame.rs — pub(crate) boundary between bytes-on-disk and (FrameKind, payload) tuples per RFC 0008 §6.2.2:
  • FRAME_HEADER_LEN = 12 (4 B len LE u32 + 1 B kind + 3 B _pad + 4 B crc32 LE u32).
  • write_frame — single-buffer header build, then write_all(header) + write_all(payload).
  • read_frame — validates len ≤ MAX_FRAME_BYTES, kind ∈ {0x01, 0x02}, _pad == [0, 0, 0], CRC32-C match.
  • FrameError::{CrcMismatch, UnknownKind, NonZeroPad, OversizeLen, Io} with Display + Error impls — one variant per RFC0008.5 sub-case.
• Wal::append real impl in lib.rs:
  • Up-front payload.len() > MAX_FRAME_BYTES check → AppendError::TooLarge { len, limit } before any bytes are written.
  • Records the segment file's pre-write metadata().len() as the frame's start byte (not stream_position — O_APPEND doesn't guarantee cursor synchronisation on all platforms; same lesson as PR-M4 §1.6.3).
  • Returns WalOffset { segment: current_segment_uuid, byte: pre_write_len } pointing at the start of the new frame.

Test coverage

9 colocated unit tests in src/frame.rs:

• frame_byte_layout_matches_rfc_6_2_2 — pins exact 12 B header bytes.
• frame_round_trips_through_write_then_read — both kinds × three payload sizes.
• empty_payload_frame_is_header_only — len = 0 edge.
• read_frame_rejects_crc_mismatch — RFC0008.5 sub-case 1.
• read_frame_rejects_unknown_kind — RFC0008.5 sub-case 2 (hand-crafted bytes with CRC that matches 0xFE so the kind check fires first).
• read_frame_rejects_non_zero_pad — RFC0008.5 sub-case 3.
• read_frame_rejects_oversize_len — RFC0008.5 sub-case 4; checks length is rejected before any payload allocation.
• read_frame_rejects_truncated_header / _truncated_payload — RFC0008.4 / 0008.5 disambiguation (variant-level; recovery driver disambiguates by segment position in a future slice).

4 integration tests in tests/append.rs:

• one_append_writes_one_frame_after_the_segment_header — pins on-disk layout (24 B header + 12 B frame header + payload, no double-write), pins WalOffset.byte == 24 and .segment == file stem UUID.
• consecutive_appends_pack_tight_with_monotonic_offsets — second offset = first + 12 + first_payload.len; tight packing; both in the same segment.
• max_frame_bytes_is_accepted_one_more_is_rejected — boundary at MAX_FRAME_BYTES + the TooLarge { len, limit } arm.
• append_after_reopen_extends_the_existing_segment — PR-M4 reopen contract preserved across an append.

Workspace totals: 321 passed / 0 failed / 44 ignored (was 308 / 44 on main — +13 live).

RFC0008.X status

The §5 RFC0008.X integration tests stay #[ignore]'d this slice — they need sync (RFC0008.1, 0008.8, 0008.9) or replay (RFC0008.4, 0008.5). The corruption sub-cases that drive RFC0008.5's audit-event reasons are covered at the function-level in the colocated frame.rs tests — same pattern as PR-M4's segment-header colocated tests.

Invariants touched

• §3.4 (WAL-before-ack) — append is the first half of the contract; the durability half lands with sync in the next slice. append returning Ok(offset) still means "frame in OS page cache", not "acked-durable". Receiver MUST NOT ack until sync(offset) returns.
• §6.2.2 frame format — pinned byte-for-byte by the colocated frame_byte_layout_matches_rfc_6_2_2 test.
• §6.9 invariants — MAX_FRAME_BYTES enforced at append time before any I/O.

CLAUDE.md §6.6

• cargo fmt --all --check — clean.
• cargo clippy --all-targets --all-features -- -D warnings — clean.
• cargo test --all-features — 321 passed / 0 failed / 44 ignored.
• mdbook build — clean.

Out of scope (next slices)

• Wal::sync — fdatasync (or F_FULLFSYNC on macOS), the durability gate the receiver acks against.
• Wal::replay — segment scan + frame-by-frame walk; lights up RFC0008.4 + RFC0008.5 integration tests.
• Rotation, checkpoint, recovery from torn writes.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Implemented on-disk WAL frame format with per-frame CRC validation and strict header checks.
  • WAL append now returns precise post-append byte offsets and truncates partial writes on failure.

• Bug Fixes

  • Rejects oversized or malformed frames and surfaces clear error outcomes for corruption, unknown kinds, non-zero padding, oversize lengths, and truncated IO.

• Tests

  • Added unit and integration tests covering write→read round-trips, CRC/validation failures, truncation behavior, and append offset/packing semantics.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 055da280-ffc3-4d4d-9e61-55b88679c80b

📥 Commits

Reviewing files that changed from the base of the PR and between 391d14e and 9c04ed8.

📒 Files selected for processing (3)

• crates/ourios-wal/src/frame.rs
• crates/ourios-wal/src/lib.rs
• crates/ourios-wal/tests/append.rs

🚧 Files skipped from review as they are similar to previous changes (2)

• crates/ourios-wal/tests/append.rs
• crates/ourios-wal/src/frame.rs

---

📝 Walkthrough

Walkthrough

Implements the RFC 0008 §6.2.2 WAL frame byte format with CRC32-C validation and read/write helpers, integrates frame serialization into Wal::append (with pre-write length, rollback on failure, and post-append offset), and adds unit and integration tests validating layout, validation failures, and append offsets.

Changes

WAL Frame Format and Append Implementation

Layer / File(s)	Summary
Frame docs and error types crates/ourios-wal/src/frame.rs	Adds RFC docs, FrameError with variants for CRC mismatch, unknown kind, non-zero pad, oversize length, and I/O; implements Display/Error.
Frame serialization (write_frame) crates/ourios-wal/src/frame.rs	Implements write_frame that writes a 12-byte header (u32 LE len, kind, 3 pad bytes, u32 LE CRC32-C over kind
Frame parsing and validation (read_frame) crates/ourios-wal/src/frame.rs	Implements read_frame that reads and validates header, rejects unknown kind and non-zero pad, enforces len ≤ MAX_FRAME_BYTES before allocation, reads payload, verifies CRC32-C, and returns (FrameKind, Vec<u8>) or FrameError.
Frame unit tests crates/ourios-wal/src/frame.rs	Unit tests pin exact header layout and CRC behavior, exercise round-trips and failure modes (CRC mismatch, unknown kind, non-zero pad, oversize len, truncated I/O).
Wal::append integration and docs crates/ourios-wal/src/lib.rs	Adds pub(crate) mod frame; and documents O_APPEND/WalOffset semantics. Implements Wal::append: validates payload size, reads pre-write segment length, writes via frame::write_frame, truncates to pre-write length on write error, returns post-append WalOffset.
Integration tests for append behavior crates/ourios-wal/tests/append.rs	Integration tests and helpers validating single append byte layout, consecutive tight appends with monotonic offsets, MAX_FRAME_BYTES boundary acceptance/rejection, append-after-reopen continuity, and file helpers to read segment contents.

Sequence Diagram(s)

sequenceDiagram
  participant ComponentA
  participant ComponentB
  ComponentA->>ComponentB: observable interaction

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• jensholdgaard/ourios#66: Introduces the stubbed Wal::append API and shared WAL scaffolding that this PR implements.
• jensholdgaard/ourios#65: RFC/frame-format acceptance criteria referenced by the frame validation and test coverage.
• jensholdgaard/ourios#68: Adds segment header/opening machinery and Wal fields that append uses.

Poem

🐰 I wrote twelve bytes, neat and small,

kind, pad, and CRC to guard them all.
Appends land tidy at the file's end,
offsets climb true, no gaps to mend.
A rabbit's nibble of WAL, well penned.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding Wal::append and implementing the frame format per RFC §6.2.2, with the PR identifier.
Description check	✅ Passed	The description covers all required template sections: comprehensive Summary, Related links (RFC 0008), and complete Checklist with all items confirmed as done.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/wal-append-and-frame-format

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Implements the next WAL slice by adding frame (de)serialization per RFC 0008 §6.2.2 and wiring Wal::append to write frames and return a WalOffset for the newly appended record.

Changes:

• Add frame module implementing the §6.2.2 frame header/payload layout with CRC32-C validation and structured read errors.
• Implement Wal::append to enforce MAX_FRAME_BYTES, write frames to the current segment, and return a (segment_uuid, byte_offset) WalOffset.
• Add integration tests that pin on-disk layout/offset behavior for append.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
crates/ourios-wal/src/lib.rs	Implements Wal::append and introduces frame module usage + updated offset semantics docs.
crates/ourios-wal/src/frame.rs	Adds frame format implementation (write/read + CRC checks) and colocated unit tests.
crates/ourios-wal/tests/append.rs	Adds integration tests asserting append layout, offsets, and reopen behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.