Skip to content

feat(upload): upgrade legacy .doc/.ppt/.xls uploads to OpenXML - #7605

Merged
jbecke merged 1 commit into
claude/fervent-mccarthy-h6zlp1from
claude/fervent-mccarthy-h6zlp1-legacy-office-upgrade
Oct 6, 2026
Merged

jbecke merged 1 commit into
claude/fervent-mccarthy-h6zlp1from
claude/fervent-mccarthy-h6zlp1-legacy-office-upgrade

Conversation

@jbecke

@jbecke jbecke commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Stacked on #7594, which adds the .doc file type. Once that merges, this PR's base becomes main.

Uploaded legacy binary Office files are now converted on the server so they open in Macro's native surfaces:

Upload Becomes Opens in
.ppt .pptx the presentation editor (pptx_engine)
.doc .docx the regular DOCX pipeline, then the Word editor
.xls .xlsx the uploaded-workbook preview; Edit in Macro imports it into a native spreadsheet

This works for single uploads and for files inside uploaded folders.

How it works

  1. The legacy original lands ({owner}/{doc}/{version}).
    • The upload finalizer marks it ready as before, so it stays downloadable.
    • LegacyOfficeUpgrader::request_upgrade then enqueues a convert job that writes {owner}/{doc}/upgraded.{docx,pptx,xlsx}.
    • ConvertQueueMessage gets optional from_file_type/to_file_type, because versioned keys have no extension.
  2. The upgraded object lands (new DocumentKey::UpgradedOffice).
    • apply_upgrade makes it the document's content.
    • pptx/xlsx: a new DocumentInstance (with the upgraded file's sha) becomes the latest version, and the object is copied to its versioned key.
    • docx: a new DocumentBom is created, the object is copied to the DOCX staging bucket, and content is set pending ConvertedPdf. From there docx_unzip_handler treats it exactly like a DOCX upload.
  3. Safety.
    • The file type swap is a compare-and-swap (WHERE LOWER("fileType") = $from), done before the copy, so the copy's own object-created event sees the new type and never requests a second upgrade.
    • A failed copy rolls back the type and the new version.
    • Redelivered events are no-ops.
    • The search upload relay skips the intermediate upgraded.* key.
  4. Convert service.
    • Adds doc→docx, ppt→pptx and xls→xlsx. LibreOfficeKit picks the OOXML export filters from the target format.
    • Upgrades get a 120s timeout.
    • Bug fix: with the Ubuntu 24.2 LOK used here, Writer segfaults while tearing down a document it has already saved. That made a successful conversion report failure. The child now skips teardown and _exits after a successful save. The existing docx→pdf export also goes through this function, and a test covers it.
  5. Web.
    • For a legacy upload less than 10 minutes old, the file view says "Converting … to a PowerPoint presentation…" and still offers Download.
    • It polls metadata, then reopens the split in the right block. Workbooks switch in place.
    • After 3 minutes it falls back to "No preview available".
    • The in-sheet Import now explains that .xls files convert when uploaded.
  6. Infra.
    • Finalizer env: ENVIRONMENT (selects the stack's convert queue), DOCUMENT_STORAGE_BUCKET, DOCX_DOCUMENT_UPLOAD_BUCKET.
    • Finalizer permissions: s3:PutObject on doc storage, s3:PutObject on the docx bucket, sqs:SendMessage on the convert queue.
    • The docx bucket policy now allows the finalizer role.
    • These are Lambda env vars set by Pulumi; the finalizer has no Doppler config bin.

Hexagonal boundary checked.

  • The upgrade policy (which types upgrade, ordering, rollback, idempotency) lives in the domain service crates/documents/src/domain/legacy_office_upgrade.rs, behind LegacyOfficeUpgradeRepoPort and LegacyOfficeUpgradeStoragePort.
  • Postgres and S3/SQS are outbound adapters: pg_document_repo/legacy_upgrade.rs and the finalizer's outbound.rs.
  • No authorization is involved; it is an internal, event-driven use case.

Note: an upgraded .doc still gets a PDF rendition, because every .docx does today. Search, AI document reads and citations read that PDF.

Testing

  • Domain: 18 tests with fake ports, covering request mapping, each target's call order, skips, lost compare-and-swap cleanup, rollback on copy failure, idempotency and bad targets.
  • Repo: 6 sqlx::tests against Postgres: instance/BOM create and delete, compare-and-swap including uppercase stored types, and content.
  • Finalizer: 12 tests for key routing and the convert message and destination.
  • s3_key: upgraded.* build/parse/reject.
  • Convert service:
    • Filter table and file-type resolution.
    • Real LibreOfficeKit conversions of committed legacy fixtures (memo.doc, deck.ppt, budget.xls):
      • .doc → docx lays out to 2+ pages in docx_engine;
      • .ppt → pptx renders all 4 slides in pptx_engine, with table, picture and notes;
      • .xls → xlsx keeps formulas, a cross-sheet formula, the merge, the number format and cached values;
      • unreadable input fails;
      • the upgraded docx/pptx still export to PDF.
    • These are #[ignore] without LibreOffice. Run them with LOK_PATH=/usr/lib/libreoffice/program cargo test -p convert_service -- --ignored; all pass here with LibreOffice 24.2.
  • Engines, run in CI:
    • docx_engine and pptx_engine integration tests on the LibreOffice-upgraded fixtures: open, lay out/render, edit, save, reopen.
    • xlsx-upgraded-legacy.test.ts: the upgraded workbook imports into a native spreadsheet, recalculates to the expected totals after Loro save/reopen, and exports back to Excel.
  • Web: legacy-office.test.ts (11 tests) and UnknownContent.test.tsx; all block-spreadsheet/block-unknown suites (711 tests) pass.
  • Checks:
    • cargo test for documents (400, Postgres + Redis), finalizer, convert_service, s3_key, search_upload_handler, docx_unzip_handler.
    • clippy (no new warnings), SQLX_OFFLINE=true cargo check with the 5 new .sqlx entries, cargo x deps --check.
    • Web: tsc, biome, oxlint, ast-grep.
    • Infra: tsc and biome.
  • Not exercised end to end: the finalizer Lambda and convert service are not in the local stack, and this environment can't reach a deployed backend.

Generated by Claude Code

@jbecke
jbecke requested a review from a team as a code owner October 6, 2026 14:57
@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: macro-inc/macro/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 16a3d038-84a2-4895-a134-8addd3a93fc8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

@jbecke
jbecke force-pushed the claude/fervent-mccarthy-h6zlp1-legacy-office-upgrade branch from 4c51515 to bed8843 Compare October 6, 2026 15:09
@jbecke
jbecke force-pushed the claude/fervent-mccarthy-h6zlp1-legacy-office-upgrade branch from bed8843 to 5ea2c96 Compare October 6, 2026 15:41
Uploaded legacy binary Office files are converted on the server so they
open in the native surfaces: .doc -> .docx (then the DOCX pipeline),
.ppt -> .pptx (presentation editor) and .xls -> .xlsx (uploaded-workbook
preview, importable into a native spreadsheet).

- The upload finalizer asks the convert service to write
  upgraded.{docx,pptx,xlsx} when a legacy original lands, keeping the
  original ready so it stays downloadable if conversion fails.
- When the upgraded object lands, the finalizer makes it the document's
  latest version (a DocumentInstance, or a DOCX BOM staged for the DOCX
  pipeline) and swaps the file type with a compare-and-swap, rolling back
  if the copy fails. Redelivered events are no-ops.
- The convert service supports doc->docx, ppt->pptx and xls->xlsx,
  accepts explicit file types for extensionless versioned keys, gives
  upgrades 120s, and exits its LibreOffice child without teardown, which
  could crash Writer after a successful save.
- The file view shows a converting state for fresh legacy uploads and
  reopens the document in its new block when the upgrade lands.
- Infra: finalizer env, S3/SQS permissions and docx bucket policy.

Tests cover the domain flow, the repository queries, finalizer routing,
key parsing, real LibreOffice conversions (ignored without LOK_PATH), and
the docx/pptx engines and spreadsheet importer on LibreOffice output.
@jbecke
jbecke force-pushed the claude/fervent-mccarthy-h6zlp1-legacy-office-upgrade branch from 5ea2c96 to 0cabada Compare October 6, 2026 16:16
@jbecke
jbecke merged commit beb27e1 into claude/fervent-mccarthy-h6zlp1 Oct 6, 2026
8 checks passed
@jbecke
jbecke deleted the claude/fervent-mccarthy-h6zlp1-legacy-office-upgrade branch October 6, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants