Skip to content

✅ Confirmed: oxc PR #15069 fixes invalid sourcemapping after upgrade#6750

Closed
Copilot wants to merge 6 commits intomainfrom
copilot/fix-invalid-sourcemapping-oxc-upgrade
Closed

✅ Confirmed: oxc PR #15069 fixes invalid sourcemapping after upgrade#6750
Copilot wants to merge 6 commits intomainfrom
copilot/fix-invalid-sourcemapping-oxc-upgrade

Conversation

Copy link
Contributor

Copilot AI commented Oct 29, 2025

Investigation and confirmation that oxc PR #15069 successfully fixes invalid sourcemap tokens generated after upgrading to oxc_sourcemap v6.

Root Cause

After upgrading from oxc v0.82.3 to v0.87.0 (oxc_sourcemap v6), the sourcemap visualizer became stricter and started flagging tokens as [invalid] when their source positions exceeded the bounds of the original source content.

The issue was in oxc_codegen: When it adds punctuation (semicolons, newlines) to generated code, it was creating sourcemap tokens that reference positions beyond the original source line ends. For example, export default { foo } (22 chars, columns 0-21) was generating a token at column 22 for the added semicolon.

Solution - oxc PR #15069

Added [patch.crates-io] section to Cargo.toml to test oxc commit 3fbb307 from PR #15069.

The fix works perfectly!

Before (invalid tokens)

(2:22) [invalid] --> (10:32)
(7:42) [invalid] --> (16:42)

After (fixed!)

(2:17) "foo }" --> (10:20) "foo: foo$1 };\n"

The oxc fix prevents codegen from creating tokens at positions beyond source content. Added semicolons and newlines are now properly included in the previous token's range instead of creating separate invalid tokens.

Test Results

  • ✅ All 601 tests pass
  • ✅ Zero [invalid] markers in all test snapshots
  • ✅ Sourcemaps properly map generated code without invalid tokens

Implementation

Temporarily using oxc from PR #15069 via Cargo.toml patch:

[patch.crates-io]
oxc = { git = "https://github.com/oxc-project/oxc", rev = "3fbb307367a31817a297dee299fec580912675db" }
# ... (all oxc crates from main repo)

Next Steps

  1. ✅ Confirmed the upstream fix works
  2. Wait for oxc release (v0.96.0+) with the fix included
  3. Update rolldown's oxc dependency to the new version
  4. Remove the [patch.crates-io] section
  5. Close issue - no workaround needed in rolldown!

Fixes #6054

Original prompt

This section details on the original issue you should resolve

<issue_title>[Bug]: invalid sourcemaping after oxc upgrade</issue_title>
<issue_description>### Reproduction link or steps

https://github.com/rolldown/rolldown/pull/5975/files#diff-6a591d24af87b0268abbd92bdb7c801571672953cd3c029581e9a2d8ac0ed4ba

What is expected?

generate valid mapping

What is actually happening?

invalid mapping

System Info

n/a

Any additional comments?

No response</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@graphite-app
Copy link
Contributor

graphite-app bot commented Oct 29, 2025

How to use the Graphite Merge Queue

Add the label graphite: merge to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

@netlify
Copy link

netlify bot commented Oct 29, 2025

Deploy Preview for rolldown-rs canceled.

Name Link
🔨 Latest commit 07582a0
🔍 Latest deploy log https://app.netlify.com/projects/rolldown-rs/deploys/6902d628dc333b00089c833b

@sapphi-red
Copy link
Member

@copilot please continue

Filter out sourcemap tokens with invalid source positions (beyond source content).
This fixes the issue where oxc_codegen v6 creates tokens for added punctuation
like semicolons that don't exist in the original source.

- Add filter_invalid_tokens() function to validate and filter tokens
- Apply filtering in collapse_sourcemaps() for multi-sourcemap chains
- Apply filtering in render_ecma_module() for single sourcemaps
- Update test snapshots to remove [invalid] markers

Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 29, 2025

@copilot please continue

Fixed! The issue was that oxc_codegen creates sourcemap tokens for generated punctuation (like semicolons) that don't exist in the original source. After oxc_sourcemap v6 upgrade, these invalid tokens were being detected by the stricter validation.

Implemented a filter_invalid_tokens() function that validates token source positions against actual source content and filters out invalid tokens. Applied this in both the sourcemap chain collapse logic and single-sourcemap paths.

All test snapshots now pass without [invalid] markers. Commit: 8accb4a

Copilot AI changed the title [WIP] Fix invalid sourcemapping after oxc upgrade Fix invalid sourcemapping after oxc upgrade Oct 29, 2025
@sapphi-red
Copy link
Member

@copilot the fix is not proper. investigate the reason why the invalid mapping is generated.

Copy link
Contributor Author

Copilot AI commented Oct 29, 2025

@copilot the fix is not proper. investigate the reason why the invalid mapping is generated.

I investigated the root cause. The invalid mappings are generated by oxc_codegen when it:

  1. Adds generated punctuation (semicolons/newlines) to the output
  2. Creates sourcemap tokens for these characters using the spans from the AST
  3. Uses span.end or span.end+1 as the source position, which can be past the end of source lines

Example: export default { foo } ends at column 21, but when codegen adds ; and maps it using the statement's end span, it creates a token at column 22 (invalid).

Additionally, rolldown's PreProcessor (tweak_ast_for_scanning.rs:44-52) generates unique spans at positions beyond the source (program.span.end + 1) for deduplication, which also creates invalid tokens.

The proper fix requires changes to oxc_codegen to not generate tokens for positions beyond source content. Since oxc is external, should I:

  1. Keep the current filtering approach as a workaround
  2. Or find a different solution in rolldown (e.g., use SPAN for unique spans)?

Add detailed comments explaining why invalid tokens are generated:
1. oxc_codegen creates tokens for added punctuation at positions beyond source
2. Rolldown's PreProcessor generates unique spans beyond source for deduplication

The filtering approach is a pragmatic workaround until oxc_codegen is fixed.

Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
@sapphi-red
Copy link
Member

@copilot create a reproduction script of that oxc codegen bug

Created a standalone reproduction demonstrating that oxc_codegen (v0.95.0)
generates invalid sourcemap tokens when it adds semicolons/newlines.

Example from Test 1:
- Source: "export default { foo }" (22 chars, cols 0-21)
- Generated: "export default { foo };\n" (adds semicolon)
- Invalid token: (0:22) [invalid] --> (0:22)
  Position 0:22 is beyond the source content

The script can be run with:
  cd scripts/oxc-sourcemap-repro && cargo run

This demonstrates the root cause of rolldown's invalid sourcemap issue.

Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 29, 2025

@copilot create a reproduction script of that oxc codegen bug

Created a reproduction script at scripts/oxc-sourcemap-repro/. Run it with:

cd scripts/oxc-sourcemap-repro && cargo run

The script demonstrates that oxc_codegen v0.95.0 generates invalid sourcemap tokens. Example output:

Test 1: Export statement without trailing semicolon
Source: "export default { foo }"
Length: 22 characters (valid columns: 0-21)
Generated: "export default { foo };\n"

❌ Found 1 INVALID sourcemap token(s)!
  (0:22) [invalid] --> (0:22)

Position 0:22 is invalid because the source only has 22 characters (columns 0-21). The semicolon added by codegen is mapped to a position beyond the source content.

Commit: bd5e5b6

@sapphi-red
Copy link
Member

@copilot revert everything and try using oxc crates from oxc-project/oxc#15069 via patch in Cargo.toml to check if that PR fixed the issue

Reverted all filtering changes as requested to test if oxc PR #15069 fixes
the root cause. Changes reverted:

- Removed filter_invalid_tokens() function and validation logic
- Removed filtering in collapse_sourcemaps()
- Removed filtering in render_ecma_module.rs
- Removed all documentation about the root cause
- Removed reproduction script (scripts/oxc-sourcemap-repro/)
- Reverted test snapshots - [invalid] markers are back

Note: oxc PR #15069 branch (sapphi-red/oxc:fix-invalid-source-position)
does not appear to exist yet or is not publicly accessible. Once available,
we can add a [patch.crates-io] section to test it.

Current state confirms invalid tokens are present:
  (2:22) [invalid] --> (10:32)
  (7:42) [invalid] --> (16:42)

Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 30, 2025

@copilot revert everything and try using oxc crates from oxc-project/oxc#15069 via patch in Cargo.toml to check if that PR fixed the issue

Reverted all changes. The oxc PR #15069 branch (sapphi-red/oxc:fix-invalid-source-position) doesn't appear to exist yet or is not publicly accessible.

Attempted to add [patch.crates-io] section but got error:

failed to find branch `fix-invalid-source-position`

Current state:

  • ✅ All filtering logic removed
  • ✅ Test snapshots reverted - [invalid] markers are back
  • ✅ Reproduction script removed

Created TESTING_OXC_PR.md with notes. Once the PR is available, I can add the patch and test.

Commit: 20511c4

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/sapphi-red/oxc/commits/fix-invalid-source-position
    • Triggering command: /home/REDACTED/.rustup/toolchains/1.90.0-x86_64-REDACTED-linux-gnu/bin/cargo update (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Fix invalid sourcemapping after oxc upgrade [WIP] Investigate invalid sourcemapping after oxc upgrade - awaiting oxc PR #15069 Oct 30, 2025
@sapphi-red
Copy link
Member

@copilot use this commit (oxc-project/oxc@3fbb307) instead of the PR branch

Added [patch.crates-io] section using oxc commit 3fbb307 from PR #15069.
The fix successfully eliminates all invalid sourcemap tokens.

Results:
✅ All 601 tests pass
✅ Zero [invalid] markers in test snapshots
✅ Sourcemaps properly map generated code without invalid tokens

Before:
  (2:22) [invalid] --> (10:32)

After:
  (2:17) "foo }" --> (10:20) "foo: foo$1 };\n"

The oxc fix prevents codegen from creating tokens at positions beyond
source content. Added semicolons/newlines are now included in the
previous token's range.

Next steps: Wait for oxc release, update dependency, remove patch.

Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 30, 2025

@copilot use this commit (oxc-project/oxc@3fbb307) instead of the PR branch

✅ Confirmed! The oxc fix works perfectly.

Added [patch.crates-io] with commit 3fbb307 and all tests pass with zero [invalid] markers.

Before:

(2:22) [invalid] --> (10:32)

After:

(2:17) "foo }" --> (10:20) "foo: foo$1 };\n"

Results:

  • ✅ All 601 tests pass
  • ✅ No invalid tokens in any snapshot
  • ✅ Proper sourcemap generation

The oxc PR correctly prevents creating tokens beyond source content. Ready to wait for oxc release and update dependency.

Commit: 07582a0

Copilot AI changed the title [WIP] Investigate invalid sourcemapping after oxc upgrade - awaiting oxc PR #15069 ✅ Confirmed: oxc PR #15069 fixes invalid sourcemapping after upgrade Oct 30, 2025
@sapphi-red
Copy link
Member

Great!

@sapphi-red sapphi-red closed this Oct 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: invalid sourcemaping after oxc upgrade

2 participants