✅ Confirmed: oxc PR #15069 fixes invalid sourcemapping after upgrade#6750
✅ Confirmed: oxc PR #15069 fixes invalid sourcemapping after upgrade#6750
Conversation
How to use the Graphite Merge QueueAdd the label graphite: merge to this PR to add it to the merge queue. You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. |
✅ Deploy Preview for rolldown-rs canceled.
|
|
@copilot please continue |
Filter out sourcemap tokens with invalid source positions (beyond source content). This fixes the issue where oxc_codegen v6 creates tokens for added punctuation like semicolons that don't exist in the original source. - Add filter_invalid_tokens() function to validate and filter tokens - Apply filtering in collapse_sourcemaps() for multi-sourcemap chains - Apply filtering in render_ecma_module() for single sourcemaps - Update test snapshots to remove [invalid] markers Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Fixed! The issue was that oxc_codegen creates sourcemap tokens for generated punctuation (like semicolons) that don't exist in the original source. After oxc_sourcemap v6 upgrade, these invalid tokens were being detected by the stricter validation. Implemented a All test snapshots now pass without |
|
@copilot the fix is not proper. investigate the reason why the invalid mapping is generated. |
I investigated the root cause. The invalid mappings are generated by oxc_codegen when it:
Example: Additionally, rolldown's The proper fix requires changes to oxc_codegen to not generate tokens for positions beyond source content. Since oxc is external, should I:
|
Add detailed comments explaining why invalid tokens are generated: 1. oxc_codegen creates tokens for added punctuation at positions beyond source 2. Rolldown's PreProcessor generates unique spans beyond source for deduplication The filtering approach is a pragmatic workaround until oxc_codegen is fixed. Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
|
@copilot create a reproduction script of that oxc codegen bug |
Created a standalone reproduction demonstrating that oxc_codegen (v0.95.0)
generates invalid sourcemap tokens when it adds semicolons/newlines.
Example from Test 1:
- Source: "export default { foo }" (22 chars, cols 0-21)
- Generated: "export default { foo };\n" (adds semicolon)
- Invalid token: (0:22) [invalid] --> (0:22)
Position 0:22 is beyond the source content
The script can be run with:
cd scripts/oxc-sourcemap-repro && cargo run
This demonstrates the root cause of rolldown's invalid sourcemap issue.
Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Created a reproduction script at cd scripts/oxc-sourcemap-repro && cargo runThe script demonstrates that oxc_codegen v0.95.0 generates invalid sourcemap tokens. Example output: Position 0:22 is invalid because the source only has 22 characters (columns 0-21). The semicolon added by codegen is mapped to a position beyond the source content. Commit: bd5e5b6 |
|
@copilot revert everything and try using oxc crates from oxc-project/oxc#15069 via |
Reverted all filtering changes as requested to test if oxc PR #15069 fixes the root cause. Changes reverted: - Removed filter_invalid_tokens() function and validation logic - Removed filtering in collapse_sourcemaps() - Removed filtering in render_ecma_module.rs - Removed all documentation about the root cause - Removed reproduction script (scripts/oxc-sourcemap-repro/) - Reverted test snapshots - [invalid] markers are back Note: oxc PR #15069 branch (sapphi-red/oxc:fix-invalid-source-position) does not appear to exist yet or is not publicly accessible. Once available, we can add a [patch.crates-io] section to test it. Current state confirms invalid tokens are present: (2:22) [invalid] --> (10:32) (7:42) [invalid] --> (16:42) Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
Reverted all changes. The oxc PR #15069 branch ( Attempted to add Current state:
Created TESTING_OXC_PR.md with notes. Once the PR is available, I can add the patch and test. Commit: 20511c4 Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
|
@copilot use this commit (oxc-project/oxc@3fbb307) instead of the PR branch |
Added [patch.crates-io] section using oxc commit 3fbb307 from PR #15069. The fix successfully eliminates all invalid sourcemap tokens. Results: ✅ All 601 tests pass ✅ Zero [invalid] markers in test snapshots ✅ Sourcemaps properly map generated code without invalid tokens Before: (2:22) [invalid] --> (10:32) After: (2:17) "foo }" --> (10:20) "foo: foo$1 };\n" The oxc fix prevents codegen from creating tokens at positions beyond source content. Added semicolons/newlines are now included in the previous token's range. Next steps: Wait for oxc release, update dependency, remove patch. Co-authored-by: sapphi-red <49056869+sapphi-red@users.noreply.github.com>
✅ Confirmed! The oxc fix works perfectly. Added Before: After: Results:
The oxc PR correctly prevents creating tokens beyond source content. Ready to wait for oxc release and update dependency. Commit: 07582a0 |
|
Great! |
Investigation and confirmation that oxc PR #15069 successfully fixes invalid sourcemap tokens generated after upgrading to oxc_sourcemap v6.
Root Cause
After upgrading from oxc v0.82.3 to v0.87.0 (oxc_sourcemap v6), the sourcemap visualizer became stricter and started flagging tokens as
[invalid]when their source positions exceeded the bounds of the original source content.The issue was in oxc_codegen: When it adds punctuation (semicolons, newlines) to generated code, it was creating sourcemap tokens that reference positions beyond the original source line ends. For example,
export default { foo }(22 chars, columns 0-21) was generating a token at column 22 for the added semicolon.Solution - oxc PR #15069
Added
[patch.crates-io]section to Cargo.toml to test oxc commit 3fbb307 from PR #15069.The fix works perfectly! ✅
Before (invalid tokens)
After (fixed!)
The oxc fix prevents codegen from creating tokens at positions beyond source content. Added semicolons and newlines are now properly included in the previous token's range instead of creating separate invalid tokens.
Test Results
[invalid]markers in all test snapshotsImplementation
Temporarily using oxc from PR #15069 via Cargo.toml patch:
Next Steps
[patch.crates-io]sectionFixes #6054
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.