-
Notifications
You must be signed in to change notification settings - Fork 408
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: make frontend normalize line endings to LF (#3903)
To eliminate parsing differences between Windows and other platforms, the frontend now normalizes all CRLF line endings to LF, like [in Rust](rust-lang/rust#62865). Effects: - This makes Lake hashes be faithful to what Lean sees (Lake already normalizes line endings before computing hashes). - Docstrings now have normalized line endings. In particular, this fixes `#guard_msgs` failing multiline tests for Windows users using CRLF. - Now strings don't have different lengths depending on the platform. Before this PR, the following theorem is true for LF and false for CRLF files. ```lean example : " ".length = 1 := rfl ``` Note: the normalization will take `\r\r\n` and turn it into `\r\n`. In the elaborator, we reject loose `\r`'s that appear in whitespace. Rust instead takes the approach of making the normalization routine fail. They do this so that there's no downstream confusion about any `\r\n` that appears. Implementation note: the LSP maintains its own copy of a source file that it updates when edit operations are applied. We are assuming that edit operations never split or join CRLFs. If this assumption is not correct, then the LSP copy of a source file can become slightly out of sync. If this is an issue, there is some discussion [here](#3903 (comment)).
- Loading branch information
Showing
13 changed files
with
137 additions
and
88 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
/-! | ||
# Test `String.crlfToLf` | ||
-/ | ||
|
||
/-! | ||
Leaves single `\n`'s alone. | ||
-/ | ||
/-- info: "hello\nworld" -/ | ||
#guard_msgs in #eval String.crlfToLf "hello\nworld" | ||
|
||
/-! | ||
Turns `\r\n` into `\n`. | ||
-/ | ||
/-- info: "hello\nworld" -/ | ||
#guard_msgs in #eval String.crlfToLf "hello\r\nworld" | ||
|
||
/-! | ||
In a string of `\r...\r\n`, only normalizes the last `\r\n`. | ||
-/ | ||
/-- info: "hello\x0d\nworld" -/ | ||
#guard_msgs in #eval String.crlfToLf "hello\r\r\nworld" | ||
|
||
/-! | ||
Two in a row. | ||
-/ | ||
/-- info: "hello\n\nworld" -/ | ||
#guard_msgs in #eval String.crlfToLf "hello\r\n\r\nworld" | ||
|
||
/-! | ||
Normalizes at the end. | ||
-/ | ||
/-- info: "hello\nworld\n" -/ | ||
#guard_msgs in #eval String.crlfToLf "hello\r\nworld\r\n" | ||
|
||
/-! | ||
Can handle a loose `\r` as the last character. | ||
-/ | ||
/-- info: "hello\nworld\x0d" -/ | ||
#guard_msgs in #eval String.crlfToLf "hello\r\nworld\r" |
Oops, something went wrong.