Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(css_parser): fix is at unicode range check #3745

Merged
merged 6 commits into from
Sep 3, 2024
+457 −388
Merged

fix(css_parser): fix is at unicode range check #3745

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
581cca4
done
suxin2017 Aug 31, 2024
57bee95
format
suxin2017 Aug 31, 2024
0c6b78d
comment
suxin2017 Aug 31, 2024
632e1fd
revert quick test
suxin2017 Aug 31, 2024
0c1a716
revert
suxin2017 Aug 31, 2024
068b798
Merge branch 'main' into fix-3743
suxin2017 Sep 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions crates/biome_css_parser/src/syntax/property/unicode_range.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,28 @@
use crate::lexer::{CssLexContext, CssReLexContext};
use crate::parser::CssParser;
use biome_css_syntax::CssSyntaxKind::{
CSS_BOGUS_UNICODE_RANGE_VALUE, CSS_UNICODE_CODEPOINT, CSS_UNICODE_CODEPOINT_LITERAL,
CSS_UNICODE_RANGE, CSS_UNICODE_RANGE_INTERVAL, CSS_UNICODE_RANGE_WILDCARD,
CSS_UNICODE_RANGE_WILDCARD_LITERAL,
self, CSS_BOGUS_UNICODE_RANGE_VALUE, CSS_DIMENSION_VALUE, CSS_NUMBER_LITERAL,
CSS_UNICODE_CODEPOINT, CSS_UNICODE_CODEPOINT_LITERAL, CSS_UNICODE_RANGE,
CSS_UNICODE_RANGE_INTERVAL, CSS_UNICODE_RANGE_WILDCARD, CSS_UNICODE_RANGE_WILDCARD_LITERAL,
};
use biome_css_syntax::{TextRange, T};
use biome_parser::diagnostic::{expected_any, expected_node, ParseDiagnostic};
use biome_parser::parsed_syntax::ParsedSyntax;
use biome_parser::parsed_syntax::ParsedSyntax::{Absent, Present};
use biome_parser::Parser;
use biome_parser::{token_set, Parser, TokenSet};

const UNICODE: TokenSet<CssSyntaxKind> = token_set![
// u+;
T![+],
// u+000;
CSS_NUMBER_LITERAL,
// u+00ff?;
CSS_DIMENSION_VALUE,
];
Comment on lines +14 to +21
Copy link
Member

@Conaclos Conaclos Aug 31, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dummy question: does not this reveal an issue in our lexer? If I understand correctly our lexer can output three different token sequences that are all Unicode range: id(u) token(+), id(u) number(_) and id(u) dimension(_). Should we output a dedicated token UnicodeRange?

Note: this can be left for a future PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon observing the lexer code, I found that the corresponding T![U+] as start, CSS_UNICODE_RANGE_WILDCARD_LITERAL CSS_UNICODE_CODEPOINT_LITERAL represents a numeric value, but we need to pass a Context to get the corresponding Token.I don't see an example of using context in other is_at methods， I'm not sure it's right to just default to set Unicode_Range_Context

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@togami2864 @denbezrukov any opinion?


/// Checks if the parser is positioned to potentially start parsing a Unicode range in CSS, identified by "U" or "u".
pub(crate) fn is_at_unicode_range(p: &mut CssParser) -> bool {
matches!(p.cur_text(), "U" | "u")
matches!(p.cur_text(), "U" | "u") && p.nth_at_ts(1, UNICODE)
}

/// Parses a Unicode range in CSS starting from "U+" or "u+".
Expand Down
2 changes: 2 additions & 0 deletions crates/biome_css_parser/tests/css_test_suite/ok/property/property_generic.css
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
div {
/* Unknown property, always generic */
unknown-property: u;
unknown-property: a value;
unknown-property: one-value;
unknown-property: multiple values;
Expand All @@ -8,6 +9,7 @@ div {
unknown-property: mixed, delimiters / can be, used;

/* Custom property, always generic */
--custom-property: u;
--custom-property: a value;
--custom-property: one-value;
--custom-property: multiple values;
Expand Down
Loading
Loading