Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(css_parser): fix is at unicode range check #3745

Merged
merged 6 commits into from
Sep 3, 2024

Conversation

suxin2017
Copy link
Contributor

Summary

fix unicode_range check #3743

Test Plan

add snapshot test

@github-actions github-actions bot added A-Parser Area: parser L-CSS Language: CSS labels Aug 31, 2024
@suxin2017 suxin2017 changed the title fix(css_parser): is at unicode range fix(css_parser):fix is at unicode range check Aug 31, 2024
@suxin2017 suxin2017 changed the title fix(css_parser):fix is at unicode range check fix(css_parser): fix is at unicode range check Aug 31, 2024
Copy link

codspeed-hq bot commented Aug 31, 2024

CodSpeed Performance Report

Merging #3745 will not alter performance

Comparing suxin2017:fix-3743 (068b798) with main (7bcda36)

Summary

✅ 99 untouched benchmarks

Comment on lines +14 to +21
const UNICODE: TokenSet<CssSyntaxKind> = token_set![
// u+;
T![+],
// u+000;
CSS_NUMBER_LITERAL,
// u+00ff?;
CSS_DIMENSION_VALUE,
];
Copy link
Member

@Conaclos Conaclos Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dummy question: does not this reveal an issue in our lexer? If I understand correctly our lexer can output three different token sequences that are all Unicode range: id(u) token(+), id(u) number(_) and id(u) dimension(_). Should we output a dedicated token UnicodeRange?

Note: this can be left for a future PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon observing the lexer code, I found that the corresponding T![U+] as start, CSS_UNICODE_RANGE_WILDCARD_LITERAL CSS_UNICODE_CODEPOINT_LITERAL represents a numeric value, but we need to pass a Context to get the corresponding Token.I don't see an example of using context in other is_at methods, I'm not sure it's right to just default to set Unicode_Range_Context

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@togami2864 @denbezrukov any opinion?

Copy link
Member

@Conaclos Conaclos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@suxin2017

Thanks for the fix! I think we can merge the fix and decide later if we need to move some logics to the lexer.

@Conaclos Conaclos merged commit 49e2103 into biomejs:main Sep 3, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Parser Area: parser L-CSS Language: CSS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants