Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(grit): parse Grit literal snippets #3051

Merged
merged 2 commits into from
Jun 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions crates/biome_grit_patterns/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,12 @@ biome_console = { workspace = true }
biome_diagnostics = { workspace = true }
biome_grit_parser = { workspace = true }
biome_grit_syntax = { workspace = true }
biome_js_parser = { workspace = true }
biome_js_syntax = { workspace = true }
biome_parser = { workspace = true }
biome_rowan = { workspace = true }
grit-pattern-matcher = { version = "0.2" }
grit-util = { version = "0.2" }
grit-pattern-matcher = { version = "0.3" }
grit-util = { version = "0.3" }
im = { version = "15.1.0" }
rustc-hash = { workspace = true }
serde = { workspace = true, features = ["derive"] }
Expand Down
7 changes: 7 additions & 0 deletions crates/biome_grit_patterns/src/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,13 @@ pub enum CompileError {

/// A pattern is required to compile a Grit query.
MissingPattern,

/// Bracketed metavariables are only allowed on the right-hand side of
/// rewrite.
InvalidBracketedMetavariable,

/// Unknown variable.
UnknownVariable(String),
}

impl Diagnostic for CompileError {}
Expand Down
69 changes: 69 additions & 0 deletions crates/biome_grit_patterns/src/grit_analysis_ext.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
use biome_diagnostics::{Diagnostic, PrintDescription, Severity};
use grit_util::{AnalysisLog, Position, Range};
use std::path::{Path, PathBuf};

pub trait GritAnalysisExt {
fn to_log(&self, path: Option<&Path>) -> AnalysisLog;
}

impl<T> GritAnalysisExt for T
where
T: Diagnostic,
{
fn to_log(&self, path: Option<&Path>) -> AnalysisLog {
let location = self.location();
let source = location.source_code;
let range =
match (location.span, source) {
(Some(range), Some(source)) => source.text[..range.start().into()]
.lines()
.enumerate()
.last()
.map(|(i, line)| {
let start = Position {
line: (i + 1) as u32,
column: line.len() as u32,
};
let end = source.text[range].lines().enumerate().last().map_or(
start,
|(j, line)| Position {
line: start.line + j as u32,
column: if j == 0 {
start.column + line.len() as u32
} else {
line.len() as u32
},
},
);
Range {
start,
end,
start_byte: range.start().into(),
end_byte: range.end().into(),
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about the intention of this code, but we have a utility to extract columns and rows from the source code, I suggest to look here and see how we use it:

let source = SourceFile::new(source_code);
let start = source.location(span.start())?;
let end = source.location(span.end())?;

}),
_ => None,
};

AnalysisLog {
engine_id: Some("biome".to_owned()),
file: path.map(Path::to_path_buf).or_else(|| {
location
.resource
.and_then(|r| r.as_file().map(PathBuf::from))
}),
level: Some(match self.severity() {
Severity::Hint => 1,
Severity::Information => 2,
Severity::Warning => 3,
Severity::Error => 4,
Severity::Fatal => 5,
}),
message: PrintDescription(self).to_string(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an utility called markup_to_string that you might want to look at:

let text = markup_to_string(markup! {
{PrintDiagnostic::verbose(&error)}
});

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately that one is only available in the biome_cli crate. I'll see how far we get with the description and range. If necessary, I'll try to move the util.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not exactly that one, but we have it

pub fn print_diagnostic_to_string(diagnostic: &Error) -> String {
let mut buffer = termcolor::Buffer::no_color();
Formatter::new(&mut Termcolor(&mut buffer))
.write_markup(markup! {
{PrintDiagnostic::verbose(diagnostic)}
})
.expect("failed to emit diagnostic");
let mut content = String::new();
writeln!(
content,
"{}",
std::str::from_utf8(buffer.as_slice()).expect("non utf8 in error buffer")
)
.unwrap();
content
}

position: range.as_ref().map(|r| r.start),
range,
syntax_tree: None,
source: source.map(|s| s.text.to_owned()),
}
}
}
11 changes: 10 additions & 1 deletion crates/biome_grit_patterns/src/grit_context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ impl QueryContext for GritQueryContext {
type ResolvedPattern<'a> = GritResolvedPattern;
type Language<'a> = GritTargetLanguage;
type File<'a> = GritFile;
type Tree = GritTree;
type Tree<'a> = GritTree;
}

#[derive(Debug)]
Expand Down Expand Up @@ -81,4 +81,13 @@ impl<'a> ExecContext<'a, GritQueryContext> for GritExecContext {
fn name(&self) -> Option<&str> {
todo!()
}

fn load_file(
&self,
_file: &<GritQueryContext as QueryContext>::File<'a>,
_state: &mut State<'a, GritQueryContext>,
_logs: &mut AnalysisLogs,
) -> Result<bool> {
todo!()
}
}
53 changes: 53 additions & 0 deletions crates/biome_grit_patterns/src/grit_js_parser.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
use crate::{grit_analysis_ext::GritAnalysisExt, grit_tree::GritTree};
use biome_js_parser::{parse, JsParserOptions};
use biome_js_syntax::JsFileSource;
use grit_util::{AnalysisLogs, FileOrigin, Parser, SnippetTree};
use std::path::Path;

pub struct GritJsParser;

impl Parser for GritJsParser {
type Tree = GritTree;

fn parse_file(
&mut self,
body: &str,
path: Option<&Path>,
logs: &mut AnalysisLogs,
_old_tree: FileOrigin<'_, GritTree>,
) -> Option<GritTree> {
let parse_result = parse(body, JsFileSource::tsx(), JsParserOptions::default());

for diagnostic in parse_result.diagnostics() {
logs.push(diagnostic.to_log(path));
}

Some(GritTree::new(parse_result.syntax().into()))
}

fn parse_snippet(
&mut self,
prefix: &'static str,
source: &str,
postfix: &'static str,
) -> SnippetTree<GritTree> {
let context = format!("{prefix}{source}{postfix}");

let len = if cfg!(target_arch = "wasm32") {
|src: &str| src.chars().count() as u32
} else {
|src: &str| src.len() as u32
};

let parse_result = parse(&context, JsFileSource::tsx(), JsParserOptions::default());

SnippetTree {
tree: GritTree::new(parse_result.syntax().into()),
source: source.to_owned(),
prefix,
postfix,
snippet_start: (len(prefix) + len(source) - len(source.trim_start())),
snippet_end: (len(prefix) + len(source.trim_end())),
}
}
}
6 changes: 0 additions & 6 deletions crates/biome_grit_patterns/src/grit_node.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,6 @@ impl GritAstNode for GritNode {
}
}

fn full_source(&self) -> &str {
// This should not be a problem anytime soon, though we may want to
// reconsider when we implement rewrites.
unimplemented!("Full source of file not available")
}

fn walk(&self) -> impl AstCursor<Node = Self> {
GritNodeCursor::new(self)
}
Expand Down
2 changes: 2 additions & 0 deletions crates/biome_grit_patterns/src/grit_node_patterns.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ use grit_util::AnalysisLogs;
pub(crate) struct GritNodePattern;

impl AstNodePattern<GritQueryContext> for GritNodePattern {
const INCLUDES_TRIVIA: bool = true;

fn children(&self) -> Vec<PatternOrPredicate<GritQueryContext>> {
todo!()
}
Expand Down
7 changes: 5 additions & 2 deletions crates/biome_grit_patterns/src/grit_query.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use crate::variables::{VarRegistry, VariableLocations};
use crate::CompileError;
use anyhow::Result;
use biome_grit_syntax::{GritRoot, GritRootExt};
use grit_pattern_matcher::pattern::{Matcher, Pattern, State};
use grit_pattern_matcher::pattern::{FileRegistry, Matcher, Pattern, State};
use std::collections::BTreeMap;

/// Represents a top-level Grit query.
Expand All @@ -32,7 +32,10 @@ impl GritQuery {

let binding = GritResolvedPattern;
let context = GritExecContext;
let mut state = State::new(var_registry.into(), Vec::new());
let mut state = State::new(
var_registry.into(),
FileRegistry::new_from_paths(Vec::new()),
);
let mut logs = Vec::new().into();

self.pattern
Expand Down
46 changes: 42 additions & 4 deletions crates/biome_grit_patterns/src/grit_target_language.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@ mod js_target_language;

pub use js_target_language::JsTargetLanguage;

use crate::grit_js_parser::GritJsParser;
use crate::grit_target_node::{GritTargetNode, GritTargetSyntaxKind};
use crate::grit_tree::GritTree;
use biome_rowan::SyntaxKind;
use grit_util::Language;
use grit_util::{Ast, CodeRange, EffectRange, Language, Parser, SnippetTree};
use std::borrow::Cow;

/// Generates the `GritTargetLanguage` enum.
///
Expand All @@ -13,7 +16,7 @@ use grit_util::Language;
/// implement the slightly more convenient [`GritTargetLanguageImpl`] for
/// creating language-specific implementations.
macro_rules! generate_target_language {
($($language:ident),+) => {
($([$language:ident, $parser:ident]),+) => {
#[derive(Clone, Debug)]
pub enum GritTargetLanguage {
$($language($language)),+
Expand All @@ -32,11 +35,26 @@ macro_rules! generate_target_language {
}
}

fn get_parser(&self) -> Box<dyn Parser<Tree = GritTree>> {
match self {
$(Self::$language(_) => Box::new($parser)),+
}
}

fn is_alternative_metavariable_kind(&self, kind: GritTargetSyntaxKind) -> bool {
match self {
$(Self::$language(_) => $language::is_alternative_metavariable_kind(kind)),+
}
}

pub fn parse_snippet_contexts(&self, source: &str) -> Vec<SnippetTree<GritTree>> {
let source = self.substitute_metavariable_prefix(source);
self.snippet_context_strings()
.iter()
.map(|(pre, post)| self.get_parser().parse_snippet(pre, &source, post))
.filter(|result| !result.tree.root_node().kind().is_bogus())
.collect()
}
}

impl Language for GritTargetLanguage {
Expand Down Expand Up @@ -65,12 +83,32 @@ macro_rules! generate_target_language {
|| (self.is_alternative_metavariable_kind(node.kind())
&& self.exact_replaced_variable_regex().is_match(&node.text_trimmed().to_string()))
}

fn align_padding<'a>(
&self,
_node: &Self::Node<'a>,
_range: &CodeRange,
_skip_ranges: &[CodeRange],
_new_padding: Option<usize>,
_offset: usize,
_substitutions: &mut [(EffectRange, String)],
) -> Cow<'a, str> {
todo!()
}

fn pad_snippet<'a>(&self, _snippet: &'a str, _padding: &str) -> Cow<'a, str> {
todo!()
}

fn get_skip_padding_ranges(&self, _node: &Self::Node<'_>) -> Vec<CodeRange> {
Vec::new()
}
}
};
}
}

generate_target_language! {
JsTargetLanguage
[JsTargetLanguage, GritJsParser]
}

/// Trait to be implemented by the language-specific implementations.
Expand Down
24 changes: 17 additions & 7 deletions crates/biome_grit_patterns/src/grit_target_node.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::util::TextRangeGritExt;
use biome_js_syntax::{JsSyntaxKind, JsSyntaxNode, JsSyntaxToken};
use biome_rowan::{SyntaxNodeText, TextRange};
use biome_rowan::{SyntaxKind, SyntaxNodeText, TextRange};
use grit_util::{AstCursor, AstNode as GritAstNode, ByteRange, CodeRange};
use std::{borrow::Cow, str::Utf8Error};

Expand Down Expand Up @@ -73,6 +73,14 @@ macro_rules! generate_target_node {
$(Self::$lang_node(node) => node.text_trimmed_range()),+
}
}

pub fn start_byte(&self) -> u32 {
self.text_trimmed_range().start().into()
}

pub fn end_byte(&self) -> u32 {
self.text_trimmed_range().end().into()
}
}

impl GritAstNode for GritTargetNode {
Expand Down Expand Up @@ -144,12 +152,6 @@ macro_rules! generate_target_node {
}
}

fn full_source(&self) -> &str {
// This should not be a problem anytime soon, though we may want to
// reconsider when we implement rewrites.
unimplemented!("Full source of file not available")
}

fn walk(&self) -> impl AstCursor<Node = Self> {
GritTargetNodeCursor::new(self)
}
Expand Down Expand Up @@ -184,6 +186,14 @@ macro_rules! generate_target_node {
Self::$lang_kind(value)
}
})+

impl GritTargetSyntaxKind {
pub fn is_bogus(&self) -> bool {
match self {
$(Self::$lang_kind(kind) => kind.is_bogus()),+
}
}
}
};
}

Expand Down
Loading