You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Both template and expression lexing functions are currently defined in liquid.lex, and parsers for template tags and tag expressions are bundled into liquid.parse. Moreover, all tag expressions are parsed through liquid.parse.ExpressionParser.parse_expression(), which handles liquid identifiers, loops and boolean expressions.
For reasons of easier maintenance and potential improvements in performance, I intend to move and refactor each of the expression lexers into their own package, along with a specialised parser and independent TokenStream (independent from the top-level token stream).
Built-in tags will transition to use these new parsers now, via liquid.Environment.parse_*_expression_value functions. Existing tokenize* functions and the ExpressionParser will be maintained until at least Python Liquid version 2.0, which is quite some time away, for those who use them in custom tags.
Some possible optimisations that can be realised include:
Lexers that yield tuples rather than NamedTuples. Benchmarks show the former to be faster.
Lexers that recognise identifiers with bracketed indexes and string literals. Doing this with regular expressions in the lexer will be much faster than stepping through a token stream in the parser.
Don't do unnecessary infix operator parsing when handling loop or output expressions. They don't have any infix operators.
Don't do unnecessary token precedence look-ups when handling expression that don't have any precedence rules. Only boolean expression use precedence rules.
Remove unnecessary prefix parsing. No built-in tag expression uses prefix operators. Negative numbers can be handled during tokenization.
The text was updated successfully, but these errors were encountered:
Both template and expression lexing functions are currently defined in
liquid.lex
, and parsers for template tags and tag expressions are bundled intoliquid.parse
. Moreover, all tag expressions are parsed throughliquid.parse.ExpressionParser.parse_expression()
, which handles liquid identifiers, loops and boolean expressions.For reasons of easier maintenance and potential improvements in performance, I intend to move and refactor each of the expression lexers into their own package, along with a specialised parser and independent
TokenStream
(independent from the top-level token stream).Built-in tags will transition to use these new parsers now, via
liquid.Environment.parse_*_expression_value
functions. Existingtokenize*
functions and theExpressionParser
will be maintained until at least Python Liquid version 2.0, which is quite some time away, for those who use them in custom tags.Some possible optimisations that can be realised include:
NamedTuple
s. Benchmarks show the former to be faster.The text was updated successfully, but these errors were encountered: