Golden Liquid

A test suite for Liquid, the safe, customer-facing template language for flexible web apps.

Intended for situations where template authors are untrusted and, perhaps, not software developers, Liquid is deliberately both restrictive and forgiving. It is restrictive in that Liquid expressions are somewhat limited compared to those found in other template languages, and forgiving in the way it automatically coerces data types and handles undefined variables.

The tests defined in golden_liquid.json attempt to cover many of Liquid's limitations, as well as the behavior of all standard tags and filters.

• Standard Liquid
• Test File Schema
• Results Summary
• More Information

Standard Liquid

For our purposes, "standard" Liquid is the one described here, with Shopify/Liquid being the reference implementation. Not to be confused with the extended variation of Liquid that is used for Shopify stores.

All tests pass with Liquid version 5.6.0.alpha and Ruby 3. Some round filter test cases fail with Ruby 2.7 due to some changes with Ruby's BigDecimal library (see issue #1590). If you have Ruby installed, you can run the test suite against the reference implementation by cloning this repository and running the following commands.

cd liquid
bundle install
bundle exec ruby golden_liquid.rb

Test File Schema

In golden_liquid.json, tests are grouped. Each group has a name and an array of test cases. Including a version number, golden_liquid.json looks like this.

{
    "version": "0.23.0",
    "test_groups": [
        {
            "name": "liquid.golden.abs_filter",
            "tests": [
                {
                    "name": "negative float",
                    "template": "{{ -5.4 | abs }}",
                    "want": "5.4",
                    "context": {},
                    "partials": {},
                    "error": false,
                    "strict": false
                },
                .
                .
            ]
        },
        .
        .
    ]
}

golden_liquid.yaml is the same test suite in YAML format.

version: 0.23.0
test_groups:
  - name: liquid.golden.abs_filter
    tests:
      - name: negative float
        template: "{{ -5.4 | abs }}"
        want: "5.4"
        context: {}
        partials: {}
        error: false
        strict: false

For each test case:

name - A descriptive name for the test case. Together with the group name, it should uniquely identify the test case.

template - The Liquid template source text as a string.

want - The expected result of rendering the template with the associated context.

context - A JSON object mapping strings to arbitrary, possibly nested, strings, numbers, arrays, objects and booleans. These are the variables that the associated template should be rendered with.

partials - A JSON object mapping strings to strings. You can think of it as a mock file system for testing {% include %} and {% render %}.

error - A boolean indicating if the test case should raise/throw an exception/error.

strict A boolean indicating if the test should be rendered in "strict mode", if the target environment has a strict mode.

Results Summary

This table summarizes the results of running version 0.23.0 of this test suit against the five Liquid engines with runners included in this repository.

Engine	Version	Passed	Failed
Shopify/Liquid	5.6.0.alpha	888	0
LiquidJS**	10.16.3	647	241
liquidpy	0.8.2	467	421
LiquidScript	1.8.2	876	12
Python Liquid	1.12.2	888	0

** It's worth noting that many, but not all, of the failed test cases for LiquidJS are due to the way it handles excess and/or unexpected filter arguments, and its lack of distinct float and int types.

More Information

golden_liquid.json and golden_liquid.yaml are currently generated from these files in the Python Liquid repository. The plan is to move these source files (or some equivalent files) to this repository, so we might add test cases for behavior that Python Liquid chooses not to implement.

In the mean time, be sure to keep an eye on Python Liquid's known issues page and issue tracker.