| Directive | Description |
|---|---|
| name | +
+
+The name of the benchmark. This will be used as part of the display name used by criterion. Example: + name Q${QUERY_NUMBER_PADDED}+ +The `name` directive also makes the value available to benchmark-file replacements as `BENCH_NAME`. This is separate +from the `BENCH_NAME` environment variable used to select which benchmark group to run. + + |
+
| group | +
+
+The group name of the benchmark used for grouping benchmarks together. Example: + group imdb+ + |
+
| subgroup | +
+
+The sub group name of the benchmark used for filtering to a specific sub group. Example: + subgroup window+ + |
+
| load | +
+
+The load directive called during initialization of the benchmark. If a path to a file is provided on the same
+line as the load directive that path will be parsed and any sql statements in that file will be executed during
+initialization. If no path is specified the next line is required to be the sql statement to execute. The +load directive (including any following sql statement) must be followed by a blank line. Example: + load sql_benchmarks/h2o/init/load_${BENCH_SUBGROUP:-groupby}_${BENCH_SIZE:-small}_${BENCH_FILE_TYPE:-csv}.sql+or + +load+ + |
+
| init | +
+
+The init directive is called after the load directive prior to benchmark execution. If a path to a file is
+provided on the same line as the init directive that path will be parsed and any sql statements in that file will be
+executed during the benchmark initialization. If no path is specified the next line is required to be the sql statement
+to execute. The init directive (including any following sql statement) must be followed by a blank +line. Example: + +init+ + |
+
| run | +
+
+The run directive called during execution of the benchmark. If a path to a file is provided on the same line as
+the run directive that path will be parsed and any sql statements in that file will be executed during the benchmark
+run. If no path is specified the next line is required to be the sql statement to execute. Multiple +statements are allowed within a single run directive, however a benchmark file may contain only one run directive. When +running with `BENCH_PERSIST_RESULTS` or `BENCH_VALIDATE`, only the last `SELECT` or `WITH` statement from that run +directive will be used for comparison. The run directive (including any following sql statement) must be +followed by a blank line. Example: + run sql_benchmarks/imdb/queries/${QUERY_NUMBER_PADDED}.sql+ + |
+
| cleanup | +
+
+The cleanup directive is called after all other directives and can be used to cleanup after the benchmark -
+e.g. to drop tables. If a path to a file is provided on the same line as the cleanup directive that path will be parsed
+and any sql statements in that file will be executed during cleanup. If no path is specified the next line is
+required to be the sql statement to execute. The cleanup directive (including any following sql statement) +must be followed by a blank line. Example: + +cleanup+ + |
+
| expect_plan | +
+
+The expect_plan directive will check the physical plan for the string provided on the same line. This
+can be used to validate that a particular join was used. Example: + expect_plan NestedLoopJoinExec+ + |
+
| assert | +
+
+The assert directive is run between the init and run directives and can be used to validate system state correctness
+prior to running the benchmark sql. The format is
++assert II+ +The number of I's corresponds to the number of columns in the result. The expected results can be either tab delimited +or pipe delimited. + + |
+
| result_query | +
+
+The result_query directive is run during the verify phase and can be used to verify a different set of results than any
+that might come from queries executed from the `run` directive. The format is the same as the `assert` directive
+above. Example: + +result_query III+ + +Note that the results of the run query are not automatically stored into a table in datafusion. If you want to +verify a result from queries executed from the `run` directive those queries will have to be saved to a table directly +using `CREATE TABLE AS (..)` or similar. + + |
+
| result | +
+
+The result directive declares the expected result file used during verification. A path to a file is required on the
+same line as the result directive. The file is parsed only during verification, and must be a pipe-delimited CSV file
+with a header row. During verification, these expected rows are compared with the rows produced by the last saved
+`SELECT` or `WITH` statement from the `run` directive. Example: + +result sql_benchmarks/imdb/results/${QUERY_NUMBER_PADDED}.csv ++ + |
+
| template | +
+
+The template directive allows for inclusion of another file in a benchmark file. A path to a file is
+required on the same line as the template directive which will be parsed as a benchmark file. Parameters can be passed
+to the template file using the format `KEY=value`, one per line after the template directive followed by a blank line.
+ Example: + +template sql_benchmarks/smj/smj.benchmark.template+ + |
+
| include | +The include directive is similar to the template directive except that it does not support parameters. | +
| echo | +
+
+The echo directive allows for echoing a string to stdout during the execution of the benchmark and may be useful for
+debugging. Example: + +echo The value for batch size is ${BATCH_SIZE:-8192} ++ + |
+