Document how to test examples in user guide, add some more coverage (#…

…11178)

* Document testing examples, add some more coverage

* add docs

datafusion/core/src/lib.rs

@@ -587,8 +587,47 @@ pub mod test_util;
 #[cfg(doctest)]
 doc_comment::doctest!("../../../README.md", readme_example_test);
 
+// Instructions for Documentation Examples
+//
+// The following commands test the examples from the user guide as part of
+// `cargo test --doc`
+//
+// # Adding new tests:
+//
+// Simply add code like this to your .md file and ensure your md file is
+// included in the lists below.
+//
+// ```rust
+// <code here will be tested>
+// ```
+//
+// Note that sometimes it helps to author the doctest as a standalone program
+// first, and then copy it into the user guide.
+//
+// # Debugging Test Failures
+//
+// Unfortunately, the line numbers reported by doctest do not correspond to the
+// line numbers of in the .md files. Thus, if a doctest fails, use the name of
+// the test to find the relevant file in the list below, and then find the
+// example in that file to fix.
+//
+// For example, if `user_guide_expressions(line 123)` fails,
+// go to `docs/source/user-guide/expressions.md` to find the relevant problem.
+
 #[cfg(doctest)]
 doc_comment::doctest!(
     "../../../docs/source/user-guide/example-usage.md",
-    user_guid_example_tests
+    user_guide_example_usage
+);
+
+#[cfg(doctest)]
+doc_comment::doctest!(
+    "../../../docs/source/user-guide/configs.md",
+    user_guide_configs
+);
+
+#[cfg(doctest)]
+doc_comment::doctest!(
+    "../../../docs/source/user-guide/expressions.md",
+    user_guide_expressions
 );

docs/source/contributor-guide/testing.md

@@ -49,6 +49,31 @@ You can run these tests individually using `cargo` as normal command such as
 cargo test -p datafusion --test parquet_exec
 ```
 
+## Documentation Examples
+
+We use Rust [doctest] to verify examples from the documentation are correct and
+up-to-date. These tests are run as part of our CI and you can run them them
+locally with the following command:
+
+```shell
+cargo test --doc
+```
+
+### API Documentation Examples
+
+As with other Rust projects, examples in doc comments in `.rs` files are
+automatically checked to ensure they work and evolve along with the code.
+
+### User Guide Documentation
+
+Rust example code from the user guide (anything marked with \`\`\`rust) is also
+tested in the same way using the [doc_comment] crate. See the end of
+[core/src/lib.rs] for more details.
+
+[doctest]: https://doc.rust-lang.org/rust-by-example/testing/doc_testing.html
+[doc_comment]: https://docs.rs/doc-comment/latest/doc_comment
+[core/src/lib.rs]: https://github.com/apache/datafusion/blob/main/datafusion/core/src/lib.rs#L583
+
 ## Benchmarks
 
 ### Criterion Benchmarks

docs/source/user-guide/expressions.md

@@ -26,8 +26,10 @@ available for creating logical expressions. These are documented below.
 Most functions and methods may receive and return an `Expr`, which can be chained together using a fluent-style API:
 
 ```rust
+use datafusion::prelude::*;
 // create the expression `(a > 6) AND (b < 7)`
-col("a").gt(lit(6)).and(col("b").lt(lit(7)))
+col("a").gt(lit(6)).and(col("b").lt(lit(7)));
+
 ```
 
 :::