docs: document SessionConfig #8771
Conversation
| @@ -306,7 +322,7 @@ config_namespace! { | |||
| pub metadata_size_hint: Option<usize>, default = None | |||
|
|
|||
| /// If true, filter expressions are be applied during the parquet decoding operation to | |||
| /// reduce the number of rows decoded | |||
| /// reduce the number of rows decoded. This optimization is sometimes called "late materialization". | |||
There was a problem hiding this comment.
Adding this because it was the option I was seeking out with the keyword late materialization.
wjones127
force-pushed
the
doc-session-config
branch
from
ebaec3a
to
d1dbe78
Compare
There was a problem hiding this comment.
Thank you @wjones127 -- I think this is a great improvement. I had some small suggestions but I think they could be done as a follow on PR or not at all.
I also rendered this locally and it was very nice 👨🍳 👌
datafusion/execution/src/config.rs
Outdated
| /// Configuration options for Execution context | ||
| /// Configuration options for [`SessionContext`]. | ||
| /// | ||
| /// Can be passed to `SessionContext::with_config` to customize the configuration of DataFusion. |
There was a problem hiding this comment.
| /// Can be passed to `SessionContext::with_config` to customize the configuration of DataFusion. | |
| /// Can be passed to [`SessionContext::with_config`] to customize the configuration of DataFusion. |
There was a problem hiding this comment.
Added a manual link to this as well.
Also, discovered that SessionContext::with_config is deprecated so changed to new_with_config.
| /// | ||
| /// | Namespace | Config struct | | ||
| /// | --------- | ------------- | | ||
| /// | `datafusion.catalog` | [CatalogOptions][datafusion_common::config::CatalogOptions] | |
There was a problem hiding this comment.
Should we also mention add AggregateOptions ParquetOptions and SqlParserOptions?
There was a problem hiding this comment.
SqlParserOptions is already there, but added the other two.
|
Thanks again @wjones127 |
Which issue does this PR close?
Closes #8770.
Rationale for this change
It took me a surprisingly long time to figure out how to set these configuration options.
What changes are included in this PR?
SessionConfigfrom various*Optionsstructs so users can figure out where they are supposed to set them.SessionConfig.options()andoptions_mut()methods to beginning of theimplso that they are near the top of the documentation page. This makes them more visible. Also added simple examples for each.Are these changes tested?
The documentation changes include doc-tested examples.
Are there any user-facing changes?
These are just documentation changes.