Skip to content

Commit

Permalink
docs changes after moving code around
Browse files Browse the repository at this point in the history
separate commit for review purposes
  • Loading branch information
findepi committed Jul 17, 2024
1 parent bd1c50d commit 6c399c2
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 20 deletions.
12 changes: 4 additions & 8 deletions datafusion/catalog/src/catalog.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,10 @@ use datafusion_common::Result;
///
/// To implement a catalog, you implement at least one of the [`CatalogProviderList`],
/// [`CatalogProvider`] and [`SchemaProvider`] traits and register them
/// appropriately the [`SessionContext`].
///
/// [`SessionContext`]: crate::execution::context::SessionContext
/// appropriately in the `SessionContext`.
///
/// DataFusion comes with a simple in-memory catalog implementation,
/// [`MemoryCatalogProvider`], that is used by default and has no persistence.
/// `MemoryCatalogProvider`, that is used by default and has no persistence.
/// DataFusion does not include more complex Catalog implementations because
/// catalog management is a key design choice for most data systems, and thus
/// it is unlikely that any general-purpose catalog implementation will work
Expand Down Expand Up @@ -79,12 +77,10 @@ use datafusion_common::Result;
/// access required to read table details (e.g. statistics).
///
/// The pattern that DataFusion itself uses to plan SQL queries is to walk over
/// the query to [find all table references],
/// the query to find all table references,
/// performing required remote catalog in parallel, and then plans the query
/// using that snapshot.
///
/// [find all table references]: resolve_table_references
///
/// # Example Catalog Implementations
///
/// Here are some examples of how to implement custom catalogs:
Expand Down Expand Up @@ -135,7 +131,7 @@ pub trait CatalogProvider: Sync + Send {

/// Removes a schema from this catalog. Implementations of this method should return
/// errors if the schema exists but cannot be dropped. For example, in DataFusion's
/// default in-memory catalog, [`MemoryCatalogProvider`], a non-empty schema
/// default in-memory catalog, `MemoryCatalogProvider`, a non-empty schema
/// will only be successfully dropped when `cascade` is true.
/// This is equivalent to how DROP SCHEMA works in PostgreSQL.
///
Expand Down
12 changes: 2 additions & 10 deletions datafusion/catalog/src/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,7 @@ pub trait CatalogSession: Send + Sync {
///
/// This function will error for [`LogicalPlan`]s such as catalog DDL like
/// `CREATE TABLE`, which do not have corresponding physical plans and must
/// be handled by another layer, typically [`SessionContext`].
///
/// [`SessionContext`]: crate::execution::context::SessionContext
/// be handled by another layer, typically the `SessionContext`.
async fn create_physical_plan(
&self,
logical_plan: &LogicalPlan,
Expand All @@ -58,17 +56,11 @@ pub trait CatalogSession: Send + Sync {
/// Create a [`PhysicalExpr`] from an [`Expr`] after applying type
/// coercion, and function rewrites.
///
/// Note: The expression is not [simplified] or otherwise optimized: `a = 1
/// Note: The expression is not simplified or otherwise optimized: `a = 1
/// + 2` will not be simplified to `a = 3` as this is a more involved process.
/// See the [expr_api] example for how to simplify expressions.
///
/// # See Also:
/// * [`SessionContext::create_physical_expr`] for a higher-level API
/// * [`create_physical_expr`] for a lower-level API
///
/// [simplified]: datafusion_optimizer::simplify_expressions
/// [expr_api]: https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/expr_api.rs
/// [`SessionContext::create_physical_expr`]: crate::execution::context::SessionContext::create_physical_expr
fn create_physical_expr(
&self,
expr: Expr,
Expand Down
2 changes: 1 addition & 1 deletion datafusion/catalog/src/table.rs
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ pub trait TableProvider: Sync + Send {
/// See [`DataSinkExec`] for the common pattern of inserting a
/// streams of `RecordBatch`es as files to an ObjectStore.
///
/// [`DataSinkExec`]: crate::physical_plan::insert::DataSinkExec
/// [`DataSinkExec`]: datafusion_physical_plan::insert::DataSinkExec
async fn insert_into(
&self,
_state: &dyn CatalogSession,
Expand Down
2 changes: 1 addition & 1 deletion datafusion/core/src/datasource/file_format/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ pub trait FileFormatFactory: Sync + Send + GetExt {
/// from the [`TableProvider`]. This helps code re-utilization across
/// providers that support the same file formats.
///
/// [`TableProvider`]: crate::datasource::provider::TableProvider
/// [`TableProvider`]: crate::catalog_api::TableProvider
#[async_trait]
pub trait FileFormat: Send + Sync + fmt::Debug {
/// Returns the table provider as [`Any`](std::any::Any) so that it can be
Expand Down

0 comments on commit 6c399c2

Please sign in to comment.