Register SQL planners in SessionState::new()

[jayzhan211]

          It might make sense to move the registering of these planners into the constructor of SessionState rather than here (so users could potentially more carefully control the planner content). But we can do that as a follow on PR

Originally posted by @alamb in #11208 (comment)

[dharanad]

hello @jayzhan211 , i would like to work on this. QQ: Is this issue any straight as it sounds or there is more to it?

[jayzhan211]

hello @jayzhan211 , i would like to work on this. QQ: Is this issue any straight as it sounds or there is more to it?

A little more than it. Ideally we need to update map capacity in provider with the number of references. But, I think you can try it.

datafusion/datafusion/core/src/execution/session_state.rs

Lines 590 to 608 in 75b9c9b

	let references = self.resolve_table_references(&statement)?;
	let mut provider = SessionContextProvider {
	state: self,
	tables: HashMap::with_capacity(references.len()),
	};
	for reference in references {
	let resolved = &self.resolve_table_ref(reference);
	if let Entry::Vacant(v) = provider.tables.entry(resolved.to_string()) {
	if let Ok(schema) = self.schema_for_ref(resolved.clone()) {
	if let Some(table) = schema.table(&resolved.table).await? {
	v.insert(provider_as_source(table));
	}
	}
	}
	}
	let query = self.build_sql_query_planner(&provider);

[dharanad]

take

[dharanad]

hello @jayzhan211 , i would like to work on this. QQ: Is this issue any straight as it sounds or there is more to it?

A little more than it. Ideally we need to update map capacity in provider with the number of references. But, I think you can try it.

datafusion/datafusion/core/src/execution/session_state.rs

Lines 590 to 608 in 75b9c9b

	let references = self.resolve_table_references(&statement)?;
	let mut provider = SessionContextProvider {
	state: self,
	tables: HashMap::with_capacity(references.len()),
	};
	for reference in references {
	let resolved = &self.resolve_table_ref(reference);
	if let Entry::Vacant(v) = provider.tables.entry(resolved.to_string()) {
	if let Ok(schema) = self.schema_for_ref(resolved.clone()) {
	if let Some(table) = schema.table(&resolved.table).await? {
	v.insert(provider_as_source(table));
	}
	}
	}
	}
	let query = self.build_sql_query_planner(&provider);

Do you mean something like this

        let mut provider = SessionContextProvider {
            state: self,
            tables: HashMap::with_capacity(references.len() + self.user_defined_sql_planners.len()),
        };

If yes, why is required ? I this that we are already updating the capacity with reference count. If i misinterpreted it can you please elaborate more ?

[alamb]

Update here is I think the current formulation is about as optimal as we can get it:

datafusion/datafusion/core/src/execution/session_state.rs

Lines 234 to 242 in dce77db

	let user_defined_sql_planners: Vec<Arc<dyn UserDefinedSQLPlanner>> = vec![
	Arc::new(functions::core::planner::CoreFunctionPlanner::default()),
	// register crate of array expressions (if enabled)
	#[cfg(feature = "array_expressions")]
	Arc::new(functions_array::planner::ArrayFunctionPlanner),
	#[cfg(feature = "array_expressions")]
	Arc::new(functions_array::planner::FieldAccessPlanner),
	#[cfg(feature = "datetime_expressions")]
	Arc::new(functions::datetime::planner::ExtractPlanner),

(namely to make a Vec with a size known at compile time