Make the temporary directory (for spills) configurable through configuration API

[2010YOUY01]

Is your feature request related to a problem or challenge?

For external executions, the query might write temporary data to disk, and clean them up before the query finished.

Now the temporary path is only configurable through raw Rust code, it would be easier to set if we also support that through config API:

SET datafusion.runtime.temp_dir = '/ssd/temp';
select * from very_large_table order by c1;

Here is a initial PR to set runtime config through configuration API: #15594

DuckDB's equivalent configuration (see temp_directory): https://duckdb.org/docs/stable/configuration/overview.html

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

[delamarch3]

take