[FEA/Discussion]: allow specification of default engine via config options

[wence-]

Description

When collecting a lazyframe, one can specify an engine= parameter to switch between cpu (in memory) and gpu. Perhaps once new-streaming is ready for primetime that will be an option as well.

Some feedback from gpu-engine users suggests that it might be useful to allow users to select (via config options) a default engine. Since the gpu engine allows passing an object to select the engine (to provide extra configuration options), one would only allow selecting an engine with "defaults", since the Config object goes through os.environ and therefore can only store strings. I think that is probably fine.

Any opinion on whether this sounds reasonable, and if so how to spell it?

[quasiben]

Similar to pl.Config.set_ascii_tables(True), maybe it would be good to start with the simplest config like pl.Config.set_gpu_engine(True) . There are many options which cuDF is still developing:

    device=0,  # This is the default
    raise_on_fail=True,  # Fail loudly if can't execute on the GPU
    parquet_options={
        'chunked': True,
        'chunk_read_limit': int(1e9),
        'pass_read_limit': int(4e9)
    },
    memory_resource=mr

Rather than expose all of this into some global config, an easy on/off button may work nicely here.

[Matt711]

I'm working on this in tandem with #20039