Koldstart pins dill==0.3.5.11 which has compatibility issues with Python 3.11

[kinghuang]

Describe the bug

I'm not sure where to report issues with koldstart.

I'm using dbt-fal==1.4.1 on Python 3.11. koldstart==0.6.19 brings in dill==0.3.5.11, which lacks a fix for uqfoundation/dill#514. This causes dill to fail when used on Python 3.11.

AttributeError: 'code' object has no attribute 'co_endlinetable'.

The error is encountered when using a Python model with pandarallel, which uses dill to serialize functions.

Your environment

• OS: macOS Ventura 13.2

Core:
  - installed: 1.4.1
  - latest:    1.4.1 - Up to date!

Plugins:
  - fal:              1.4.1 - Up to date!
  - redshift:         1.4.0 - Up to date!
  - postgres:         1.4.1 - Up to date!
  - fal_experimental: 1.4.1 - Could not determine latest version

• Adapter being used: postgres

How to reproduce

Use pandarallel to parallelize a dataframe operation in a Python model.

from pandarallel import pandarallel

def model(dbt, fal):
    pandarallel.initialize(verbose=2)
    input_df = dbt.ref("foundations_wells_names_raw_minimized")
    output_df = input_df.parallel_apply(lambda x: x)
    return output_df

Expected behavior

The dataframe is processed and returned as the model output.

Actual behavior

An AttributeError occurs when pandarallel attempts to serialize the function.

AttributeError: 'code' object has no attribute 'co_endlinetable'.

Screenshots

Additional context

Forcing dill==0.3.6 works.

[chamini2]

Hey, @kinghuang it seems dill did not add official Python 3.11 support until after 0.3.6, so a version with Python 3.11 support has not been released.

That said, I will see if we can get koldstart to use 0.3.6 at least. Thanks for bringing this up!