Fix state serialization bug with pandas.DataFrame.

[richard-to]

To fix this issue we add custom JSON encoding and decoding logic.

For most accurate serialization/deserialization, we use Pandas's built in JSON serialization (via to_json) with the "table" strategy. This is more verbose, but provides enough metadata to deserialize as close as possible to the original DataFrame.

To deserialize we use pandas.read_json with orient="table".

One difference I've noticed so far is that pandas.NA becomes np.nan.

Ref: #117

[wwwillchen]

I think this change makes sense, my only hesitation is that this adds a hard dependency to pandas.

As discussed in #103, there's a fine balance in Mesop between working well with the broader Python ecosystem without necessarily dragging in a lot of Python dependencies.

If you think there's a lot more work in table which will require direct pandas manipulation (e.g. table sorting or table editing), then it may make sense to simply have a dependency on pandas, which is very popular in the Python ecosystem.

OTOH, if this is the only usage of pandas, you could do something like:

try:
  import pandas as pd
   # do pandas-speciifc dataclass logic
except ImportError:
  # non-pandas dataclass logic

[richard-to]

I think this change makes sense, my only hesitation is that this adds a hard dependency to pandas.

As discussed in #103, there's a fine balance in Mesop between working well with the broader Python ecosystem without necessarily dragging in a lot of Python dependencies.

If you think there's a lot more work in table which will require direct pandas manipulation (e.g. table sorting or table editing), then it may make sense to simply have a dependency on pandas, which is very popular in the Python ecosystem.

OTOH, if this is the only usage of pandas, you could do something like:

try:
  import pandas as pd
   # do pandas-speciifc dataclass logic
except ImportError:
  # non-pandas dataclass logic

That's a great point. About not wanting to include Pandas with the Mesop package. I definitely agree. Updated.