datetime.datetime is not recognized as the equivalent python type to types.TimestampType()

[harupy]

Actually, this isn't an issue. I was looking for a workaround for this comment, wrote the code below and got ValueError.

import databricks.koalas as ks
from datetime import datetime, timedelta

def tomorrow(date) -> datetime:
    return date + timedelta(days=1)

test_df = ks.DataFrame([(datetime(2019, 10, 24))], columns=['date'])
test_df['date'] = test_df['date'].map(tomorrow)

>>> test_df
...
ValueError: Invalid returnType: returnType can not be None

The cause is that datetime.datetime is not recognized as the equivalent python type to types.TimestampType().

_base = {
    types.StringType(): [str, 'str', 'string'],
    types.BinaryType(): [bytes],
    types.ByteType(): [np.int8, 'int8', 'byte'],
    types.ShortType(): [np.int16, 'int16', 'short'],
    types.IntegerType(): [int, 'int', np.int, np.int32],
    types.LongType(): [np.int64, 'int64', 'long', 'bigint'],
    types.FloatType(): [float, 'float', np.float],
    types.DoubleType(): [np.float64, 'float64', 'double'],

    types.TimestampType(): [np.datetime64],  # <- should be included here.

    types.DateType(): [datetime.date],
    types.BooleanType(): [bool, 'boolean', 'bool', np.bool],
    types.ArrayType(types.StringType()): []
}

https://github.com/databricks/koalas/blob/master/databricks/koalas/typedef.py#L112

I think datetime.datetime should be included in types.TimestampType().

@ueshin @HyukjinKwon What do you think?

[HyukjinKwon]

Oh, yes. looks like we should add. It works in pandas right?

[harupy]

Let me check if this code works when using pandas instead of koalas.

[harupy]

It worked with pandas. I'll create a PR.