DEPR: let's deprecate

[jreback]

xref #18202

We have some cruft, let's deprecate it (I have noted some which already have an issue associated).

From ndarray

• Series.compress() (DEPR: Series.compress #21930)
• Series.imag / Series.real (DEPR: Series.put, Series.real, Series.imag, Index.dtype_str #27106)
• Series.nonzero() (Deprecate series.nonzero (GH18262) #24048)
• Series.put() (DEPR: Series.put, Series.real, Series.imag, Index.dtype_str #27106)
• Series.itemsize
• Series.flags
• Series.strides

timeseries specific

• Series.first()
  let head / tail take a timedelta
• Series.last()

other

• Series.swapaxes()
  • No reason once panel is gone.

non-controversial

• MultiIndex.to_hierarchical (only used for Panel) (DEPR: MultiIndex.to_hierarchical #21613)
• Series/DataFrame.compound() (DEPR: deprecate Series/DataFrame.compound #26405)
• Series.ptp() (DEPR: Series.ptp() #21614)
• Series.from_array (Deprecate Series.from_array ? #18213)
• Series.valid() (DEPR: Deprecate Series.valid #18800)
• Series/DataFrame.slice_shift() (DEPR: DataFrame/Series.slice_shift #37601)
• Series/DataFrame.tshift() (DEPR: Deprecate tshift and integrate it to shift #34545)
• Series/DataFrame.get_values() (DEPR: .get_values() #19617)
• Index.dtype_str (DEPR: Series.put, Series.real, Series.imag, Index.dtype_str #27106)
• Index.summary() (Deprecate Index.summary #18217)
• .get_ftype_counts (DEPR: deprecate get_ftype_counts #18243) (DEPR: deprecate get_ftype_counts (GH18243) #20404)
• .get_dtype_counts DEPR: DataFrame.get_dtype_counts #27145
• Index/Series.asobject (Deprecate Series.asobject, Index.asobject, rename to _asobject #18237) (DEPR: deprecate .asobject property #18572)
• Index.to_native_types() (make private) (DEPR: Index.to_native_types #36418)
• DataFrame/Series.as_matrix (DEPR: Deprecate NDFrame.as_matrix #18458)
• .clip_upper/.clip_lower (replace by .clip) (DEPR: Deprecating .clip_lower and .clip_upper #24203)

Potentially

• .ftypes (DEPR: deprecate get_ftype_counts #18243) (Deprecated Series.ftype, Series.ftypes and DataFrame.ftypes features #26744)
• .xs() (CLN: implement xs in terms of loc #6249)
• .iat/.at
• .take
• .lookup (DEPR: DataFrame.lookup #35224)
• DataFrame.from_items
• Series/DataFrame.add_prefix/add_suffix (DEPR: deprecate .add_prefix and .add_suffix #18347)
  • Maybe add suffix / prefix to concat?
• NDFrame.filter

[jreback]

cc @jorisvandenbossche @TomAugspurger if any comments / objections pls note and I will update the top section.

[TomAugspurger]

I think I'm -1 on deprecating xs.

-0 on deprecating ptp

What's the alternative to tshift? I think that's sometimes useful when a shift won't quite work.

[jorisvandenbossche]

I was just writing up a similar issue :-) (but only for Series, as by fixing the api docs and docstrings, I bumped into quite some methods unknown to me)

Overview of methods that could be considered for removal (note that this list is very long, and are methods that I would not miss if they are gone, which does not mean that they are not useful to others, it's just for stirring discussion):

• Related to the original ndarray subclassing:

  • Series.compress()
  • Series.flags
  • Series.imag / Series.real
  • Series.item()
  • Series.itemsize
  • Series.nonzero()
  • Series.put()
  • Series.strides
  • Series.ptp

• Time series specific ones (the question here is if they are all worth it as method, while very specific in application):

  • Series.at_time()
  • Series.first()
  • Series.last()
  • Series.between_time()
  • Series.tshift()

• Finance? specific

  • Series.compound()

• Other:

  • Series.asobject
  • Series.as_matrix
  • Series.between()
  • Series.first_valid_index() / Series.last_valid_index()
  • Series.from_array()
  • Series.slice_shift()
  • Series.swapaxes()
  • Series.truncate()
  • Series.valid()

[jorisvandenbossche]

What's the alternative to tshift? I think that's sometimes useful when a shift won't quite work.

shift already seems to have a freq keyword as well, and it dispatches to tshift if freq is specified

Note that my above list is very long. The more obvious ones to me that are not yet in the list in the top post are: as_matrix (and maybe swapaxes ?)

[topper-123]

Could .add_prefix and .add_suffix be added to the deprecation list?

The dataframe/Series namespace is huge and cutting down can make the API easier to grasp. I would also think it more logical and idiomatic to operate directly on the columns, rather than on the dataframe.

[jreback]

@topper-123 added, feel free to submit PR's for any of these!

[manrajgrover]

@jorisvandenbossche @jreback I would love to submit PRs for this issue. How can I go about in deprecating these APIs?

[jreback]

see for example #18258

[jorisvandenbossche]

@manrajgrover Best first post a comment here with which one you would start doing, as I think not all those listed above are uncontroversial.

[topper-123]

I think .compound, while not very useful ATM, could be more useful if it was cumulative, ie. just use .cumprod instead of .prod and return a series:

>>> s = pd.Series([0.2, 0.2, 0.2])
>>> s.compound()  # essentialy the same as (s+1).cumprod() - 1
0    0.2000
1    0.440
2    0.728
dtype: float64

The above would play excellently together with .pct_change, so data.pct_change().compound() would read really well and be very useful in many use cases.

any opinions if .compound could be changed like above rather than deprecated? If .compound returns a scalar as today, I agree it should be deprecated.

[topper-123]

I've started a PR for add_prefix and add_postfix.

I will take on Series.asobject and NDFrame.as_matrix next, unless @manrajgrover wants to to them, in which case you'll be welcome.

And yes, I like to remove superfluous methods that start with a, as these are so visible then tab-completing in the REPL :-)