Support pandas>=0.18.1 #443

yankees714 · 2017-09-26T19:40:42Z

For compatibility with zipline.

twiecki

LGTM!

twiecki · 2017-09-26T20:06:16Z

We should probably add this to travis as well.

yankees714 · 2017-09-26T20:47:38Z

@twiecki how would I go about doing that?

twiecki · 2017-09-26T20:51:15Z

See e.g. https://github.com/quantopian/empyrical/blob/master/.travis.yml#L4

yankees714 · 2017-09-26T21:01:41Z

Ah got it, thanks. Do we want all python/pandas combinations?

twiecki · 2017-09-27T07:39:17Z

Yes.

twiecki · 2017-09-27T15:20:42Z

Need to wait for #442 to fix the current breakage.

vikram-narayan · 2017-09-27T15:21:31Z

merging #442 once tests pass

twiecki · 2017-09-27T15:48:57Z

@yankees714 Can you rebase from master?

If DataFrame.columns has a name set, it was previously maintained by DataFrame.join, but no longer is in recent versions of pandas. This commit resets DataFrame.columns.name before returning, so that the same dataframe is returned across pandas versions.

vikram-narayan

LGTM, just had two small questions

vikram-narayan · 2017-09-28T04:37:31Z

.travis.yml

-  - "2.7"
-  - "3.5"
-  - "3.6"
+matrix:


is adding time to the travis build something we're worried about?

Possibly, any thoughts on this @twiecki, @richafrank?

I'm not worried about it.

Each job in the matrix would still take the same amount of time, so we wouldn't suffer any new timeouts. I definitely think we should test an environment that matches what we test for zipline; the others I'm less concerned about. I think we have 4 workers - What's the wall time for a build now?

Is it possible for us to specify the factor, so travis expands the matrix itself?

For the last build on this branch it was 20-30 minutes per job, a bit over an hour total.

Also yeah looks like we should be able to have travis do the expansion, I'll take a look at that

An hour sounds kinda painful to me, but as consumers, what do you all think?

vikram-narayan · 2017-09-28T04:48:07Z

pyfolio/risk.py

+    #       drop all-nan rows before the call to `quantile`, and then
+    #       restore them by reindexing the final result. This is fixed
+    #       in pandas 0.19.
+    longed_threshold = 100*longed_frac.dropna(how='all').quantile(


will this fail flake8 if there aren't spaces around *?

Apparently not, as I was just leaving this as it was. For some reason thought we were consistently not including the whitespace in this project, but now that I look again it seems like we do include it more often that not. I'll update this and the other occurrences in this file.

yankees714 · 2017-09-29T13:38:00Z

@vikram-narayan I realized there was an issue with one of the workarounds, so I had to modify it (a391f8a), mind taking another look?

vikram-narayan

just had one question on that commit, lgtm otherwise

vikram-narayan · 2017-09-29T15:18:26Z

pyfolio/risk.py

+    #       pandas 0.18, use np.nanpercentile by applying to each row of
+    #       the dataframe. This is fixed in pandas 0.19.
+    #
+    # longed_threshold = 100*longed_frac.quantile(percentile, axis='columns')


I was just leaving these as a reference for when we no longer need to support 0.18, does that sound alright?

yup sounds good

vikram-narayan · 2017-09-29T15:56:35Z

actually looks like the travis build failed, is that due to the most recent change?

twiecki · 2017-09-29T16:20:07Z

@vikram-narayan I think conda is still broken.

vikram-narayan · 2017-09-29T19:14:51Z

I think @yankees714 included a workaround for that, it looks like python 3.6 and pandas 0.18.1 are incompatible, from one of the travis builds:

UnsatisfiableError: The following specifications were found to be in conflict:
  - pandas 0.18.1* -> numpy 1.10* -> python 2.7* -> openssl 1.0.1*
  - python 3.6*
Use "conda info <package>" to see the dependencies for each package.

Also updates .travis.yml to build on pandas 0.18.1, 0.19.2, and 0.20.3.

yankees714 · 2017-10-02T16:49:07Z

Only remaining concern here was build time - it currently stands at 45 minutes which seems tolerable to me. Going to merge, we can cut time later if need be.

twiecki · 2017-10-02T18:35:18Z

Awesome! Should probably release a new version then.

joshpayne · 2017-10-02T18:59:38Z

@twiecki should we align to completion of the first round of perf attribution studies/tearsheets?

twiecki · 2017-10-09T20:18:51Z

@joshpayne I think we're fine with what we have now.

yankees714 requested a review from twiecki September 26, 2017 19:40

twiecki approved these changes Sep 26, 2017

View reviewed changes

Andrew Daniels added 2 commits September 27, 2017 11:52

MAINT: Use sub() when subtracting series of dts, to support pandas 18

cc4358c

yankees714 force-pushed the pandas-0.18.1-support branch from 11dca0b to 4910fbb Compare September 27, 2017 15:52

yankees714 requested a review from vikram-narayan September 27, 2017 20:56

vikram-narayan approved these changes Sep 28, 2017

View reviewed changes

vikram-narayan approved these changes Sep 29, 2017

View reviewed changes

Andrew Daniels added 2 commits October 2, 2017 10:30

MAINT: Workound for bug in DataFrame.quantile

0b5a58a

BLD: Reduce pandas req to at least 0.18.1

3cc1e94

Also updates .travis.yml to build on pandas 0.18.1, 0.19.2, and 0.20.3.

yankees714 force-pushed the pandas-0.18.1-support branch from c4ce1e0 to 3cc1e94 Compare October 2, 2017 14:32

yankees714 merged commit d3a7892 into master Oct 2, 2017

yankees714 deleted the pandas-0.18.1-support branch October 2, 2017 16:49

yankees714 mentioned this pull request Oct 2, 2017

pyfolio no longer supports pandas < 0.20 #407

Open

yankees714 mentioned this pull request Oct 10, 2017

REL: 0.8.1 #452

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support pandas>=0.18.1 #443

Support pandas>=0.18.1 #443

yankees714 commented Sep 26, 2017

twiecki left a comment

twiecki commented Sep 26, 2017

yankees714 commented Sep 26, 2017

twiecki commented Sep 26, 2017

yankees714 commented Sep 26, 2017

twiecki commented Sep 27, 2017

twiecki commented Sep 27, 2017

vikram-narayan commented Sep 27, 2017

twiecki commented Sep 27, 2017

vikram-narayan left a comment

vikram-narayan Sep 28, 2017

yankees714 Sep 28, 2017

twiecki Sep 28, 2017

richafrank Sep 28, 2017

yankees714 Sep 28, 2017

yankees714 Sep 28, 2017

richafrank Sep 28, 2017

vikram-narayan Sep 28, 2017

yankees714 Sep 28, 2017

yankees714 commented Sep 29, 2017

vikram-narayan left a comment

vikram-narayan Sep 29, 2017

yankees714 Oct 2, 2017

vikram-narayan Oct 2, 2017

vikram-narayan commented Sep 29, 2017

twiecki commented Sep 29, 2017

vikram-narayan commented Sep 29, 2017

yankees714 commented Oct 2, 2017

twiecki commented Oct 2, 2017

joshpayne commented Oct 2, 2017

twiecki commented Oct 9, 2017

Support pandas>=0.18.1 #443

Support pandas>=0.18.1 #443

Conversation

yankees714 commented Sep 26, 2017

twiecki left a comment

Choose a reason for hiding this comment

twiecki commented Sep 26, 2017

yankees714 commented Sep 26, 2017

twiecki commented Sep 26, 2017

yankees714 commented Sep 26, 2017

twiecki commented Sep 27, 2017

twiecki commented Sep 27, 2017

vikram-narayan commented Sep 27, 2017

twiecki commented Sep 27, 2017

vikram-narayan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yankees714 commented Sep 29, 2017

vikram-narayan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vikram-narayan commented Sep 29, 2017

twiecki commented Sep 29, 2017

vikram-narayan commented Sep 29, 2017

yankees714 commented Oct 2, 2017

twiecki commented Oct 2, 2017

joshpayne commented Oct 2, 2017

twiecki commented Oct 9, 2017